Tuning Ozone for faster deletion performance

Learn how to use the Ozone configuration properties to tune Ozone for faster deletion performance under various workload conditions.

For deletion performance optimization and to overcome known issues, Cloudera recommends the following additional configurations based on different workload conditions:

  1. If you are experiencing slowness specifically when deleting deeply nested directory structures or a large number of directories, set the deletion configurations in the OzoneManager (OM) to the following values:
    ozone.thread.number.dir.deletion = 30
    ozone.directory.deleting.service.interval = 30s
  2. When you observe general deletion slowness across the cluster or require faster overall delete throughput, update the following parameters in their respective services:
    • For the OzoneManager (OM) configurations, use the following command:
      ozone.block.deleting.service.interval = 30s
      ozone.key.deleting.limit.per.task = 200000
      ozone.directory.deleting.service.interval = 30s
    • For the StorageContainerManager (SCM) configurations, use the following command:
      hdds.scm.block.deletion.per-interval.max = 2000000
    • For the DataNode (DN) configurations, use the following command:
      hdds.datanode.block.deleting.limit.per.interval = 50000
      ozone.block.deleting.service.workers = 30
      hdds.datanode.block.deleting.service.interval = 30s