Cloudera Docs

Configuring the Hue Query Processor scan frequency

You can configure the frequency at which the Hue Query Processor fetches the data from a Hadoop Compatible File System by setting the time in milliseconds (ms) in the “hue.query-processor.event-pipeline.folder-scan-delay-millis” property in the Database Catalog configurations.

By default, the value of the “hue.query-processor.event-pipeline.folder-scan-delay-millis” property is 5 * 60 * 1000 ms (5 minutes). For a faster data refresh rate, you can set this value to less than 5 * 60 * 1000 ms (5 minutes). But this can result in higher storage read costs. However, if you want stronger performance, and a higher refresh rate is not a priority, then you can set it to a higher value to reduce storage read costs.

  1. Sign in to the Cloudera Data Warehouse service.
  2. Go to Database Catalog > Edit > CONFIGURATIONS > Hue query processor and select hue-query-processor.json from the Configuration files drop-down menu.
  3. Change the value of the “hue.query-processor.event-pipeline.folder-scan-delay-millis” property under the “dasConf” section.
  4. Click Apply Changes.