Cloudera Docs

Configuring the Hue Query Processor scan frequency

You can configure the frequency at which the Hue Query Processor fetches the data from a Hadoop Compatible File System by setting the time in milliseconds (ms) in the “hue.query-processor.event-pipeline.folder-scan-delay-millis” property in the Query Processor configurations.

By default, the value of the hue.query-processor.event-pipeline.folder-scan-delay-millis property is 5 * 60 * 1000 ms (5 minutes). For a faster data refresh rate, you can set this value to less than 5 * 60 * 1000 ms (5 minutes). But this can result in higher storage read costs. However, if you want stronger performance, and a faster refresh rate is not a priority, then you can set it to a higher value to reduce storage read costs.
  1. Log in to Cloudera Manager as an Administrator.
  2. Go to Clusters > Query Processor service > Configuration and add the following line in the Query Processor Extra Configurations field: 
    "hue.query-processor.event-pipeline.folder-scan-delay-millis" : [***TIME-IN-MS***]
    "hue.query-processor.event-pipeline.folder-scan-delay-millis" : 2000
  3. Click Save Changes.
  4. Restart the Query Processor service.