Cloudera Docs

Configuring the Hue Query Processor scan frequency

You can configure the frequency at which the Hue Query Processor fetches the data from a Hadoop Compatible File System by setting the time in milliseconds (ms) in the “hue.query-processor.event-pipeline.folder-scan-delay-millis” property in the Query Processor configurations.

By default, the value of the hue.query-processor.event-pipeline.folder-scan-delay-millis property is 5 * 60 * 1000 ms (5 minutes). For a faster data refresh rate, you can set this value to less than 5 * 60 * 1000 ms (5 minutes). But this can result in higher storage read costs. However, if you want stronger performance, and a faster refresh rate is not a priority, then you can set it to a higher value to reduce storage read costs.
  1. Log in to the CDP web interface as an Administrator.
  2. Go to Data Hub Clusters and select your cluster.
  3. Click CM URL or CM-UI to open Cloudera Manager.
  4. Go to Clusters > Query Processor service > Configuration and add the following line in the Query Processor Extra Configurations field: 
    "hue.query-processor.event-pipeline.folder-scan-delay-millis" : [***TIME-IN-MS***]
    "hue.query-processor.event-pipeline.folder-scan-delay-millis" : 2000
  5. Click Save Changes.
  6. Restart the Query Processor service.