Configuring the Hue Query Processor scan frequency

You can configure the frequency at which the Hue Query Processor fetches the data from a Hadoop Compatible File System by setting the time in milliseconds (ms) in the “hue.query-processor.event-pipeline.folder-scan-delay-millis” property in the Database Catalog configurations.

By default, the value of the “hue.query-processor.event-pipeline.folder-scan-delay-millis” property is 2000 ms. For a faster data refresh rate, you can set this value to less than 2000 ms. But this can result in higher storage read costs. However, if you want stronger performance, and a higher refresh rate is not a priority, then you can set it to a higher value to reduce storage read costs.

  1. Sign in to the Cloudera Data Warehouse service.
  2. Go to Database Catalog > Edit > CONFIGURATIONS > Hue query processor and select hue-query-processor.json from the Configuration files drop-down menu.
  3. Change the value of the “hue.query-processor.event-pipeline.folder-scan-delay-millis” property under the “dasConf” section.
  4. Click Apply.