Cloudera Docs

Configure Knox auto-discovery for large clusters

How to configure Knox’s service auto-discovery feature for large clusters.

By default, the auto-discovery feature of Cloudera Manager through Knox uses the API v32 endpoint:

GET /clusters/{clusterName}/services/{serviceName}/roles/{roleName}/config

This method fetches the role configuration parameters for each service role. This causes a large number of REST API calls in case of large clusters.

For large clusters, use the following endpoint:

GET /clusters/{clusterName}/services/{serviceName}/roles/configs

If the endpoint is available in your Cloudera Manager, follow these steps to configure the auto-discovery method in Knox:

  1. From Cloudera Manager > Knox > Configuration, add a new entry in Knox Gateway Advanced Configuration Snippet (Safety Valve) for conf/gateway-site.xml, and insert these properties: 
    gateway.cloudera.manager.service.discovery.role.fetch.strategy=byService
gateway.cloudera.manager.service.discovery.api.version=v57
  2. Save the changes.
  3. Refresh the Knox instances’ configuration: the Refresh needed stale configuration indicator appears; click it and wait until the refresh process completes.
  4. Validate using the Knox homepage.
  5. The default auto-discovery method can be reset by one of the following methods:
    • Remove the properties from Knox Gateway Advanced Configuration Snippet (Safety Valve) for conf/gateway-site.xml
    • Insert these properties into Knox Gateway Advanced Configuration Snippet (Safety Valve) for conf/gateway-site.xml:
      gateway.cloudera.manager.service.discovery.role.fetch.strategy=byRole
gateway.cloudera.manager.service.discovery.api.version=v32