Connecting to NiFi Registry with NiFi Toolkit CLI

In CDP Public Cloud, access from outside the cluster should go through the cluster’s Knox gateway. Knox provides basic authentication to its API topologies, allowing you to authenticate with your CDP workload username and password.

Knox has two ports that can be accessed, 443 and 8443. This procedure shows you how to connect to the 8443 port of the Knox Gateway Server from outside the Flow Management Data Hub cluster (for example, from a local machine). This port is secured by Cloudera Manager AutoTLS. If you want to use the 443 port, you can use this procedure as a template and modify it for your production certs.
  1. Confirm or set your CDP workload username and password.

    To use non-SSO interfaces, each user and machine user must set a workload password. You can manage your workload password from your user profile page under User Management in the CDP Management Console. For instructions, see Setting the workload password.

    You will need to reference your workload username and password in the basic authentication parameters (basicAuthUsername, basicAuthPassword) in Step 4.

  2. Identify the Knox Gateway Server in your Flow Management Data Hub cluster.

    You can find the hostname of the Knox Gateway Server in Cloudera Manager by checking the hostname in Clusters > Knox > Instances.

    Example hostname: flow-management-azure-management0.dataflow.xcu2-8y8x.dev.cldr.work

  3. Transfer the truststore JKS file (cm-auto-global_truststore.jks) from one of your CDP Data Hub nodes to your local machine.
    1. Connect to one of your NiFi nodes.
    2. Locate the truststore file.
      You can find it in the following directory:

      /var/run/cloudera-scm-agent/process/XYZ-nifi-NIFI_NODE/

    3. Get the truststore password with grep PASSWORD proc.json.
    4. Copy the cm-auto-global_truststore.jks file to your local machine.
  4. Create a NiFi Registry CLI properties file referring to:
    • Knox Gateway Server (specifically pointing to port 8443)
    • Truststore JKS file
    • User and workload password using Basic Authentication

    You can use this properties file to store all your settings/default values for the Registry related CLI commands, so you do not need to specify the baseUrl and other arguments for every command.

    Example properties file datahub-registry-8443.properties:
    
    baseUrl=https://flow-management-azure-management0.dataflow.xcu2-8y8x.dev.cldr.work/flow-management-azure/cdp-proxy/nifi-registry-app
    truststore=/Users/john.doe/Downloads/nifi-toolkit-1.13.2/bin/cm-auto-global_truststore.jks
    truststoreType=jks
    truststorePasswd=password
    basicAuthUsername=jdoe156
    basicAuthPassword=password1
  5. Check that the 8443 port is open.

    If the port is not open, configure the security policy in your given cloud provider to allow external traffic to reach port 8443.

  6. Set your CLI session to refer to the NiFi Registry CLI properties file you created.
    john.doe@john bin % ./cli.sh session set nifi.reg.props datahub-registry-8443.properties
  7. List the Registry buckets as a quick test for your connection.
    You get the list of available buckets in the NiFi Registry.
    In this example, the Registry has two buckets, TestProject and OtherProject:
    john.doe@john bin % ./cli.sh registry list-buckets
    
    # Name          Id                                      Description
    - ---------     ------------------------------------    -----------
    1 TestProject   2a5566b5-7380-46b8-9598-328e092e8899    (empty)
    2 OtherProject  175fb557-43a2-4abb-871f-81a354f47bc2    (empty)