After creating your cluster

The cluster you have created using the Edge Flow Management cluster definition is secured by default, and it is integrated with Knox SSO.

You can access the EFM UI from the Services section of the Data Hub cluster page. Click the CEM icon or the Edge Flow Manager UI link and you are redirected to the EFM page.

The user who creates the Data Hub cluster is automatically designated as an administrator in EFM and gains immediate access to the UI.

EFM now integrates with CDP User Management and synchronizes all available users and groups. Besides the cluster creator user, all users who belong to the admin group also gain administrator rights, providing access to all EFM features. Users who do not belong to the admin group can still log in, but need access rights granted by an administrator before they can access the data in EFM.

For secure the agent-to-EFM communication, generate and use appropriate certificates. You also need to add the agents that you want to manage with EFM.

Generating certificates for MiNiFi agents

To secure the communication between agents and EFM, you need to generate and use proper certificates.

Edge Flow Manager (EFM) is a secured application, which has to be bootstrapped with the initial admin identity. The initial admin is the person who is able to assign roles and manage permissions in EFM. In the Technical Preview, the initial admin is the workload user of the person who deploys the Data Hub. For more information about authentication and authorization, see Access control bootstrapping.

While the user traffic accessing the UI utilizes Knox, the agents running outside of the CDP deployment need to access EFM directly. To enable this, you have to open a port for the agents on the host where EFM is deployed. By default, this port is 10090, used by CEM components for C2 Protocol.

You do not have to generate the certificates from the agent host. You can generate them on any host that has access to the management node. When created, you can copy the certificates to the appropriate agent host.

In test environments it is not necessary to create different certificates for all agents. The same certificate can be configured for all agents. However, in production environments it is highly recommended to create a certificate for each agent.

Generating certificates with this approach is similar to adding a node to the cluster using Cloudera Manager.

MiNiFi agents need to set up mTLS (mutual TLS) for C2 communication to be able to communicate with EFM. For information on MiNiFi Java agent authentication, see Securing MiNiFi Java Agent. For information on MiNiFi C++ agent authentication, see Securing MiNiFi C++ Agent.

In CDP Public Cloud, certificates are managed by Cloudera Manager, acting as a certificate authority. All certificates are generated by Cloudera Manager, there is no option to use custom certificates.

  • You have a running CEM Public Cloud cluster
  • SSH access is configured to the management node of the cluster
  • You have an SSH user with keypair that has sudo privileges
  • You have the host name of the Edge Management cluster’s management node
  • An external node is available from which you are able to SSH into the Edge Management cluster’s management node
  1. Create a working directory on your external node that has SSH access to your Edge Flow Management cluster.
  2. Save the following script to the previously created working directory, and name it
    set -eo pipefail
    # input parameters
    EXAMPLE_USAGE="Example usage: ./ sshUserName ~/.ssh/userKey.pem"
    [[ -z "$SSH_USER" ]] && echo "SSH User parameter is missing. $EXAMPLE_USAGE" && exit 1
    [[ -z "$SSH_KEY" ]] && echo "SSH Key parameter is missing. $EXAMPLE_USAGE" && exit 1
    [[ -z "$CM_HOST" ]] && echo "Cloudera Manager parameter is missing. $EXAMPLE_USAGE" && exit 1
    [[ -z "$AGENT_FQDN" ]] && echo "Agent FQDN parameter is missing. $EXAMPLE_USAGE" && exit 1
    KEYSTORE_PASSWORD=$(hexdump -vn16 -e'4/4 "%08X" 1 "\n"' /dev/urandom | tr '[:upper:]' '[:lower:]')
    # constants
    rm -rf "$AGENT_FQDN"
    mkdir "$AGENT_FQDN" 
    remote_ssh_command=$(cat << EOF
    sudo /opt/rh/rh-python38/root/bin/python -c "import site; site.addsitedir('$CM_SITE_PACKAGES'); import; passwd ='$GLOBAL_TRUSTSTORE_PASSWORD_FILE'); print(passwd);"
    sudo /opt/rh/rh-python38/root/bin/python -c "import site; site.addsitedir('$CM_SITE_PACKAGES'); import;'$GLOBAL_KEY_PASSWORD_FILE', '$KEYSTORE_PASSWORD');";
    sudo /opt/cloudera/cm-agent/bin/certmanager --location "$CUSTOM_CERTMANAGER_BASE_DIR" gen_node_cert --output "$GENERATED_CREDENTIALS_REMOTE_PATH" --rotate "$AGENT_FQDN";
    ssh -i "$SSH_KEY" -o StrictHostKeyChecking=no "$SSH_USER"@"$CM_HOST" "$remote_ssh_command" > "$AGENT_FQDN/" 2> /dev/null
    scp -r -i "$SSH_KEY" -o "StrictHostKeyChecking=no" "$SSH_USER"@"$CM_HOST":"$GENERATED_CREDENTIALS_REMOTE_PATH" "$AGENT_FQDN/" 2> /dev/null
    echo "MiNiFi-Java KeyStore File":
    ls -alh "$AGENT_FQDN/cm-auto-host_keystore.jks"
    echo "MiNiFi-Java TrustStore File:"
    ls -alh "$AGENT_FQDN/cm-auto-in_cluster_truststore.jks"
    echo "MiNiFi-CPP Client certificate":
    ls -alh "$AGENT_FQDN/cm-auto-host_key_cert_chain.pem"
    echo "MiNiFi-CPP Client private key":
    ls -alh "$AGENT_FQDN/cm-auto-host_key.pem"
    echo "MiNiFi-CPP CA certificate"
    ls -alh "$AGENT_FQDN/cm-auto-in_cluster_ca_cert.pem"
    echo "KeyStore / HostKey Password: sensitive data, please check for it in $AGENT_FQDN/"
    echo "TrustStore Password: sensitive data, please check for it in $AGENT_FQDN/"
    rm -f "$AGENT_FQDN/cm-auto-global_cacerts.pem" "$AGENT_FQDN/cm-auto-global_truststore.jks" "$AGENT_FQDN/$GENERATED_CREDENTIALS_ARCHIVE" "$AGENT_FQDN/cm-auto-host_cert_chain.pem"
  3. Make the script executable.
    chmod +x
  4. Run the script with the following parameters:
    ./ **[ssh_user}** **[ssh_private_key]** **[management_node_host_name]** **[agent_fqdn]**

    For example:

    ./ adminuser ~/.ssh/adminuser.pem

    The script should print a similar output:

    credentials.tar                                                                                                                                                                                          100%  420KB 222.0KB/s   00:01
    MiNiFi-Java KeyStore File:
    -rw-------@ 1 user  group   5.2K Apr 24 13:33
    MiNiFi-Java TrustStore File:
    -rw-r-----@ 1 user  group   2.3K Apr 24 13:19
    MiNiFi-CPP Client certificate:
    -rw-------@ 1 user  group   7.1K Apr 24 13:33
    MiNiFi-CPP Client private key:
    -rw-------@ 1 user  group   2.5K Apr 24 13:33
    MiNiFi-CPP CA certificate
    -rw-r-----@ 1 user  group   3.0K Apr 24 13:19
    KeyStore / HostKey Password: sensitive data, please check for it in
    TrustStore Password: sensitive data, please check for it in

    A directory is created with the same name as the agent’s FQDN, provided as a parameter for the script. The directory contains all the necessary keystores and certificates for configuring mTLS authentication.

    The keystore and truststore passwords are not printed as they are sensitive information. You can find them in the directory that was created with the following names:
  5. Set the agent parameters.
    • For the MiNiFi Java agent:<><>
    • For the MiNiFi C++ agent:

Adding agents to your cluster

When your cluster has been created successfully, you can add agents that you want to manage with EFM. Agents are deployed outside of CDP Public Cloud, so follow the standard agent deployment instructions: