Using the Agent Deployer

Learn how you can deploy a MiNiFi application using the Agent Deployer in the Edge Flow Manager (EFM).

  1. Create a new agent class.
    1. Click Monitor on the left menu bar to access the Dashboard page.
    2. Click New Agent Class.

    3. Enter a name in the Agent Class Name text box and click Create.

    The new class is created and is added to the list of agent classes.
  2. Create an agent repository.

    Before deploying an agent, you need to create an agent repository under EFM to use the Deploy Agent functionality. The default binaries base directory is ${EFM_HOME_DIRECTORY}/agent-deployer/binaries. You can customize the path by setting the efm.agent-deployer.binariesRootPath property in the file.

    Ensure your directory structure follows the convention [{agentType}/{osArch}/{agentVersion}]

    Accepted values:
    • agentType: java or cpp
    • osArch: linux*, windows*, macos* (This allows using different binaries for different OS versions.)
    • agent version: Cloudera version of the agent (for example: 1.23.02)
    A few examples:
  3. Generate Deploy Agent CLI command.
    1. Click Monitor on the left menu bar to open the Dashboard with the list of agent classes.
    2. Click the created agent class to display the agent class details.
    3. Click Actions > Deploy Agent Command.

      If the agent repository has been created successfully in the previous step, the Agent Type, Agent Version, and OS drop-downs are pre-populated.

    4. Choose the desired options and click Generate.

      The generated command appears in the same window.

    5. To install the agent, copy and paste the generated CLI command into the host where the agent will be installed.
  4. Set the advanced options for the Deploy Agent CLI command.

    In specific scenarios, the default configuration may not meet your specific requirements. Use advanced configurations to customize parameters.

    To make the advanced options visible, select the Show Advanced Configurations checkbox.

    EFM Base URL
    The full URL of the EFM REST API base. The generated command uses this URL to access EFM from the remote host. If EFM is behind a load balancer or a proxy, you can override this URL.
    Agent ID

    The agent ID is automatically generated, but you can also set a custom value. If a custom value is set, make sure that each command generation has a unique identifier.

    Heartbeat Period

    The agent sends heartbeats periodically using this value.

    Service User
    • On Linux: A user is created with this name and the agent process runs under this user.
    • On Windows: This is not applicable at the moment because the Java agent is not started as a service and the C++ agent is parameterized with the Service User.
    Autoconfigure Security

    If security is enabled in EFM (for example any authentication method is turned on), the agent needs to connect to EFM in a secured manner. If Autoconfigure Security is enabled, EFM generates the necessary certificates for the agent, and the command downloads the certificates through a secured channel, and configures the agent automatically.

    You can provide your own Certificate Authority (CA) for signing certificates. If a custom CA is not provided, EFM generates one during startup, which will be used for the certificates.

    For more information on Autoconfigure Security features, see Configuring Agent Deployer for securing agents.

    Self-Signed Certificates and CA Cert PEM File Location

    If EFM is set up using a self-signed certificate (where EFM generated the CA and it was not provided externally), the request issued by the Agent Download command may be rejected since the issuing host will not trust EFM.

    You have two options:
    • Select the Trust Self-Signed Certificates checkbox to trust the self-signed certificate and proceed with the Agent Download command.
    • Use the CA Cert PEM File Location to reference a CA Cert on the agent file system. This allows the agent to trust EFM by using the specified CA certificate.
    Dynamic properties:
    You have the flexibility to override any arbitrary MiNiFi properties.
    • For MiNiFi Java agents: The properties are located in the bootstrap.conf file.
    • For MiNiFi C++ agents: The properties are located in the file.
  5. Run the command.
    Copy and paste the generated CLI command on the target host’s shell or command line.

    The command downloads the agent binary into the directory where the script was run. After that, the command extracts, configures, and runs the agent in the background.

    For Java agents on Linux, you should see a log message similar to the following.

    user@user-host /opt/cloudera/minifi curl -L \
     -d agentClass=new-agent-class \
     -d agentIdentifier=061c6cf6-1922-41b7-8239-ad25a0f5be9a \
     -d agentType=java \
     -d agentVersion=1.23.04 \
     -d autoConfigureSecurity=false \
     -d \
     -d hbPeriod=5000 \
     -d osArch=MacOs \
     -d serviceName=minifi \
     -d serviceUser=minifi \
     -d trustSelfSignedCertificates=false \ | bash -
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
    100 17505  100 17208  100   297  18755    323 --:--:-- --:--:-- --:--:-- 19152
     _______ __ _______ __ _______ __      __             __         __ __
    |   |   |__|    |  |__|    ___|__|    |__.-----.-----|  |_.---.-|  |  .-----.----.
    |       |  |       |  |    ___|  |    |  |     |__ --|   _|  _  |  |  |  -__|   _|
    |__|_|__|__|__|____|__|___|   |__|    |__|__|__|_____|____|___._|__|__|_____|__|
    -- Verifying if the following commands exist: tar gzip cat grep sed seq mkdir sleep
    -- Verifying if java is installed on the system...
    -- Found java at: /Users/fkis/.sdkman/candidates/java/current/bin/java
    openjdk version "1.8.0_362"
    OpenJDK Runtime Environment (Zulu (build 1.8.0_362-b08)
    OpenJDK 64-Bit Server VM (Zulu (build 25.362-b08, mixed mode)
    -- Verifying if the system is capable of handling tar.gz archives...
    -- Looking for a download utility on the system...
    -- > curl
    -- curl version: 7.88.1
    -- Downloading MiNiFi...
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
    100  234M  100  234M    0     0  1701k      0  0:02:21  0:02:21 --:--:-- 1123k
    -- MiNiFi has been downloaded to directory: minifi-1.23.04-b15
    -- Configuring MiNiFi...
    -- Starting MiNiFi as a simple background process...
    -- Waiting until MiNiFi is up...
    -- OK
    #Mon Jul 24 15:40:29 CEST 2023
      MiNiFi is now started as a background process.
      You can stop it by issuing the following commands:
        %> cd "minifi-1.23.04-b15"
        %> bin/ stop
      To start again:
        %> cd "minifi-1.23.04-b15"
        %> bin/ start
    -- Installation has successfully completed.
      In addition of the existing (default) configuration values, the followings have been applied:
      If you would like to modify this configuration, you need to perform these steps:
        1) Stop MiNiFi
        2) Edit the files located in "minifi-1.23.04-b15/conf"
        3) Start MiNiFi

    After successful installation, the agent sends heartbeats, appearing as an active agent in EFM.