Using the Agent Deployer

Learn how you can deploy a MiNiFi application using the Agent Deployer in the Edge Flow Manager (EFM). With the help of a generated one-liner command, you can download and autoconfigure agents directly.

Create a new agent class.
1. Click Monitor on the left menu bar to access the Dashboard page.
2. Click New Agent Class.
3. Enter a name in the Agent Class Name text box and click Create.
The new class is created and is added to the list of agent classes.
note

If you already have an existing agent class, you may skip this step. Proceed with caution, when adding a new agent to an existing class. It is recommended to only add agents with the same version as the ones in the existing agent class to ensure the compatibility of agent manifests. If you are uncertain about the versions, start with a new agent class.
Create an agent repository.
Before deploying an agent, you need to create an agent repository under EFM to use the Deploy Agent functionality. The default binaries base directory is ${EFM_HOME_DIRECTORY}/agent-deployer/binaries. You can customize the path by setting the efm.agent-deployer.binariesRootPath property in the efm.properties file.

Ensure your directory structure follows the convention [{agentType}/{osArch}/{agentVersion}]
Accepted values:
- agentType: java or cpp
- osArch: linux*, windows*, macos* (This allows using different binaries for different OS versions.)
- agent version: Cloudera version of the agent (for example: 1.23.02)
note
MacOS is only supported for testing purposes.

Each directory should contain only one file. While the file name can be any valid name, the extension must adhere to the following constraints:

On Linux: Only the tar.gz file type should be used.

On Windows:

For the Java agent type, the tar.gz format should be used.

For the CPP agent type, the MSI format should be used.
A few examples:
```
${EFM_HOME_DIRECTORY}/agent-deployer/binaries/java/linux/1.23.02/minifi.tar.gz
${EFM_HOME_DIRECTORY}/agent-deployer/binaries/java/linux/1.23.04/minifi.tar.gz
${EFM_HOME_DIRECTORY}/agent-deployer/binaries/java/windows/1.23.02/minifi.tar.gz
${EFM_HOME_DIRECTORY}/agent-deployer/binaries/java/windows/1.23.04/minifi.tar.gz
${EFM_HOME_DIRECTORY}/agent-deployer/binaries/cpp/linux/1.23.03/minifi.tar.gz
${EFM_HOME_DIRECTORY}/agent-deployer/binaries/cpp/linux/1.23.06/minifi.tar.gz
${EFM_HOME_DIRECTORY}/agent-deployer/binaries/cpp/windows32bti/1.23.03/minifi.msi
${EFM_HOME_DIRECTORY}/agent-deployer/binaries/cpp/windows64bit/1.23.06/minifi.msi
```
Generate Deploy Agent CLI command.
1. Click Monitor on the left menu bar to open the Dashboard with the list of agent classes.
2. Click the created agent class to display the agent class details.
3. Click Actions > Deploy Agent Command.
  
  If the agent repository has been created successfully in the previous step, the Agent Type, Agent Version, and OS drop-downs are pre-populated.
4. Choose the desired options and click Generate.
  
  The generated command appears in the same window.
5. To install the agent, copy and paste the generated CLI command into the host where the agent will be installed.
  
  note
  
  Ensure the agent host has a network connection to EFM to successfully run the command.
Set the advanced options for the Deploy Agent CLI command.
In specific scenarios, the default configuration may not meet your specific requirements. Use advanced configurations to customize parameters.

To make the advanced options visible, select the Show Advanced Configurations checkbox.
EFM Base URL

The full URL of the EFM REST API base. The generated command uses this URL to access EFM from the remote host. If EFM is behind a load balancer or a proxy, you can override this URL.
note
For a more sophisticated way to handle EFM behind a proxy, set the efm.proxy.c2ProxyPath property in the efm.properties file. This automatically sets the EFM Base URL to the correct value.

Agent ID

The agent ID is automatically generated, but you can also set a custom value. If a custom value is set, make sure that each command generation has a unique identifier.

Heartbeat Period

The agent sends heartbeats periodically using this value.

Service User
- On Linux: A user is created with this name and the agent process runs under this user.
- On Windows: This is not applicable at the moment because the Java agent is not started as a service and the C++ agent is parameterized with the Service User.
Autoconfigure Security
If security is enabled in EFM (for example any authentication method is turned on), the agent needs to connect to EFM in a secured manner. If Autoconfigure Security is enabled, EFM generates the necessary certificates for the agent, and the command downloads the certificates through a secured channel, and configures the agent automatically.

You can provide your own Certificate Authority (CA) for signing certificates. If a custom CA is not provided, EFM generates one during startup, which will be used for the certificates.
note
To enable this feature, the following values must be set in the efm.properties file:
efm.agent-deployer.security.autoConfiguration=true efm.agent-deployer.security.ca.privateKeyPassword=<strong_password>
For more information on Autoconfigure Security features, see Configuring Agent Deployer for securing agents.
Self-Signed Certificates and CA Cert PEM File Location
If EFM is set up using a self-signed certificate (where EFM generated the CA and it was not provided externally), the request issued by the Agent Download command may be rejected since the issuing host will not trust EFM.
You have two options:

Select the Trust Self-Signed Certificates checkbox to trust the self-signed certificate and proceed with the Agent Download command.

Use the CA Cert PEM File Location to reference a CA Cert on the agent file system. This allows the agent to trust EFM by using the specified CA certificate.
Dynamic properties:
You have the flexibility to override any arbitrary MiNiFi properties.

For MiNiFi Java agents: The properties are located in the bootstrap.conf file.

For MiNiFi C++ agents: The properties are located in the minifi.properties file.

Run the command.

Copy and paste the generated CLI command on the target host’s shell or command line.

The command downloads the agent binary into the directory where the script was run. After that, the command extracts, configures, and runs the agent in the background.

For Java agents on Linux, you should see a log message similar to the following.

user@user-host /opt/cloudera/minifi curl -L \
 -d agentClass=new-agent-class \
 -d agentIdentifier=061c6cf6-1922-41b7-8239-ad25a0f5be9a \
 -d agentType=java \
 -d agentVersion=1.23.04 \
 -d autoConfigureSecurity=false \
 -d baseUrl=http%3A%2F%2Funsecure.cemcldr.link%2Fefm%2Fapi \
 -d hbPeriod=5000 \
 -d osArch=MacOs \
 -d serviceName=minifi \
 -d serviceUser=minifi \
 -d trustSelfSignedCertificates=false \
 http://unsecure.cemcldr.link/efm/api/agent-deployer/script | bash -
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 17505  100 17208  100   297  18755    323 --:--:-- --:--:-- --:--:-- 19152
 _______ __ _______ __ _______ __      __             __         __ __
|   |   |__|    |  |__|    ___|__|    |__.-----.-----|  |_.---.-|  |  .-----.----.
|       |  |       |  |    ___|  |    |  |     |__ --|   _|  _  |  |  |  -__|   _|
|__|_|__|__|__|____|__|___|   |__|    |__|__|__|_____|____|___._|__|__|_____|__|

-- Verifying if the following commands exist: tar gzip cat grep sed seq mkdir sleep
-- Verifying if java is installed on the system...
-- Found java at: /Users/fkis/.sdkman/candidates/java/current/bin/java
openjdk version "1.8.0_362"
OpenJDK Runtime Environment (Zulu 8.68.0.19-CA-macos-aarch64) (build 1.8.0_362-b08)
OpenJDK 64-Bit Server VM (Zulu 8.68.0.19-CA-macos-aarch64) (build 25.362-b08, mixed mode)
-- Verifying if the system is capable of handling tar.gz archives...
-- Looking for a download utility on the system...
-- > curl
-- curl version: 7.88.1
-- Downloading MiNiFi...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  234M  100  234M    0     0  1701k      0  0:02:21  0:02:21 --:--:-- 1123k
-- MiNiFi has been downloaded to directory: minifi-1.23.04-b15
-- Configuring MiNiFi...
-- Starting MiNiFi as a simple background process...
-- Waiting until MiNiFi is up...
-- OK
#Mon Jul 24 15:40:29 CEST 2023
port=64281
pid=87120
secret.key=451f4e40-15a5-4a58-8d18-63983fec927c

  MiNiFi is now started as a background process.

  You can stop it by issuing the following commands:

    %> cd "minifi-1.23.04-b15"
    %> bin/minifi.sh stop

  To start again:

    %> cd "minifi-1.23.04-b15"
    %> bin/minifi.sh start

-- Installation has successfully completed.

  In addition of the existing (default) configuration values, the followings have been applied:

  ---
c2.agent.identifier=061c6cf6-1922-41b7-8239-ad25a0f5be9a
c2.rest.path.heartbeat=/c2-protocol/heartbeat
c2.rest.path.acknowledge=/c2-protocol/acknowledge
c2.rest.url=http://unsecure.cemcldr.link/efm/api/c2-protocol/heartbeat
c2.rest.url.ack=http://unsecure.cemcldr.link/efm/api/c2-protocol/acknowledge
c2.agent.class=new-agent-class
c2.agent.heartbeat.period=5000
c2.enable=true
c2.rest.path.base=http://unsecure.cemcldr.link/efm/api
  ---

  If you would like to modify this configuration, you need to perform these steps:

    1) Stop MiNiFi
    2) Edit the files located in "minifi-1.23.04-b15/conf"
    3) Start MiNiFi

After successful installation, the agent sends heartbeats, appearing as an active agent in EFM.