Using the Agent Deployer
Learn how you can deploy a MiNiFi application using the Agent Deployer in the Edge Flow Manager (EFM).
-
Create a new agent class.
- Click Monitor on the left menu bar to access the Dashboard page.
-
Click New Agent Class.
-
Enter a name in the Agent Class Name text box and click
Create.
The new class is created and is added to the list of agent classes. -
Create an agent repository.
Before deploying an agent, you need to create an agent repository under EFM to use the Deploy Agent functionality. The default binaries base directory is ${EFM_HOME_DIRECTORY}/agent-deployer/binaries. You can customize the path by setting the efm.agent-deployer.binariesRootPath property in the efm.properties file.
Ensure your directory structure follows the convention [{agentType}/{osArch}/{agentVersion}]
Accepted values:- agentType: java or cpp
- osArch: linux*, windows*, macos* (This allows using different binaries for different OS versions.)
- agent version: Cloudera version of the agent (for example: 1.23.02)
A few examples:${EFM_HOME_DIRECTORY}/agent-deployer/binaries/java/linux/1.23.02/minifi.tar.gz ${EFM_HOME_DIRECTORY}/agent-deployer/binaries/java/linux/1.23.04/minifi.tar.gz ${EFM_HOME_DIRECTORY}/agent-deployer/binaries/java/windows/1.23.02/minifi.tar.gz ${EFM_HOME_DIRECTORY}/agent-deployer/binaries/java/windows/1.23.04/minifi.tar.gz ${EFM_HOME_DIRECTORY}/agent-deployer/binaries/cpp/linux/1.23.03/minifi.tar.gz ${EFM_HOME_DIRECTORY}/agent-deployer/binaries/cpp/linux/1.23.06/minifi.tar.gz ${EFM_HOME_DIRECTORY}/agent-deployer/binaries/cpp/windows32bti/1.23.03/minifi.msi ${EFM_HOME_DIRECTORY}/agent-deployer/binaries/cpp/windows64bit/1.23.06/minifi.msi
-
Generate Deploy Agent CLI command.
- Click Monitor on the left menu bar to open the Dashboard with the list of agent classes.
- Click the created agent class to display the agent class details.
-
Click Actions > Deploy Agent Command.
If the agent repository has been created successfully in the previous step, the Agent Type, Agent Version, and OS drop-downs are pre-populated.
-
Choose the desired options and click Generate.
The generated command appears in the same window.
-
To install the agent, copy and paste the generated CLI command into the host where
the agent will be installed.
-
Set the advanced options for the Deploy Agent CLI command.
In specific scenarios, the default configuration may not meet your specific requirements. Use advanced configurations to customize parameters.
To make the advanced options visible, select the Show Advanced Configurations checkbox.
- EFM Base URL
-
The full URL of the EFM REST API base. The generated command uses this URL to access EFM from the remote host. If EFM is behind a load balancer or a proxy, you can override this URL.
- Agent ID
-
The agent ID is automatically generated, but you can also set a custom value. If a custom value is set, make sure that each command generation has a unique identifier.
- Heartbeat Period
-
The agent sends heartbeats periodically using this value.
- Service User
-
- On Linux: A user is created with this name and the agent process runs under this user.
- On Windows: This is not applicable at the moment because the Java agent is not started as a service and the C++ agent is parameterized with the Service User.
- Autoconfigure Security
-
If security is enabled in EFM (for example any authentication method is turned on), the agent needs to connect to EFM in a secured manner. If Autoconfigure Security is enabled, EFM generates the necessary certificates for the agent, and the command downloads the certificates through a secured channel, and configures the agent automatically.
You can provide your own Certificate Authority (CA) for signing certificates. If a custom CA is not provided, EFM generates one during startup, which will be used for the certificates.
For more information on Autoconfigure Security features, see Configuring Agent Deployer for securing agents.
- Self-Signed Certificates and CA Cert PEM File Location
-
If EFM is set up using a self-signed certificate (where EFM generated the CA and it was not provided externally), the request issued by the Agent Download command may be rejected since the issuing host will not trust EFM.
You have two options:- Select the Trust Self-Signed Certificates checkbox to trust the self-signed certificate and proceed with the Agent Download command.
- Use the CA Cert PEM File Location to reference a CA Cert on the agent file system. This allows the agent to trust EFM by using the specified CA certificate.
- Dynamic properties:
-
You have the flexibility to override any arbitrary MiNiFi properties.
- For MiNiFi Java agents: The properties are located in the bootstrap.conf file.
- For MiNiFi C++ agents: The properties are located in the minifi.properties file.
-
Run the command.
Copy and paste the generated CLI command on the target host’s shell or command line.
The command downloads the agent binary into the directory where the script was run. After that, the command extracts, configures, and runs the agent in the background.
For Java agents on Linux, you should see a log message similar to the following.
user@user-host /opt/cloudera/minifi curl -L \ -d agentClass=new-agent-class \ -d agentIdentifier=061c6cf6-1922-41b7-8239-ad25a0f5be9a \ -d agentType=java \ -d agentVersion=1.23.04 \ -d autoConfigureSecurity=false \ -d baseUrl=http%3A%2F%2Funsecure.cemcldr.link%2Fefm%2Fapi \ -d hbPeriod=5000 \ -d osArch=MacOs \ -d serviceName=minifi \ -d serviceUser=minifi \ -d trustSelfSignedCertificates=false \ http://unsecure.cemcldr.link/efm/api/agent-deployer/script | bash - % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 17505 100 17208 100 297 18755 323 --:--:-- --:--:-- --:--:-- 19152 _______ __ _______ __ _______ __ __ __ __ __ | | |__| | |__| ___|__| |__.-----.-----| |_.---.-| | .-----.----. | | | | | ___| | | | |__ --| _| _ | | | -__| _| |__|_|__|__|__|____|__|___| |__| |__|__|__|_____|____|___._|__|__|_____|__| -- Verifying if the following commands exist: tar gzip cat grep sed seq mkdir sleep -- Verifying if java is installed on the system... -- Found java at: /Users/fkis/.sdkman/candidates/java/current/bin/java openjdk version "1.8.0_362" OpenJDK Runtime Environment (Zulu 8.68.0.19-CA-macos-aarch64) (build 1.8.0_362-b08) OpenJDK 64-Bit Server VM (Zulu 8.68.0.19-CA-macos-aarch64) (build 25.362-b08, mixed mode) -- Verifying if the system is capable of handling tar.gz archives... -- Looking for a download utility on the system... -- > curl -- curl version: 7.88.1 -- Downloading MiNiFi... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 234M 100 234M 0 0 1701k 0 0:02:21 0:02:21 --:--:-- 1123k -- MiNiFi has been downloaded to directory: minifi-1.23.04-b15 -- Configuring MiNiFi... -- Starting MiNiFi as a simple background process... -- Waiting until MiNiFi is up... -- OK #Mon Jul 24 15:40:29 CEST 2023 port=64281 pid=87120 secret.key=451f4e40-15a5-4a58-8d18-63983fec927c MiNiFi is now started as a background process. You can stop it by issuing the following commands: %> cd "minifi-1.23.04-b15" %> bin/minifi.sh stop To start again: %> cd "minifi-1.23.04-b15" %> bin/minifi.sh start -- Installation has successfully completed. In addition of the existing (default) configuration values, the followings have been applied: --- c2.agent.identifier=061c6cf6-1922-41b7-8239-ad25a0f5be9a c2.rest.path.heartbeat=/c2-protocol/heartbeat c2.rest.path.acknowledge=/c2-protocol/acknowledge c2.rest.url=http://unsecure.cemcldr.link/efm/api/c2-protocol/heartbeat c2.rest.url.ack=http://unsecure.cemcldr.link/efm/api/c2-protocol/acknowledge c2.agent.class=new-agent-class c2.agent.heartbeat.period=5000 c2.enable=true c2.rest.path.base=http://unsecure.cemcldr.link/efm/api --- If you would like to modify this configuration, you need to perform these steps: 1) Stop MiNiFi 2) Edit the files located in "minifi-1.23.04-b15/conf" 3) Start MiNiFi
After successful installation, the agent sends heartbeats, appearing as an active agent in EFM.