Agents registering to EFM

Learn about the estimated number of requests, latencies, CPU usage, memory usage, and network usage when agents register to Edge Flow Manager (EFM).

In this scenario, 100000 agents are registering into EFM under the same agent class. 120 new agents are registered every second. Registering the agents takes 833 seconds. Heartbeat interval is 60 seconds. Each agent heartbeats 14 times. This results in nearly 1800 heartbeat TPS maximum. After an agent sends all of the 14 heartbeats, it does not send more so after the heartbeat TPS reaches its maximum. The heartbeat TPS then starts to decrease slowly.

The goal of the test is to demonstrate that all agents can be registered successfully with the given configuration.

Number of requests

You can see, in the following images, that the number of active agents builds up in 15 minutes until it reaches the maximum. At the top, EFM is under the load of approximately 1900 requests per second - 600 heartbeats and 40 acknowledgements per agent per second.

Latencies

Most of the time latencies are below 100 ms with peaking up in 5 seconds.

The first chart shows the full latency including the network latency, while the second chart shows results without the network latency.

CPU usage

CPU usage is measured in two ways. The first chart shows the JVM level metrics, and the second chart shows values measured by Kubernetes. The results are in line with each other. Since you have 6 CPUs allocated, the peak CPU consumption of approximately 35 percent is equal to 2 CPU cores. You can observe higher CPU usage on one specific EFM node. This is because one node is always selected as master node, which has some extra tasks resulting in higher CPU usage.

Memory usage

The heap consumption fluctuates between 1.5 GB and 7.5 GB with maximum Garbage Collection (GC) pauses of 100 ms; sometimes peaking up to 400 ms.

Network usage

The aggregated network in-bound rate tops out at approximately 100 MB/s, 30 MB/s per EFM node, and 5 MB/s for the database instance. Higher network traffic can be observed in the first half of the test. The reason behind this is when the agents are registering to EFM with sending a lightweight heartbeat, EFM requests the agents to send the agent manifest. Agent manifests are an order of magnitude higher in size than lightweight heartbeats.

In the second half of the test all agents are registered, and only lightweight heartbeats are sent. Output traffic rate is lower than 10 MB/s at the peak.