Agents heartbeating to Edge Flow Manager

Learn about the estimated number of requests, latencies, CPU usage, memory usage, and network usage when agents heartbeat to the Edge Flow Manager.

After all agents have successfully registered in the previous test, the same agent class and the agents are reused in this scenario. Agents are brought online in 1 minute with a heartbeat interval of 60 seconds. Each agent heartbeats 25 times and then exits. Because the agents are brought online almost at the same time, you see a steep increase in the heartbeat TPS at the start of the test and a steep decrease in the heartbeat TPS at the end of the test.

The goal of the test is to see that the system is able to serve the agents’ heartbeat requests under maximum TPS during a longer period of time.

Number of requests

You can see, in the following images, the number of active agents builds up in 1 minute until it reaches the maximum. At the top, the Edge Flow Manager is under the load of approximately 1800 requests per second - 600 heartbeats per agent per second.

Latencies

Most of the time latencies are below 100 ms with peaking up in 2 seconds.

The first chart shows the full latency including the network latency, while the second chart shows results without the network latency.

CPU usage

CPU usage is measured in two ways. The first chart shows the JVM level metrics, and the second chart shows values measured by Kubernetes. The results are in line with each other. Since you have 6 CPUs allocated, the peak CPU consumption of approximately 16-25 percent is equal to 1.0-1.5 CPU cores.

In this case as well, you can observe higher CPU utilization for the master node.

Memory usage

The heap consumption fluctuates between 1.5 GB and 7.5 GB with maximum garbage collection pauses of 50 ms - 100 ms; sometimes peaking up to 400 ms.

You can observe that compared to the previous test case there is less pressure on the Garbage Collection (GC) in this test case. Agent registration is more expensive in terms of resources than just serving heartbeats.

Network usage

The aggregated network inbound rate tops out at approximately 25 MB/s, 8 MB/s per Edge Flow Manager node, and 1 MB/s for the database instance. Output traffic rate is lower than 10 MB/s at the peak.

The lower network usage is expected in this case. Only heartbeat messages without the agent manifest part are sent in this test, whereas in the previous test each agent sends one acknowledgement including the agent manifest, which is approximately 5 times larger.