Performance test of MQTT maximum number of device connections

Part 01

MQTT Protocol

The MQTT protocol is one of the most common protocols for the IoT platform and is also the primary device access protocol for the OneNET platform. The IoT platform must support massive device access, but how many devices can the MQTT access service support online at the same time? Understanding this indicator can better provide a scientific basis for the platform's operation and maintenance.

But how to test the maximum online volume indicator quickly and easily? How to choose tools and create scripts?

When testing performance, we first think of commonly used performance testing tools such as Jmeter and Locust. However, these tools are good at testing the concurrency and throughput of services, which are not suitable for current testing scenarios.

Then we can think of using a third-party Jar package or a protocol library implemented by a third-party library, and using multi-threaded startup of the device. However, the startup of the press thread is limited, which is simply a drop in the bucket for services that support hundreds of thousands or millions of device accesses, and it is difficult to estimate how many presses are needed.

The next thing that comes to mind is the Select method to manage the Socket connections of devices in batches. The problem arises again, the asynchronous IO managed by Select also has its limits, so this method is eventually abandoned.

Part 02

Test Method  

After the previous analysis and practice, the final method was determined to use asynchronous IO to batch simulate devices connecting to the server, report registration messages at a certain frequency, continuously traverse the cache data received by the device Socket, parse the service message to determine whether the device is connected successfully, and keep the device online by periodically reporting the heartbeat. The implementation details are as follows:

(1) Implement basic device class: encapsulate some MQTT protocol message methods, including device registration, subscription, publishing, heartbeat, etc.

MQTT registration message encapsulation example (Java)

(2) Implement the device class: mainly record the device registration status, subscription status, keep-alive interval, the most important service message parsing and response methods, and the non-blocking Socket (SocketChanel in Java) for the device to connect to the server

Service message parsing code example (Java)

(3) Implement the main program class: manage batch devices, control the device registration frequency, when the device reports data, monitor the service to send data, count the number of device connections, continuously report heartbeats, keep the device online, etc. The specific implementation logic is as follows:

• Initialize the device list in batches;
• Start three threads at the same time;
• Start the device registration thread, initialize the connection between the device and the service and send the registration message uplink. You can specify the number of devices that can be registered at the same time according to the settings, and automatically exit after all devices are registered;
• Start the connection statistics thread to periodically count the number of successful device connections, successful subscriptions, and failed connection devices;
• Start the device socket traversal thread, continuously poll the received data of each socket, process and respond to the received data, and after each round of traversal, send heartbeat messages to the devices that have not reported heartbeats for a long time to keep the devices alive.

Program theme class receiving thread logic code snippet (Java)

After the program code is completed, the test tool is completed.

This tool has been actually used in project performance testing. It can use all available ports of the press machine to test the maximum number of online devices. It can actually complete the registration of 50,000+ devices in a Linux virtual machine within a few minutes and keep the devices online for a long time.

This tool can maximize the use of press port resources and improve test execution efficiency. For services with large online volumes, you only need to run this tool on more presses.