A new player in the home automation space had come up with a smart home device to capitalise on the smart home revolution. The pre-launch marketing campaign had yielded hundreds of pre-bookings for the product. Tismo was brought on board to set up a scalable IoT architecture and carry out performance testing of the IoT cloud application.
With a robust architecture set up on AWS IoT Platform, performance testing of the IoT cloud application was the last checkpoint before market launch. The challenge for the Tismo team was to simultaneously simulate thousands of end devices, each exhibiting different behavior and functionality, and measure the performance of the web server. This involved investigating the server behaviour when thousands of users start using the system. Aspects such as the server’s request handling capability, crashing conditions and the performance of REST API were to be examined.
Products with poor performance metrics almost never meet expected sales goals. Thus, this was the ‘break-or-make’ moment for the client, an up and coming player in the market.
Performance Testing ensures that software applications perform optimally under their expected workload. In addition to testing the features and functionality supported by the system, performance indicators like response time, stability, scalability, reliability etc are evaluated to avoid performance bottlenecks. Performance testing unearths what needs to be improved before the product hits the market.
Tismo understood that the requirement called for a specialized performance testing tool to analyze and measure the performance of the IoT cloud application. Apache JMeter was chosen due to its tailor-made features for performance testing of IoT cloud applications and servers and the comprehensive test coverage that it offered.
The basic element in a JMeter test is referred to as a thread. Each thread executes the test plan, completely independent of other test threads. Tismo created multiple threads to run performance testing of the IoT cloud application while simultaneously sending requests corresponding to every one of the numerous device functionalities. Loop controllers to send requests to individual devices were set up and the requests to test device functionalities were sourced from pre-defined CSV files. Tismo ensured that the server responses were accurate by running HTTPS code assertions.
A pronounced advantage of using JMeter for performance testing is its highly extensible core, that allows virtually unlimited testing capabilities. Tismo used the following plugins to carry out performance testing of the server using JMeter:
Tismo was able to identify performance bottlenecks and optimize the architecture to support thousands of devices. Performance variation was kept minimal by tweaking certain server-side configurations.
The Apache JMeter is an open source software, designed to analyze and measure the performance of web applications and servers. It simulates user behaviour and returns statistical information about the target server through graphical outputs.
Tismo carried out performance testing of an IoT cloud application using Apache JMeter. Multiple threads and loop controllers were set up and simultaneous requests, corresponding to each device functionality, were sent to the server. HTTPS code assertions were carried out and various plugins were used to ensure maximum coverage for performance testing.