As application developers, how do we build a performance testing plan?

[51CTO.com Quick Translation] Developing a performance test plan is not just about putting together a guide document, but about in-depth consideration to ensure that you can answer various questions about system performance with the minimum number of execution steps.

First, it is necessary to understand what questions the test results need to be used to answer. Here are some common examples:

• Stress testing: What is the maximum number of concurrent users that the system can withstand and provide an acceptable user experience? Where is the inflection point?
• Load testing: How does the application perform under the current system load scenario? Can further improvement be achieved?
• Persistence testing: How well does the system work after it has been running for a while? (Sometimes we need to reboot the server every night until we can find the root cause of a gradual performance degradation.)
• Peak testing: If a normally operating system is overwhelmed by a sudden peak, how long will it take us to recover the system?

In addition, when planning performance testing, we should also consider our own capabilities:

Is the team prepared to handle this type of situation or does it require additional training?

How quickly can we find solutions to performance issues?

What physical resources do we already have? How difficult is it to obtain the ones we don’t have?

With these questions, we can review the system in a targeted manner and think about how to improve it.

To limit the length of this article, today we will focus on just one of the topics: After defining the expected load, how do we run the exact corresponding load scenario?

Performance test planning: number of concurrent users

It is important to emphasize again that we cannot simulate all concurrent users at the beginning of the test . In this case, multiple problems will occur at the same time, and it will be difficult to find out the real root cause. Therefore, you should increase the load intensity in an iterative manner and observe the problems that arise in the process.

• Example: Load Testing

Now, let's assume that we need to test whether the current system can support 1,000 concurrent users.

1. First test : 1 user, no concurrency. We use this as a baseline, but you can also choose 5 or 10 users, but make sure the specific number is far lower than the expected level.

2. Test 2 : 200 concurrent users (or 20% of the expected load). Now we will be able to get a lot of information, which will ultimately determine whether the testing process is smooth.

During initial testing, we first address important issues and default configurations (number of connection pools or Java heap size, etc.) to understand how the response time compares to the baseline. After analysis and troubleshooting, we repeat the test again until we achieve an acceptable time.

Based on the actual results, we considered using 40% load intensity (with 20% as the basic increment) or 50% (followed by 75% and 100%) in the third test. No matter which choice we make, we can understand the changes in the system response time in the process and understand its changes as the load increases.

In this example, we use 20% as the increment. In addition, we will repeat the test until the expected SLA is achieved under various load intensities, and then do a new round of increments.

• Example: Stress Testing

In the second example, we want to use stress testing to find the performance inflection point of the system. This means that we want to increase the number of users and analyze whether it will bring about a synchronous increase in throughput. Once the concurrency increases but the throughput remains the same, it means that we have reached the inflection point and the system has reached saturation.

Here we use the so-called exploratory performance test, which assumes that the inflection point must exist within a range of 1,000 concurrent users.

Currently, various load simulation tools can increase the load over time, for example, starting with 0 concurrent users and increasing to 1,000 concurrent users after 1 hour. In this way, we can observe when the system throughput decreases. Assuming that the inflection point occurs at about 650 concurrent users, we can further refine it to determine the exact inflection point location.

• Example: Persistence Test

In the persistence test, we run a constant load under acceptable conditions, such as between 50% and 70% of maximum capacity. Of course, the specific situation depends on your actual system scenario.

Generally speaking, this type of testing should be done after stress or load testing to find other types of problems (such as memory leaks and hanging connections). If you have time, you can extend the testing cycle to find more problems.

• Example: Peak Test

As mentioned before, the significance of peak testing is to understand how quickly the system can recover. Here, we can try to set a traffic peak of 1 minute, then reduce the load and observe whether the system can respond correctly or temporarily suspend the request.

Of course, you can also try to establish a small peak first, and then gradually increase the intensity to study the actual response of the system. It should be emphasized that the relevant simulation should correspond to the analysis conclusions of user behavior as much as possible, especially based on relevant log information.

Summarize

The specific design of the performance test plan depends on the actual questions you want to answer, but the common point is that we need to reduce the number of tests as much as possible, optimize the test costs and increase the benefits. Therefore, iterative incremental (for load, persistence and peak testing) and fine-grained (for stress testing) methods are undoubtedly worth trying.

Original title: How to Make a Performance Test Plan

Original author: Federico Toledo

[Translated by 51CTO. Please indicate the original translator and source as 51CTO.com when reprinting on partner sites]