10 Performance Tests for an optimized software quality

3 October 2019
test de performance
Let’s take a tour of the various performance tests that exist...

 

Test Unitaire
UNIT TEST

This test consists in simulating several successive iterations of a user scenario with a single user.

The aim is to obtain a critical point of comparison through the evaluation of application latency, i.e. the time necessary for the application to process the query and return the response data.

 

 

 

test nominal
Nominal TEST

This test consists in simulating one or more user scenarios in parallel with the number of concurrent users expected in production.

The aim is to evaluate the impact  of the current or planned load on the platform, to scale the target platform (number of servers, drive/memory size and to determine the user experience (response times).

 

 

Test Destructif

DESTRUCTIVE TEST OR LIMIT TEST

The test consists in simulating one or more user scenarios in parallel by gradually increasing the number of concurrent users.

The aim is to determine from when the system degrades and to identify the bottlenecks (=which element(s) is(are) responsible for the application limit).

In this example, the limit is reached for 100% CPU consumption of the application server. If it is necessary to go further, the server's physical capacity must be increased, or other application servers must be added.

 

Test d'Endurance
ENDURANCE TEST

This test consists in simulating one or more user scenarios in parallel with a nominal number of concurrent users over a long period (24/48 hours).

The aim is to determine the stability of the system over the long term and to assess the resource defects (memory leak).

For this test, it is important to check that all the metrics recorded (CPU consumption, memory, response times) are stable throughout the test, i.e. that there is no increase or reduction over the duration.

 

Test de Scalabilité
SCALABILITY TEST OR INCREMENTAL TEST

This test consists in simulating one or more user scenarios in parallel with the several increments of concurrent users.

The aim is to determine whether the consumption of application resources follows the load increase and frees up resources if the load is reduced.

It is important to verify that the consumption of resources is proportional to the load added or reduced. Response times are supposed to remain stable in this test (no increment).

 

Test de non Regression
NON-REGRESSION TEST

This test consists in simulating one or more user scenarios in parallel with a nominal number of concurrent users with different code levels.

The aim is to compare resources and response times between two deliveries.

In this example, we can observe a degradation in the server's CPU consumption in version N+1 in the 3rd increment of the scalability test (60 users).

 

Test de diagnostic - Script par Script
DIAGNOSTIC TEST – Script BY Script

This test consists in simulating each script separately with a nominal number of concurrent users.

The aim is to identify the script responsible for a degradation among a set of scripts.

In this example, we can observe that the script responsible for the degradation observed in terms of CPU consumption is script 2.

 

 

Test de diagnostic - Transaction par transaction
Diagnostic TEST – Transaction BY Transaction

This test consists in simulating each transaction of a script separately with a nominal number of concurrent users.

The aim is to identify the script responsible for a degradation among a script transaction sequence.

In this example, we can observe that the transaction responsible for the degradation observed in terms of CPU consumption is transaction C.

 

Test Opérationnel
fond blanc2
OPERATIONAL TEST

This test consists in simulating one or more user scenarios in parallel with a nominal number of concurrent users and to simulate a disruptive event (failure, restart, batch execution). This is to determine the impact of a failure caused on the system or to secure maintenance operations in production.

In this example, we have simulated the stoppage of application server 2 after 20 minutes of testing. We can observe that application server 1 retrieves the load of server 2 while it is stopped, causing an increase in its CPU consumption as expected, without the number of transactions per second being affected. Once server 2 has restarted, we regain normal activity.

 

 

LOAD INCREASE TEST
Test de montée en charge

This test consist in simulating a mass arrival of queries (e.g. connections) with many concurrent users.

The aim is to evaluate the system's speed in accepting new connections.

In this example, we can observe a peak in the application server's CPU consumption at the mass arrival of concurrent users. This stabilises once the user peak has passed.

 

 

Description of the tools

The performance test tools break down into three parts: scripting, injection and investigation.

Scripting

A script generator must be chosen that will record users’ actions corresponding to a business process (i.e. the sequence of user actions, service calls) and transcribe them in a programming language. It is the tester's responsibility to then add page controls/break times or to make certain parameters variable.

Injection

The performance test requires one or more tools to create test scenarios, inject a large number of users, monitor the business information (response times, transactions per second, speed) and the test environment resources and finally to analyse and store the test results.

Investigation

Some diagnostic tools can facilitate the detection of the origin of problems encountered (memory leak, blocked threads, loop in the code or cache problem).

Several tools enable us to carry out performance tests. They are differentiated by:

  • the type of application tested (web, SAP, Oracle, RDP, Citrix, Mainframe, Windows Sockets, etc.),
  • the price of licences,
  • the cost of implementation,
  • the maturity and efficiency of the product,
  • integration with other tools.

The market leader is Hewlett Packard which offers a full suite of tools:

  • HP Loadrunner: injector that also includes a script generator (Virtual User Generator),
  • HP ALM Performance Center: requirements management, injection planning monitoring, storage of results,
  • HP Analysis: results analyser and report generator.

There are also open source tools for scripting/injection such as Apache Jmeter or Gatling.

For profiling tools, Dynatrace is a very efficient diagnostic tool to analyse the code tested in depth. Also note tools such as Yourkit or Eclipse MAT for the analysis of memory leaks or Visual Studio SQL Profiler for database profiling.

The performance test is an activity that above all aims to optimise software quality. It is essential for the implementation and success of an IT project.

 

Don’t hesitate to check out our web page on: Software Testing.

Let's have a chat about your projects. 

bouton-contact-en