Software Load Test Scripting: Best Practices

1.0 Plan your Business Process scripting effort
Successful performance testing relies on in-depth understanding of the application, the business requirements of the customer and careful planning of the script development phase.

1.1 Gather relevant data
Make sure you get the document that describes the business objectives of the application, maps the customer environment and provides the most appropriate solution at each stage of deployment. This document provides the foundation for the scripting effort and must be created before the script planning and development by the business analysts/Solution Architects.

• Understand the organization’s goals
• Identify the key IT-enabled business processes to achieve these goals
• Meet the stakeholders
• Identify pain points and current state of environment
• Map and analyze the tested applications
• Define the project deliverables and objectives
• Plan the test environment
• For further details refer to the Performance Testing Product Best Practices document.

1.2 Map the application
Get familiar with the application you’re about to test. Understanding the real user workflows that map together and form a business process is the most important step to creating the script design. With a thorough script design, the emulation of real user interaction is legitimate, traceable, and logical. Technical errors are more easily discovered through good test design, and logical errors do not become an issue. Additionally such step might be helpful to detect certain actions that may need special attention (e.g. a web page that often returns an error in a web based application). Interview the system administrator and the application developers (if available) and learn how the application works and its internal architecture. Focus on gathering the following data for each application:

• How the applications is being used by its regular users
• Application architecture (2/3 tier client/server, ASP/Java, etc)
• The internal process of the application and how they work
• Protocols used (HTTP, JavaScript/JSP, Tuxedo/JOLT, etc)
• Development Source: Vendor or in-house, or combination?
• Communication paths to the application (WAN/LAN, link speed, known congestion issues)
• Security issues (firewalls, username/passwords etc.)
• Data required to interact with the application.

1.3 Create the Performance Testing requirements document
The Performance Testing Requirements document is the cornerstone of any Performance Testing project. It must include precise definitions of what would be achieved during the project implementation and set the Performance Testing strategy. Within the Performance Testing Requirements document should reside Use Case scenarios that define the business processes necessary to emulate the load. Thus, this document becomes the basis for the script design, and should contain the following aspects:

• Business processes that need to be tested.
• The flow of each business process.
• Key Performance Indicators (KPI)s per business process. KPIs map what the expected results of the script execution should indicate. For example, a KPI for a script that emulates a login-logout business process would be the number of seconds that is appropriate for the transaction to be considered acceptable.

Make sure you get the stakeholders approval for the test plan before moving forward.

1.4 Plan the Performance Testing
Performance testing in a pre-production environment should be the final step before deploying applications into production. One of the significant contributors to the high cost of application testing is duplication of script development efforts. In order to reduce such inefficiencies, the planning phase should begin weeks before the actual script development and can be performed during other functional testing efforts. In this stage create the detailed Performance testing plan. The plan should cover the following aspects of performance testing:
Decide which functions should be tested. These functions represent the business processes of real users interacting with the system under test
Identify the required Performance test types/Performance test cases.
Align the application lifecycle with the available tests. Make sure that the performance tests you have chosen are suitable for the application lifecycle (development, pre-production or production).
Define in detail the performance test scenarios.

1.5 Validate the test environment

1.5.1 Align with the production environment

• It is important to run Performance tests against an environment that is as similar as possible to the production environment; otherwise this could lead to inaccurate test results.
• The performance test environment requires at least two sets of testing environments:
• Production infrastructure
• Test environment

Note: Normally most of the performance testing would be done in the test environment; however there are situations where the business requirement would be to run certain performance tests on the production environment to validate its readiness for the next business day.

1.5.2 Decide what to script
Analyze the Use Case scenarios and understand the impact they have on the system under test. Oftentimes, several Use Case scenarios have the same effect when executed, making the scripting of them redundant. Therefore, it is efficient to identify these redundancies and decide to script only a small subset of the larger set of business processes. For example, the “login” use case and the “logout” use case both impact the same middleware code and the same database tables, so it may not be necessary to script both processes and execute them.

There are three criteria for deciding what to script:
Volume: the most common business processes that generate the highest total number of interaction.
Dynamic Content: The business processes that cause the most workload for the system, especially workload that involves dynamic information to be produced.
Mission Critical: Not necessarily those business processes which fulfill the above two requirements, but are use cases that are considered the most important aspects of the application usage.

Normally you will find that the Pareto 80-20 rule applies in such situations: 20 percent of the transactions cause 80 percent of the system load. Identifying those 20 percent in the script design phase (and deciding they are the most relevant to your script) ensures that every performance test accurately mimics the load generated by users.

1.6 Scripting design document
Prepare a script design document which explains the Use Case Scenarios and fulfills the framework of the overall Performance Test Plan.

• The script design document should contain, at minimum, the following details:
• The Business Process name (and the technical script name that represents its emulation).
• The Performance Test to which the script is being executed for.
• The specific transactions within the script that are being measured for performance (the script may itself be measured, as well as individual steps within it).
• The KPIs that will contrast the results of the transactions against expectations.
• Technical issues (such as protocol considerations, network considerations, etc).
• Environmental configuration (mapped to the Run-Time Settings of the Performance test scenario).
• Data considerations.

Business process        

Script main purpose          

 Script technical name

Search Friends in Facebook             

Web transaction: Login, Search Friends, Logout.

 Facebook_SearchFriends_01