Software Load Test Scripting: Best Practices

1.0 Plan your Business Process scripting effort
Successful performance testing relies on in-depth understanding of the application, the business requirements of the customer and careful planning of the script development phase.

1.1 Gather relevant data
Make sure you get the document that describes the business objectives of the application, maps the customer environment and provides the most appropriate solution at each stage of deployment. This document provides the foundation for the scripting effort and must be created before the script planning and development by the business analysts/Solution Architects.

• Understand the organization’s goals
• Identify the key IT-enabled business processes to achieve these goals
• Meet the stakeholders
• Identify pain points and current state of environment
• Map and analyze the tested applications
• Define the project deliverables and objectives
• Plan the test environment
• For further details refer to the Performance Testing Product Best Practices document.

1.2 Map the application
Get familiar with the application you’re about to test. Understanding the real user workflows that map together and form a business process is the most important step to creating the script design. With a thorough script design, the emulation of real user interaction is legitimate, traceable, and logical. Technical errors are more easily discovered through good test design, and logical errors do not become an issue. Additionally such step might be helpful to detect certain actions that may need special attention (e.g. a web page that often returns an error in a web based application). Interview the system administrator and the application developers (if available) and learn how the application works and its internal architecture. Focus on gathering the following data for each application:

• How the applications is being used by its regular users
• Application architecture (2/3 tier client/server, ASP/Java, etc)
• The internal process of the application and how they work
• Protocols used (HTTP, JavaScript/JSP, Tuxedo/JOLT, etc)
• Development Source: Vendor or in-house, or combination?
• Communication paths to the application (WAN/LAN, link speed, known congestion issues)
• Security issues (firewalls, username/passwords etc.)
• Data required to interact with the application.

1.3 Create the Performance Testing requirements document
The Performance Testing Requirements document is the cornerstone of any Performance Testing project. It must include precise definitions of what would be achieved during the project implementation and set the Performance Testing strategy. Within the Performance Testing Requirements document should reside Use Case scenarios that define the business processes necessary to emulate the load. Thus, this document becomes the basis for the script design, and should contain the following aspects:

• Business processes that need to be tested.
• The flow of each business process.
• Key Performance Indicators (KPI)s per business process. KPIs map what the expected results of the script execution should indicate. For example, a KPI for a script that emulates a login-logout business process would be the number of seconds that is appropriate for the transaction to be considered acceptable.

Make sure you get the stakeholders approval for the test plan before moving forward.

1.4 Plan the Performance Testing
Performance testing in a pre-production environment should be the final step before deploying applications into production. One of the significant contributors to the high cost of application testing is duplication of script development efforts. In order to reduce such inefficiencies, the planning phase should begin weeks before the actual script development and can be performed during other functional testing efforts. In this stage create the detailed Performance testing plan. The plan should cover the following aspects of performance testing:
Decide which functions should be tested. These functions represent the business processes of real users interacting with the system under test
Identify the required Performance test types/Performance test cases.
Align the application lifecycle with the available tests. Make sure that the performance tests you have chosen are suitable for the application lifecycle (development, pre-production or production).
Define in detail the performance test scenarios.

1.5 Validate the test environment

1.5.1 Align with the production environment

• It is important to run Performance tests against an environment that is as similar as possible to the production environment; otherwise this could lead to inaccurate test results.
• The performance test environment requires at least two sets of testing environments:
• Production infrastructure
• Test environment

Note: Normally most of the performance testing would be done in the test environment; however there are situations where the business requirement would be to run certain performance tests on the production environment to validate its readiness for the next business day.

1.5.2 Decide what to script
Analyze the Use Case scenarios and understand the impact they have on the system under test. Oftentimes, several Use Case scenarios have the same effect when executed, making the scripting of them redundant. Therefore, it is efficient to identify these redundancies and decide to script only a small subset of the larger set of business processes. For example, the “login” use case and the “logout” use case both impact the same middleware code and the same database tables, so it may not be necessary to script both processes and execute them.

There are three criteria for deciding what to script:
Volume: the most common business processes that generate the highest total number of interaction.
Dynamic Content: The business processes that cause the most workload for the system, especially workload that involves dynamic information to be produced.
Mission Critical: Not necessarily those business processes which fulfill the above two requirements, but are use cases that are considered the most important aspects of the application usage.

Normally you will find that the Pareto 80-20 rule applies in such situations: 20 percent of the transactions cause 80 percent of the system load. Identifying those 20 percent in the script design phase (and deciding they are the most relevant to your script) ensures that every performance test accurately mimics the load generated by users.

1.6 Scripting design document
Prepare a script design document which explains the Use Case Scenarios and fulfills the framework of the overall Performance Test Plan.

• The script design document should contain, at minimum, the following details:
• The Business Process name (and the technical script name that represents its emulation).
• The Performance Test to which the script is being executed for.
• The specific transactions within the script that are being measured for performance (the script may itself be measured, as well as individual steps within it).
• The KPIs that will contrast the results of the transactions against expectations.
• Technical issues (such as protocol considerations, network considerations, etc).
• Environmental configuration (mapped to the Run-Time Settings of the Performance test scenario).
• Data considerations.

Business process        

Script main purpose          

 Script technical name

Search Friends in Facebook             

Web transaction: Login, Search Friends, Logout.

 Facebook_SearchFriends_01

Creating dll files and using it in LR scripts

Creating DLLs have tons of advantage, especially when you want to take your scripts away from the beaten path. I won’t blabber much in this post and would like to take you people right into the topic I want to cover.

In the previous post I mentioned about Search And Replace function. To serve the purpose of an example, I have recreated the same Search And Replace function in VC++(2010) expresss edition and built a DLL file.

Below is the code that I created in VC++

#include "stdafx.h"
#include "C:\Program Files\HP\LoadRunner\include\lrun.h"

extern "C" __declspec( dllexport ) int lr_searchReplace(char* inputStr, char* outputStr, char lookupChar, char repChar);

int lr_searchReplace(char* inputStr, char* outputStr, char lookupChar, char repChar)
{

char *ptr =inputStr;
char xchar;
int len=0;
int i=0;

lr_output_message("%s",inputStr);
xchar = *ptr;//Copy initial
len=strlen(inputStr);
while (len>0)
{

len--;
xchar = *ptr;
if(xchar==' ')
{
inputStr[i]= repChar;

}

ptr++;
i++;

}

lr_save_string(inputStr,outputStr);
lr_output_message("%s",inputStr);

return 0;
}

Before attempting to build this dll project in VC++, ensure that you have included lrun50.lib in the aditional linker file settings which is in Project Options, else the project won't successfully build/compile. lrun.lib should be present inside \lib folder. Also make sure you include the lrun.h header file as shown in the above code snippet.

After the project is built, dig into the debug folder to find the dll file and then copy it inside the LoadRunner script folder and use it as shown below:

Action()
{
lr_load_dll("SearchNReplace.dll");
lr_save_string("this is a dummy text", "cProjectName");
r_searchReplace(lr_eval_string("{cProjectName}"), "convProjName", ' ', '+');
lr_output_message("%s",lr_eval_string("{convProjName}"));
return 0;
}

How to efficiently utilize the resources of Load Generators?

“How do I calibrate the total number of Load generators required to generate a load of 300 Virtual user if I have each load generators equipped with 3 GB of RAM and 3.3 GHz of CPU clock speed?”

Solution:
Memory foot print which HP as a vendor provides is not very accurate in a general sense. The impact of Memory and CPU footprint of any protocol on a Load generator also depends greatly on the length of the business flow and on the workload profile which the client expects the system to achieve. 

Example: Hypothetically, if in a SAP GUI application if you have 100 virtual users just entering employee name and number in a field and submitting it to the server and are iterating it only 2 times in a half hour period (which if is your test duration) then I would say you may not need more than 1 load generator which is sized with 3 GB of RAM and 3.3 GHz dual core Intel processor.

 Adding to the above, HP does not provide any kind of CPU footprint report. Hence I generally go about telling people to calculate the heaviness/payload of load runner process on their own. This can be done by identifying metrics of resource consumption by 1 user and theoretically determining total resource needed by multiplying the observed consumption for single virtual user with the expected number of concurrent users.

Using Windows Task manager(Process tab) while executing a single Vu will help you determine the memory consumed by a single ‘MDRV’ process which if multiplied with expected concurrent virtual user count will help you in calibrating total memory required to run a load test. Similarly to estimate the CPU resource for a certain protocol, I would recommend you to monitor and record few CPU counters of your Load Generator through ‘perfmon’ while deriving baseline result and multiply the metrics with the target Vu count which is mentioned in the workload profile.

It is ideally recommended to have as many boxes as possible to match the theoretical requirement, but as we all know, one of the many challenges a performance tester faces is lack of sufficient load generator machines. Hence for the benefit of time and to avoid further pestering from your boss to somehow resolve the issue – try utilizing your load generator boxes as efficiently as possible by:

1. Log only when error occurs.
2. Precisely calculate think-time between transactions. This can help
in exerting less load on the load generators. Strictly follow real user screen pause patterns.(note this point as
very important!).
3. Uninstall any daemon applications running on Load generators
which is not a part of AUT.
4. Avoid excessively using the “Show Vuser” option during test
execution.
5. Declaration of variables with large size should be avoided, which
directly helps in conserving overall memory. Consult the development
team if your unable to estimate the right size(This is a major concern
in Java Vuser and VB Vuser scripts).
6. Never over-iterate a script.

Explicitly Dealing with HTTP Redirect Requests in Load runner

By default, Load Runner does not show Redirect requests in the script, though it take care of redirects during replay. This becomes an issue during transaction analysis, because all redirect requests and corresonding responses are combined into one step and shows up as one transaction in the results report.

The web_url/web_custom_request/web_submit_data functions automatically detects a 302 response from the server and redirects to the location specified in the response header.

The problem with automatic redirection in LoadRunner is that you will NOT be able to figure out the amount of time it takes for each redirected request. Redirects are invisible to the user in LoadRunner. But in few test requirements you would want to know the break up of response timings between primary hit and the redirected server hits. The bigger problem comes in when you are dealing with recursive redirects.

To make an explicit redirect request you will first have to turn off automatic following of redirect instructions. This can be accomplished by setting MaxRedirectionDepth to ’0′. The key thing to remember is to identify redirected location by analyzing “Location: http://” line item in the HTTP response header of the request in question.

Example:
//In your action file
Place this in the beginning of the action file

web_set_option("MaxRedirectionDepth", "0", LAST ); //This is the key

recursiveRedirect_open();
web_url("url_which_gets_redirected","http://sampleredirect.com/",LAST);
recursiveRedirect_close();


Here is how you do it:

//place this in global.h

int HttpRetCode;
int i=0;
char depthVal[10];
char cTransactName[20000];

recursiveRedirect_open()
{
web_set_max_html_param_len("10000");

web_reg_save_param("cRedirectUrl",
"LB=Location: ",
"RB=\r\n",
"notfound=warning",
"Search=Headers",
LAST);

web_reg_save_param("cTransactionName",
"LB=https://Domain.com/",
"RB=\r\n",
"Search=Headers",
"notfound=warning",
LAST);

web_reg_save_param("httpCode",
"LB=HTTP/1.1 ",
"RB= ",
"Search=Headers",
"ORD=1",
"notfound=warning",
LAST);
}

recursiveRedirect_close()
{

HttpRetCode = atoi(lr_eval_string("{httpCode}"));
lr_output_message("xReturnCode=%d", HttpRetCode);

if(HttpRetCode == 302)//If redirect
{
i++;
web_reg_save_param("cRedirectUrl",
"LB=Location: ",
"RB=\r\n",
"Search=Headers",
"notfound=warning",
LAST);

web_reg_save_param("cTransactionName",
"LB=https://https://Domain.com/",
"RB=\r\n",
"Search=Headers",
"notfound=warning",
LAST);
web_reg_save_param("httpCode",
"LB=HTTP/1.1 ",
"RB= ",
"ORD=1",
"notfound=warning",
LAST);
sprintf(cTransactName, "Redirect_depth_%d_%s", i,lr_eval_string("{cTransactionName}"));
lr_start_transaction(cTransactName);
web_url(cTransactName, "URL={cRedirectUrl}", "Mode=HTTP", LAST);
lr_end_transaction(cTransactName, LR_AUTO);
HttpRetCode = web_get_int_property(HTTP_INFO_RETURN_CODE);
recursiveRedirect_close();

}
else
{
return;
}

}
The above code automatically generates new transactions for each unique follow redirect request’s made by the script. It keeps a track of the HTTP response code of each request and exits from recursion state when the response code is not 302.

The first challenge is to find the requests in the script which has URL redirections. If you find it hard to identify the locations where redirects are performed in the script, run the script after including web_set_option(“MaxRedirectionDepth”, “0″, LAST ); with standard log enabled. After the script completes executing, check the logs for “redirection depth exceeded” message, because whenever a web server prompts for a redirection of the resource, it sends back a 302 HTTP code with the new location, but since MaxRedirectionDepth is set to 0 in LoadRunner it will not explicitly hit the new URL. This is where the above script comes into play. Now, double click on that line, and Vugen will highlight the step where the redirection was attempted. Now place the recursiveRedirect_open() and recursiveRedirect_close() function call at the start and end of the concerned step/request and you should be good to go!

converting a C function to a floating number to string in Load Runner

I would like to call this function as ftoa

here is the code…Note: To adjust the radix/floating point position...you will have to further customize this function. 

ftoa(float floatNum, char *convFloatString) 

{

 char new[10];

 float number,dTemp,temp_val;

 int base, floatVal, radxFlag;

 char *token;

 char *temp; char cfloatVal[10],

 cBase[10]; 

 char cfloatValx[10] = "0";

 int DEBUG = 1; 

//Turn DEBUG OFF/ON by switch 0/1 

 radxFlag = 0;

 //Separate the number before and after the "." 

 number = floatNum; 

 base=number; 

 dTemp = number-base; if(DEBUG == 1) 

 { 

 lr_output_message("Base Value = %f\n", number);

 } 

 sprintf(cBase, "%d", base);

 if(DEBUG == 1) 

 { 

 lr_output_message("Floating Value = %.2f\n", dTemp);

 }

 if(dTemp == 0) //If number is a whole number then return!

 {

 lr_save_string(cBase, convFloatString); 

 return 0; 

 }

 sprintf(cfloatVal, "%.2f", dTemp);

 //Place the decimal point to suit your requirement. Default is 2

 temp = (char *)strtok(cfloatVal, "0."); 

 temp_val = atoi(temp);

 if((dTemp - 0.1) < 0) 

 radxFlag=1; 

 else

 radxFlag=0;

 if(temp_val == 0)

//If decimal values equals to 0 then return!

 { 

 strcat(cfloatVal, ".00");

 //increase the number of zero to suit your requirement. lr_save_string(cfloatVal, convFloatString); return;

 }

 if (radxFlag ==1) 

 { 

 strcat(cfloatValx,temp); strcpy(temp,cfloatValx); 

 } 

 if(DEBUG == 1) 

 {

 lr_output_message("Final decimal value = %s\n", temp); 

 } 

 if(strlen(temp) == 1 && radxFlag == 0) 

 { strcat(temp,cfloatValx); //strcpy(temp,cfloatValx); if(DEBUG == 1) 

 {

 lr_output_message("Appending a 0 %s", temp); 

 }

 } strcat(cBase, "."); strcat(cBase, temp); 

 if(DEBUG == 1) 

 {

 lr_output_message("Final decimal value = %s\n", cfloatVal); 

 }

 if(DEBUG == 1) 

 {

 lr_output_message("Final coverted floating number = %s", cBase);

 } 

 lr_save_string(cBase, convFloatString); }

Sample usage:
float floatNum; 

 floatNum = 34.102; 

 ftoa(floatNum, "convFloatStr");

 lr_output_message("Converted String = %s", lr_eval_string("{convFloatStr}"));

 return 0;

RPT Troubleshooting help

Problem:-> Test run aborted

Execution failure. No status received from location computername in interval seconds.
Explanation
The workbench has lost communication with one or more agent computers.
System action
The test run stops.
Solution :->
For each agent computer:
Check that the playback Java process is running. If it is still running, the problem might be on the workbench computer where RPT is installed. Stop the Java process and all related typeperf and vmstat processes.
Examine the problem determination log for error messages or exception messages.
If the playback Java process is not running, search for javacore.* files. The contents of these files might help you determine the cause of the problem.
Run tests again and monitor the memory size of the playback Java process. If the playback Java process is consistently running at its maximum heap size, the process might not have enough memory.
If the problem is not on an agent computer, check that the workbench computer has sufficient memory. To increase the available memory, either increase the workbench heap size or reduce the level and amount of execution history.

How to increase memory allocation for a localhost location

It is possible to see an error like: "Test Run aborted due to error"

The Driver: has become unresponsive, possibly due to an out-of-memory condition.
At last notification this driver was using 90+ percent of its allocated memory.
Please refer to the "Increasing memory allocation" Help topic for more information on how to increase memory allocation.

during playback of a Rational(R) Performance Tester (RPT) Test with many virtual users running on the test controller machine.

Cause
There are actually two heap sizes that RPT utilizes. The first one is the workbench heap, which is related to space available for controlling and monitoring the playback and reviewing results.
The second is the playback heap used by the virtual testers. This is the heap size you need to change to avoid out-of-memory conditions. When you're running the virtual testers on the same system as the workbench, there are several steps necessary to change this heap.

Solution
First you will need to create an "Agent" on the local host:

1) Decide upon a place to store the Deployment Location (either the project's root directory or a folder in the project under Test Navigator)
2) Right click at the folder from step 1 and select New and Other, then browse to Test Elements -> Location.
3) Name the Location Local and press Next and then Finish.
4) In the General Information section of the Local Location, make sure that 'Host name' is "localhost"

Problem: "Test run aborted due to error" 

Virtual users have exited prior to stage completion.
Explanation
In a schedule with multiple stages, at the end of a stage that is configured with n virtual users, there were fewer than n virtual users running. The schedule did not assign enough work to the virtual users to keep them active for the duration of the stage, or the virtual users stopped because of an a different error. The workload problem can occur if you use multiple stages but do not enclose the workload in an infinite loop.
System action
The test run stops.

Solution:
On the workbench computer, examine the workload for each user group in the schedule. Check that the entire workload is inside an infinite loop, so that the virtual users always remain active regardless of the stage duration.
If the tests already use infinite loops, then the agent computers might have encountered errors while running tests. To determine why virtual users stopped before the stage completed:
Examine the test log for error or exception messages.
Increase the test log level and decrease the number of users. Try running the schedule again. Examine the test log for error or exception messages.
Simplify the workload by running one user in one stage. Check to see whether the single user takes the expected amount of time to complete all actions. If not, examine the test log and problem determination log for error or exception messages.

Problem:" No local agent controller"

Local computer is not running the Agent Controller.
Explanation
The agent controller coordinates schedule playback between the workbench and agent computers. The agent controller is not running on the local computer.
System action
No tests run.

Solution :
Make sure that the agent controller is installed and running on the workbench computer. On Windows computers, the agent controller is a process called ACWinService. On Linux computers, the agent controller is a process called RAServer.
To start the agent controller, do one of the following steps:

1. At a command prompt, change to the directory AgentController\bin in the product installation directory.

Start->run then type cmd

>cd “C:\Program Files\IBM\SDP70Shared\AgentController\bin”

Then Typenet start "IBM Rational Agent Controller"
On Linux, change to the directory AgentController/bin in the product installation directory; then enter this command:./RAStart.sh

Problem:"Connection refused"

Connection refused on host computername
Explanation
The agent controller can be configured to restrict playback to the local computer only. If the agent controller on an agent computer is configured for local playback only, it will not accept connections from other agent controllers.
System action
The test run stops.

Solution :

Make sure that the agent controller is configured correctly.
To configure the agent controller on Windows:
2. At a command prompt, change to the directory AgentController\bin in the product installation directory.

Start->run then type cmd

>cd “C:\Program Files\IBM\SDP70Shared\AgentController\bin”

3. Type net stop "IBM Rational Agent Controller" to stop the agent controller.

4. Type SetConfig.bat to run the configuration utility.

5. Type net start "IBM Rational Agent Controller" to start the agent controller.
To configure the agent controller on Linux:
1. At a command prompt, change to the directory AgentController/bin in the product installation directory.
2. Type ./RAStop.sh to stop the agent controller.
3. Type ./SetConfig.sh to run the configuration utility.
4. Accept all of the default values except for network access mode. Set network access mode to ALL.
5. Type ./RAStart.sh to start the agent controller.

Problem:-> Unable to connect to license server
Solution :->
1) Check if "FLEXlm License Manager" service is running in Services [Control Panel>Administrator>Services: WinXP/Win2000].
Or
Start->run then type services.msc press

Restart this service "FLEXlm License Manager"

2) Launch Rational License Administrator and check if all the floating licenses are served properly [Menu Item: Setting >Client Server Config >check if Use License Server Option is checked. Check if correct server name is entered and then click on show licenses]

3) Check if Installation manager is configured for the same server.

Once above steps are verified, launch RPT go to help>license Status and check the expiration date [Should be same as seen in license server] and then run you're schedule.

Problem:-> Start recording aborted, The Agent Controller on localhost: 10002 is unavailable

Symptom

When you start a recording in RPT, the following error message can occur:

Start recording aborted.

Reason:
IWAT3034E The Agent Controller on localhost:10002 is unavailable.

IWAT3035E Confirm the Agent Controller is still running, and verify the location in Window > Preferences > Test.

Solution :->

This problem typically indicates that the installation of the Remote Agent Controller (RAC) is not correct. If it is and if the IBM_Agent_Controller directory is present, RAC does not start because of other issues. The following suggestions could help to overcome the problem:
Check the IBM Rational Agent Controller service.
In the Tab Log on, it should read:
Local System Account
Allow service to interact with desktop

Check RPT Windows > Preferences > Test.
The Agent Controller local port should be 10002.

Check the serviceconfig.xml file
Path :-> C:\Program Files\IBM\SDP70Shared\AgentController\config

Open the serviceconfig.xml and verify the followings
Make sure that the Agent Controller uses port 10002
One of these two lines must exist:


or alternatively

Check the C:\Windows\System32\drivers\etc\host file.
Make sure it contains the exact line:127.0.0.1 localhost
Bring up RPT

Windows > Preferences > Profile and Logging > Hosts.
Select the localhost: 10002 selections in the Default Hosts list.
Click the Test Connection button.

Open a command prompt,

Run the command
netstat -a -o
Look for a line like:
TCP hostname:10002 hostname:0 LISTENING 3528.
Hostname is the actual name of the computer. The right-most value in this line (3528 above) should match the process ID (PID) for the RAService.exe process.
From a command prompt, run the command
telnet localhost 10002
Note any error message that appears.

Make sure, that no firewall or other security application prevents it to start.

Common paths in Loadrunner

C:\Program Files\HP\LoadRunner\bin\certs
you can make changes in index file for Certificate entry

C:\Program Files\HP\LoadRunner\config

open wlrun7 and you will see all default setting of loadrunner.

Like default script path, result path, timeout, etc...

AgentMaxThreadsPerDriver :It is where you can specify number of threads per process(mdrv).

HostConnectTO,HostDisconnectTO etc.

you can make changes and restart Loadrunner.You can see recent scripts on the path
C:\Program Files\HP\LoadRunner\config

For Error file
C:\..Script-Name\logfile

Execution detail
C:\..Script-Name\output.txt

C:\..Script-Name\data\
Recordinglog
CorrelationLog
codeGenerationLog
also..downloaded image will be saved under data

 Result Set
C:\..Script-Name\Result

 Recorded Script
C:\..saved-Script-Name\

e.g script name is dataEntry.usr. can be launched script by clicking on it.

LR Controller online Graphs Directory to modify

1. By default, the Controller online monitor shows a maximum of 20 measurements for each graph. To increase it, go to the LoadRunner\dat\online_graphs directory to modify the value ofMaxDispMeasurments= in the file controlling each type of graph:

Description------------------------------------------------------------File Name
All---------------------------------------------------------------------- generalsettings.ini
System Resource Graphs -------------------------------------------online_resource_graphs.rmd
Runtime Graphs ------------------------------------------------------online_runtime_graphs.def
Transaction Graphs--------------------------------------------------- online_transaction_graphs.def
Web Resource Graphs----------------------------------------------- online_web_graphs.def
Streaming Media ------------------------------------------------------online_web_graphs_mms.def

2. Default counters for the System Resource, Microsoft IIS, Microsoft ASP, or SQL Server monitors are defined in the res_mon.dft file within the LoadRunner/dat folder. Its values can be pasted from the [MonItemPlus] section within scenario .lrs files.

Performance goal be in terms of concurrent users?

We often come across of  a question of how many users would require for performance testing.

1. First Approach:

Assume that we have 5000 average visitors per day and working window is 12 hours
So calculate like this ....

Average visits per hour = (5000 average visitors/day) / 12 hours = 417

Average Concurrent Users = Visits per hour / (60 min/hour / average visit.

2. Second Approach

This approach is going to tell us how many users we have on average.

U = V / (60/D)

Where:
U is the number of load test virtual users (that's what we are trying to figure out)
V is the average number of visitors per hour
D is the average duration of a visitor
60 is the number of minutes in an hour

Also , we can use

Users = (Pages views per hours * Avg. duration of script execution)/60 * Total No. of pages in the script

Web Services Performance Testing

Web Services Performance Testing using SOAPUI

Recently i had a chance to recommend a performance testing tool to carryout performance testing for web services system. I evaluated a series of open source and licensed tools and finally concluded SOAPUI would be meet the requirement for our system.

soapUI provides extensive load testing functionality allowing you to do the following:

Functional LoadTesting : validate functionality under load using standard TestCase methods.
Behavioral LoadTesting : analyze performance behaviour under varying load with different load strategies.
Performance LoadTesting : find maximal performance available using thread strategies and Command Line LoadTest execution.

Requirements Driven LoadTesting : define performance requirements and continuously validate using Load Test assertions.

SOAPUI helps in the following.

> Validate a Web Services performance under different Load scenarios.
> Maintain Functional validations to see that they don't "break" under load.
> Run several load tests simultaneously to see how they affect each other.

Its a freeware tool which best suits web services load testing. Its commercial version is called SOAPUI Pro includes productivity enahancements.

In Overall SOAPUI meets most the requirements for a Web services performance testing and it is a apt tool to meet most of the web services.

Web Log Analysis

A typical issue which i faced during web log analysis...

While conducting Performance testing for applications which are falling into the category of Post-Production case, the most important activity that needs to be done is web log analysis. The web server logs are the gold mines which hold all the information about the server during the load condition.

During the web log analysis, look for the peak hour traffic time and identify what is the server load in terms of hits/sec and in terms of users. Based on this statistics, derive performance test goals by considering the business forecasts.

I would like to share a typical issue which I faced while doing web log analysis for a typical post production category of an application. The application under test uses a shared web-app server (IIS) in which there was 2 other applications deployed. Now, the problem is the web server log files have the traces of all 3 applications together as these files are common for a server and not specific for each of the applications running on the server.

The web log analyzer tool which I was using doesn’t have any option to set filters. In that case, I tried using a web log analyzer tool (123LogAnalyzer tool) which provides a filter option to filter out the traces pertaining to a specific application and export the filtered log file into a text file. For example, if the application under test has a context root as ‘methodologies’ (http://132.23.34.21/methodologies/......), then try setting up this URL in the filter and create the text files for your application of interest.

Now you could use the newly generated text file to feed it into any of the log analyzer tool which you are confident enough and proceed with your performance goal setting. My kind advice is that before choosing the log analysis tool of your choice, analyze the accuracy of log analysis thoroughly.

A Comprehensive Report on Web Log Analysis

In today’s dynamic world, the IT sector of all the businesses want to use more sophisticated techniques to support their application scalability. All the web site owners are interested to know the usage pattern of their site in order to make any business decisions. The necessity to know the visitor profile is very high in any web site as most of time the visitors are anonymous & unpredictable for any open systems (which uses internet).

The Web analytics activity deals with measuring the end user behaviors on a web site. Of the two major web analytics techniques (Web server log analysis & Page Tagging) available, this paper deals with the Web Server Log Analysis concepts. This paper provides an overview of various log analyzer tools available in the market & their comparative ratings.

The Web log analysis is used by various types of users with different backgrounds. The different sets of users are interested to know about specific data from the web server log. This has lead to the development of lot of log analyzer tools with specific extra features, though the basic features are available in most of the tools.

Is the system Response time directly / inversely proportional to the server utilization?(Response Time Vs Queue Length Vs Server Utilization)

 Will there be queues(for CPU/Memory/Disk) if Utilization is less than 100% 

Actually speaking, there is no correlation between system response time and the server utilization. The Response time increase might happen even when the server is less utililised. 

Most of us dont understand this relationship. Increase in response time (during high load) is caused only because of queueing of the requests. The user request arrival pattern contributes to the high response time. Due to the adhoc user request arrival pattern, long queues are formed in various service centers which leads to high response time.

Response Time - What are the various components of Response time?

Waiting time (Queuing time) + Processing time = Response Time.

For a server, the processing time is always the same irrespective of load. For example, the server might take 2 seconds of processing during 1 user load and during 1000 users load for a transaction. But the waiting time would be high during 1000 users load, which drastically increases the overall response time. So, response time increases, if the waiting time increases which is caused by long queue.

Also, dont think that there will be queues only if Utilization is more than 100%. There can be queues formed even when the Utilization is less. So, everything boils down to knowing the user arrival pattern of the system. This is going to determine, when the system is going to be loaded and break.

Hence, knowing the user arrival pattern of a system is very important rather than setting mere goals to perform load/stress test. Analysing the user arrival pattern of an application would help in setting realistic goals which becomes more of organization logistics issue and most organizations bypass and pay for it later.

WCF Services Performance Testing

Most of the market available performance testing tools available for web services performance testing does not provide support to WCF (Windows Communication Foundation) web services performance testing.

The following are the tools which supports WCF web servcies performance testing.

•  WCFStorm
• WCF Load Test
•  SOAtest

SOAPUI doesnt support WCF web services performance testing. Out of above 3 tools, we have finlized on the SOAtest 5.5 tool to carryout WCF web services performance testing.

Learn about Think Time in Load runner

I got a chance to interact with lot of performance testers in various organizations , not only in India, but also in abroad & found that most of us are interested in spending time for setting realistic performance test goals & identifying the frequently used business scenarios (business flows) , but very few analyze the think time of various business flows.

It becomes very important that the average think time of the end users of the application needs to be analyzed rather than estimating specific time limit by counting the total fields to be filled by the end user. For the same application(lets take Yahoo new account creation), a new user might take 4-5 minutes or more to fill the required details to create a new account. But at the same time, a frequent old user, might take 1-2 minutes to create a new account. These think times (4-5 minutes for new user & 1-2 minutes for old user) might impact the server load.

Lets assume 100 new users are accessing the yahoo application for the time interval 13.00 to 14.00 hours & 100 old users are accessing the application for the time interval 14.00 to 15.00 hours continuously. Though there are 100 users (100 active sessions created) load on the server during both 13.00 to 14.00 hours and during 14.00 to 15.00 hours, the actual server load(requests) is going to be different during these 2 hours.

When you have a close look at the server load in requests/sec (Hits/sec) , you will find that the server load during 13.00 to 14.00 hours might be lesser than 14.00 to 15.00 hours, because of high think time used by the new users. If you look at the quantified numbers, you will be shocked to know that during the first hour, the average load could be apprx 12 hits/sec whereas during the second hour, the server load could be around 25 hits/sec.

Having performance test goals in mere user load alone doesnt make the test more realistic. Detailed analysis on the end user's think time makes it more realistic.