Prerequisites for load Testing in load runner

Important Set up before load testing includes:

A suitable time to load-test the application, for instance when no development work is taking place on the server (load testing may cause the server to crash) and/or no other users are accessing the server (else the testing results would not yield the correct measures)

1.  The performance metrics, accepted levels, or SLAs and goals
2.  Objectives of the test
3.  The Internet protocol(s) the application is(are) using (HTTPS, HTTP, FTP, etc.)
4. If your application has a state, the method used to manage it (URL rewriting, cookies, etc.)
5.  The workload at normal time and at peak time
6. Before load or performance testing application should be stable at least tested at once through the manual method
7. Before load testing, do not record any page, which has 404 or server errors
8. During load test don’t do real transaction or any money ready work.
9.  During load testing try to maintain the real scenario as user experience
10.  Use meaningful test scenarios (use cases are helpful) to construct test plans with 'real-life' test cases.
11.  Make sure that the machine running Load testing tool has sufficient network bandwidth, so the network connection has little to no impact on the results.
12.  Let Load Test run for long time periods, hours or days, or for a large number of iterations. This may yield a smaller standard deviation, giving better average results. In addition, this practice may test system availability rate and may highlight any decay in server performance.
13.  Ensure that the application is stable and optimized for one user before testing it for concurrent users.
14.  Incorporate 'thinking time' or delays using Timers in your Load testing scenario Test Plan. 
15.  Keep a close watch on the four main things: processor, memory, disk, and network.
16.  Only run Load testing tool against servers that you are assigned to test, else you may be accused of causing DoS attacks.

"Object reference not set to an instance of an object" analysis error in load runner

This error comes when the following files in (Analysis Install dir)\bin\dat\ are corrupted:

loader2.mdb, loader.mdf, loader.ldf
Solution:

You need to restore these files; take these files from another machine where you have installed the Analysis. Or Re-install the Analysis on your machine after a clean uninstall.

What is Security Testing?

Security testing is related to the security of data and the functionality of the application. You should be aware of the following concepts while performing security testing:

1. Confidentiality - The application should only provide the data to the relevant party e.g. one customer's transactional data should not be visible to another customer; the irrelevant personal details of the customer should not be visible to the administrator and so on.

2. Integrity - The data stored and displayed by the application should be correct e.g. after a withdrawal, the customer's account should be debited by the correct amount.

3. Authentication - It should be possible to attribute the data transmitted in the application to either the application or the customer. In other words, no one other than the customer or the bank should be able to create or modify any data.

4. Authorization - The application or a user should only be able to perform the tasks which they are respectively authorized to perform e.g. a customer should not be able to withdraw more than the balance in their account without having an overdraft facility, the application should not be able to levy charges on a customer account without prior customer approval.

5. Availability - The data and functionality should be available to the users throughout the working period e.g. if the bank's operating times are from 8 a.m. to 8 p.m. on all working days, it should be possible for a customer to access their account and make the necessary transactions on their account.

6. Non-repudiation - At a later date, it should not be possible for a party to deny that a particular transaction or data change took place e.g. if a customer withdraws an amount from their account, this should trigger the relevant actions (posting to their transaction records, debiting their account and sending them a notification etc.).

In your question, you mentioned that you wish to avoid any data breach by hackers. You should understand that hackers are not the only people from whom the application functionality and data need to be protected. There are other people that you need to consider as well:

1. Disgruntled customers
2. Unhappy or malicious employees of the bank
3. Unprofessional service providers e.g. an unprofessional hosting company that may have access to the application and the data
4. Unprofessional auditors

Further, since financial data is so important, banking applications in certain countries have to be compliant to the relevant financial standards. Research the relevant standards that your application needs to follow.

Creating a secure application involves a lot of work in designing a secure application and designing a secure data store. Even after deployment, the application should be closely monitored to ensure that the data is being accessed by only the authorized people. If any security breach is reported, it should be analyzed carefully and the loopholes plugged.

Now, let us discuss the actual security testing. You should design security tests based on at least the following:

1. Stated security requirements
2. Security-related standards that the application should follow Assuming that it is a web application,
3. Common vulnerabilities found in web applications
4. Different browser versions on different operating systems (here you should note that implementing security only on the client-side may not suffice)

In your initial tests, you may want to use automated testing tools e.g. web vulnerability scanners, password crackers, web proxy tools etc. Based on your learning, you may want to execute the more complex security tests by hand. Keep yourself updated about the latest hacks and test them on your application before every release.

As you might now appreciate, security testing is a vast area of knowledge and practice. In order to do justice to security testing, it is better to have a dedicated team for security testing.

Creating Dynamic itemdata in Web_submit_data in load runner

I was working on a project where there was a scenario of searching for a course by its first name or wild cards. The search returns different number of results for different queries. This request is followed by another web_submit_data where the ITEMDATA depends on number of search results.

There are three dynamic variables here – length of the course variable, results returned and the third one being the ITEMDATA of the request itself.

The solution is to build dynamic data on the fly and create a web_custom_request for the web_submit_data

The following function will build the item data -

const char* BuildItemData(int pageSize,const char*ItemDataStart,const char*ItemDataEnd, int lengthOfCourseVar, const char* ArrayName){

char* ItemData;

int iCount,lengthOfOneItem;

lengthOfOneItem = lengthOfCourseVar+strlen(ItemDataStart)+strlen(ItemDataEnd);

ItemData = (char*)malloc((pageSize*(lengthOfOneItem))*sizeof(char));

ItemData[0]=”;

for(iCount=1;iCount<=(pageSize);iCount++)

{

strcat(ItemData,ItemDataStart);

strcat(ItemData,lr_paramarr_idx(ArrayName,iCount));

strcat(ItemData,ItemDataEnd);

}

return ItemData;

}
I changed the web_submit_data to web_custom_request and used the string output as a parameter like this:

lr_save_string( BuildItemData(lr_paramarr_len(“cor_Arr_ClassIDs”),”offId[]=class”,”&”,strlen(lr_paramarr_idx(“cor_Arr_ClassIDs”,1)),”cor_Arr_ClassIDs”),
“ItemDataOnTheFly” );

web_custom_request(“xxxxxxx.aspx”,

“URL=https://{par_Environment_URL}/xxxxxx/xxxxxx.aspx”,

“Method=POST”,

“TargetFrame=”,

“Resource=0″,

“RecContentType=text/html”,

“Referer=https://{par_Environment_URL}/xxx/xxx/xxx/xxxx/xx/xxxxxx?xxxxxx={par_Location}”,

“Snapshot=t12.inf”,

“Mode=HTML”,
“EncType=application/x-www-form-urlencoded; charset=utf-8″,

“Body={ItemDataOnTheFly}&lrnid=xxxxx/{par_UserID}”,

LAST);

Run Time Settings in Load runner

Generally  load runner runtime settings plays a crucial role in Vugen scripting and scenario running. The run time settings are

General

• Run Logic 
• Pacing 
• Log 
• Think Time 
• Additional attributes 
• Miscellaneous 

Network

• Speed simulation

Browser

• Browser emulation

Internet Protocol

• Proxy
• preferences
• download filters
• Content check

Data Format Extension

•  Configuration

General 

Run Logic 

Whenever I am using a Vuser type that allows multiple actions in a single script, I will create a separate action for each business process and put appropriate percentage weightings on each action. It is very unusual to have to do anything more complicated than this. I don’t usually use the “sequential” option or create blocks unless I need to have fractional percentage weightings for a business process.
Percentages must be integer values, so to run a business process 0.1% of the time you could create a block that runs 1% of the time, and put an action in the block that runs 10% of the time.
It’s also rare to set a script in a scenario to run for a specified number of iterations (mostly done by time or set to run indefinitely). Generally “number of iterations” is only used when running the script in VuGen. 

Pacing :

“As soon as the previous iteration ends” is used when running in VuGen or when loading/verifying data. Do not use this for load testing
I have never seen the point of the “After the previous iteration ends” option. Why would you want to run an unknown number of transactions per hour against the system?
Don’t use the “At fixed intervals”. If something causes your users to become “in step”, they will tend to stay that way and continue to all hit the server at the same time.
“At random intervals” is definitely the way to go. Obviously for your users to create a certain number of orders per hour the iteration time must average to 3600/num iterations in an hour. Do not make the lower boundary value any bigger than the maximum time it takes to complete the business process, or you will end up creating less transactions per hour than you intend to. 

you can check how to calculate pacing according to the requirement from this blog also.

Log :

Enable logging:once you verify that your script is functional,disable logging to conserve resources.
Logging creates additional overhead on your load generators, and can create huge log files. 




 log absolutely everything when debugging in VuGen.
When running the script as part of a scenario, I leave extended logging on but change the logging to “Send messages only when an error occurs”. This gives a little more information than turning logging off entirely, and won’t create any additional overhead while everything is running smoothly (and if the system is not running smoothly you are going to need to stop the test and investigate anyway). 

Standard log:sends a subset of functions and messages sent during script execution to a log.The subset depends on the Vuser type.

Think Time:



Just like the pacing setting, I think that it is a good idea to put some randomness in your think times.
I use a random percentage of 50-150% of recorded think times.
Use “Ignore think time” if you are debugging in VuGen or if you are loading/verifying data. 

Additional attributes :

This option is ignored by most people. It is used to create a parameter with a given value without having to edit the script (as runtime settings can be overridden in the Controller).
In the screenshot I have created a parameter of ServerName with the address of the test envioronment. If you were testing in more than one test environment at a time, this would make save some time. 

Miscellaneous :




Continue on error is generally only going to be used if you have written code to do something when you encounter an error. Usually the default behaviour of ending the current iteration and then starting the next one is sufficient). I don’t advise anyone to try to write a script that handles errors in the same way as a real user because it will create a lot of additional work for very little benefit, but doing something simple like writing some useful information to the logs and then calling lr_exit(LR_EXIT_ACTION_AND_CONTINUE , LR_FAIL) can be useful.
“Fail open transactions on lr_error_message” should always be ticked. If you are raising an error, you should fail the transaction step that you are performing.
“Generate snapshot on error” is useful. If it is a web script, any error messages should be added to your content check rules.
Run your virtual user as a thread unless you have code that is not threadsafe or there is some other reason to run your virtual users as a process. The overall memory footprint on your load generators will be higher if you run as a process.
I never use the “Define each action as a transaction” option. If I want a transaction in my script I will add it myself with lr_start_transaction.
I never use “Define each step as a transaction” either. If it is a web script, I can use the transaction breakdown graph to get this information, otherwise I will add the transactions myself. 

Network :

Not all vuser types have this option available.
Most of the time my virtual users will use the maximum bandwidth.
If I want to emulate users with bandwidth constraints, I will do this in a separate scenario.
Google calculator is handy to calculate bitrates if your bitrate is not available from the drop-down list e.g./ “256 Kbps in bps”
All of the following settings only apply to web-based scripts. Each vuser type will have its own runtime setting options. It is important to know what they mean and how they will influence your
test results before running any tests that you plan to report on. 

Browser

Browser Emulation: 




Some people get confused by the User-Agent (browser to be emulated) setting. If 90% of your users use Internet Explorer 6.0 and the rest use Firefox 1.5, you don’t have to change the runtime settings for your users to match this. All it changes is the string that is sent in the “User-Agent” field of your HTTP requests. This is completely pointless unless your application has been written to serve different content to different browsers based on the User-Agent field.

Internet Protocol:

Proxy : 




Generally people won’t be using your web applications through your proxy server, so it shouldn't be part of your test either.
If you start getting errors that are due a proxy server rather than the system under test, it will just confuse the people who have to fix the problem.
A proxy server will also make IP-based load balancing ineffective.
If it’s an intranet application and everyone will be using the application through the company’s proxy, then the proxy server should be explicity declared to be in scope for your load test. You should make sure that you have an identical proxy server for your test environment, or that you have permission to be generating load on a piece of Production infrastructure. 

Preferences :




These settings are default values specified by HP, rather than being inherited from the web browser that is installed on your workstation. Generally you will not need to change them, but be aware that they are here. 

Download Filters: 




Download filters are a quick way of preventing your scripts from downloading content from certain URLs or hosts/domains.
I generally use this feature when the web application in the test environment contains third-party images used for tracking website usage (e.g. images from Webtrends or Red Sheriff etc).
I think it is better to specify which hosts your script is allowed connect to, rather than which hosts your script can’t connect to (because it’s easy to miss one accidentally, or the application may change and refer to a new third-party domain).
Use web_add_auto_filter if you want to specify this in your script rather than your runtime settings.
Data Format Extension:

Configuration :



A LoadRunner feature that has made my life a lot easier has been ContentCheck rules, which are available in the script runtime settings. If you are using a web-based vuser type, you can configure your Load Runner script to search through all returned pages for strings that match known error messages.

Using web_reg_find functions is fine, but when you get an error LoadRunner reports it as “failed to find text” instead of something more descriptive.

I will always create rules for any error messages I find during scripting and, if I receive an error while running a scenario, I will add the error message from the snapshot in the scenario results directory (the snapshot on error feature is very useful).

All this is pretty obvious if you have taken the time to explore LoadRunner’s features or you have attended a Mercury training session, but I recommend taking things a step further.
Ask your developers for a list of all the error messages that the application can throw. This should be easy for them to provide if the application is well designed and stores all the message in some kind of message repository instead of sprinkling them throughout the source code.
Include error message for functional errors that you are likely to encounter. Creating a rule for “incorrect username or password” may save someone 20 minutes of investigation when they first run the script after the database has been refreshed.

If you prefer to have error message you are checking for in the script (where you can add comments to them) instead of the runtime settings, you can use the web_global_verification function instead. The only difference between the two is the error message that LoadRunner will include in its log:

Action.c(737): Error -26368: “Text=A runtime error occurred” found for web_global_verification (“ARuntimeErrorOccurred”) (count=1), Snapshot Info [MSH 0 21]

…compared to:

Action.c(737): Error -26372: ContentCheck Rule “ARuntimeErrorOccurred” in Application “Webshop” triggered. Text “A runtime error occurred” matched (count=1), Snapshot Info [MSH 0 21]

JVM garbage collection in young generation

It comes from Java SE 6 HotSpot[tm] Virtual Machine Garbage Collection Tuning , one stop place to learn everything about GC internals. But to address your immediate questions:

Allocating new objects using new operator (almost) always happens in Eden space. But Eden is actually a stack. When you create new object needing N bytes, single pointer advances by N bytes on that stack and that's it. Allocating is that fast, no searching for free spot, compacting, whatever.

Of course this stack is not infinite, at some point we'll reach its end, triggering minor GC. Also most likely multiple objects are already garbage. So what JVM does in minor GC is the following:

traverse graph of objects starting from GC roots

copy all objects reachable from GC roots to one of survivor spaces (no gaps, we know all of them and this is a single process)

wipe out eden space (basically just moving this stack pointer back to 0)

In subsequent minor collections there are additional steps:
one of survivor spaces is examined as well. Live objects from both eden and one of survivor spaces are copied to second survivor space. This means there is always exactly one free survivor space.

So how are objects ending in tenured generation? First young objects are copied to one of survivor spaces. Then they are copied to the other and again and again. Once given object jumps back and forth too many times (configurable, 8 by default), it is promoted to tenured space.

Major GC runs when tenured space is full.

Java (Java Virtual Memory) Memory Types

Java (JVM) Memory Types:

JVM memory area related jargons are key to understand the JVM on the whole. In this article let us discuss about the important memory areas in JVM.
Heap Memory

Class instances and arrays are stored in heap memory. Heap memory is also called as shared memory. As this is the place where multiple threads will share the same data.
Non-heap Memory

It comprises of ‘Method Area’ and other memory required for internal processing. So here the major player is ‘Method Area’.
Method Area

As given in the last line, method area is part of non-heap memory. It stores per-class structures, code for methods and constructors. Per-class structure means runtime constants and static fields.

Memory Pool:
Memory pools are created by JVM memory managers during runtime. Memory pool may belong to either heap or non-heap memory.

Runtime Constant Pool:

A run time constant pool is a per-class or per-interface run time representation of the constant_pool table in a class file. Each runtime constant pool is allocated from the Java 

virtual machine’s method area.

Java Stacks or Frames:

Java stacks are created private to a thread. Every thread will have a program counter (PC) and a java stack. PC will use the java stack to store the intermediate values, dynamic linking, return values for methods and dispatch exceptions. This is used in the place of registers.
Memory Generations

HotSpot VM’s garbage collector uses generational garbage collection. It separates the JVM’s memory into and they are called young generation and old generation.

Young Generation

Young generation memory consists of two parts, Eden space and survivor space. Shortlived objects will be available in Eden space. Every object starts its life from Eden space. When GC happens, if an object is still alive and it will be moved to survivor space and other dereferenced objects will be removed.

Old Generation – Tenured and PermGen

Old generation memory has two parts, tenured generation and permanent generation (PermGen). PermGen is a popular term. We used to error like PermGen space not sufficient.

GC moves live objects from survivor space to tenured generation. The permanent generation contains meta data of the virtual machine, class and method objects.

Discussion:
Java specification doesn’t give hard and fast rules about the design of JVM heap data area. So it is left to the JVM implementers and they can decide on things like whether to allocate fixed memory size or dynamic.

Key Takeaways

• Local Variables are stored in Frames during runtime.
• Static Variables are stored in Method Area.
• Arrays are stored in heap memory.

JVM garbage collection in young generation?

It comes from Java SE 6 HotSpot[tm] Virtual Machine Garbage Collection Tuning , one stop place to learn everything about GC internals. But to address your immediate questions:

Allocating new objects using new operator (almost) always happens in Eden space. But Eden is actually a stack. When you create new object needing N bytes, single pointer advances by N bytes on that stack and that's it. Allocating is that fast, no searching for free spot, compacting, whatever.

Of course this stack is not infinite, at some point we'll reach its end, triggering minor GC. Also most likely multiple objects are already garbage. So what JVM does in minor GC is the following:

traverse graph of objects starting from GC roots

copy all objects reachable from GC roots to one of survivor spaces (no gaps, we know all of them and this is a single process)

wipe out eden space (basically just moving this stack pointer back to 0)

In subsequent minor collections there are additional steps:
one of survivor spaces is examined as well. Live objects from both eden and one of survivor spaces are copied to second survivor space. This means there is always exactly one free survivor space.

So how are objects ending in tenured generation? First young objects are copied to one of survivor spaces. Then they are copied to the other and again and again. Once given object jumps back and forth too many times (configurable, 8 by default), it is promoted to tenured space.

Major GC runs when tenured space is full.

What is running Vuser scripts in Stand-Alone Mode in load runner

Running Vuser Scripts in Stand-Alone Mode:

In order to perform load testing with a Vuser script, you need to incorporate the script into a LoadRunner scenario. Before integrating the script into a scenario, you should check its functionality by running the script in stand-alone mode.

Stand-alone mode:

Running a script in stand-alone mode means running the script without using a LoadRunner Controller. This is done to establish how the script will execute when run from the Controller. If the script is Windows-based, you use VuGen to run it in stand-alone mode. If the script is UNIX-based, you must run it from a UNIX command line.

If the stand-alone execution is successful, then incorporate the script into a scenario.