What are the possibilities in Vugen?
1. The application that you are replaying against is not running.
Simple? But it does happen, as such, when it fails on replay, take some time and try logging into the application manually and perform the business processes.
2. There were changes to the application.
There were changes to the application in aspect that had affected the script. For a 3-tier architecture, anything could have changed such as the following:
a. Presentation layer: where links or buttons are removed,
b. Application layer: where methods are changed or removed,
c. Database layer: where calls or tables are changed or removed.
d. Architecture level: where proxies or load balancers are changed or removed.
As such, verify with the application and infrastructure team if there were any changes made. You may have to re-script or amend the script accordingly to the changes.
3. There are dynamic values that need to be correlated.
This is common in applications. They are usually in the form of Session IDs or returned values based on parameterization. You have two options here:
a. Use the auto-correlation feature to detect any dynamic values (which I don’t usually use) or,
b. Perform manual correlation by recording two scripts of the same business process. From the two scripts, compare the recording logs and the APIs that had been generated for the differences.
4. The parameterization may be causing the problem.
The parameterization may be causing the problem in the following manner:
a. Insufficient records: ensure that you have sufficient records for the replay, especially in iterations.
b. Incorrect records: verify with the application or database team that you are holding on the valid values to be passed in as parameters. Parameterization may also result in different set of data being returned by the server as mentioned previously in (3). If that happens, correlate the value accordingly.
For (1) to (4), it’s recommended to turn on Full Extended Log (all options enabled: Advanced Trace, Parameter Substitution, Data Returned From Server) in the Runtime Settings (Vugen) to verify the data that is been transmitted between the server and the client (script). Through this, you can find out what and where that could have gone wrong in the replay.
What are the possibilities in Controller?
5. Does it happen at the start of the scenario execution?
We are focusing on the script only and leaving out the load testing setups. If the script/vuser fail at the start of the execution, verify the script again in Vugen, and if it still fails; it’s a script problem where you have to re-work from the previous point (1) to (4).
6. Does it fail on a particular Load Generator (LG)?
This may be caused by inconsistent LG across the load testing environment. As such, ensure the following in order for the script to run properly (but not limited to the list):
a. Folder directories that the script may be accessing to (particularly for file upload and download business processes).
b. Java settings like JVM and CLASSPATHs that can affect Java LoadRunner scripts.
c. All LGs have the same version as the Controller (e.g. LR9.0 across all LGs in the load testing environment).
7. Does it happen at the middle of the scenario execution?
If it is the case, it may be caused by anything but unlikely script problems (since the script/vuser have already started running successfully at the start of the run in (5). When this happens, the Application under Test (AUT) maybe under load and unable to process all requests from the scripts/vusers and therefore returning errors to them. You can verify this with the following.
a. Manually log into your AUT again and verify if it is still running or experiencing any lags.
b. Reduce the amount of vusers being generated in the scenario and re-run the test. If a reduced amount results in an error-free or non-script related errors, you are sure that the AUT is under load.
The above should be sufficient for you to troubleshoot script/vuser problems in Vugen and Controller. However, take note that it may not be limited to those and it will be advisable to carefully work the problem step-by-step to eliminate and possibilities
1. The application that you are replaying against is not running.
Simple? But it does happen, as such, when it fails on replay, take some time and try logging into the application manually and perform the business processes.
2. There were changes to the application.
There were changes to the application in aspect that had affected the script. For a 3-tier architecture, anything could have changed such as the following:
a. Presentation layer: where links or buttons are removed,
b. Application layer: where methods are changed or removed,
c. Database layer: where calls or tables are changed or removed.
d. Architecture level: where proxies or load balancers are changed or removed.
As such, verify with the application and infrastructure team if there were any changes made. You may have to re-script or amend the script accordingly to the changes.
3. There are dynamic values that need to be correlated.
This is common in applications. They are usually in the form of Session IDs or returned values based on parameterization. You have two options here:
a. Use the auto-correlation feature to detect any dynamic values (which I don’t usually use) or,
b. Perform manual correlation by recording two scripts of the same business process. From the two scripts, compare the recording logs and the APIs that had been generated for the differences.
4. The parameterization may be causing the problem.
The parameterization may be causing the problem in the following manner:
a. Insufficient records: ensure that you have sufficient records for the replay, especially in iterations.
b. Incorrect records: verify with the application or database team that you are holding on the valid values to be passed in as parameters. Parameterization may also result in different set of data being returned by the server as mentioned previously in (3). If that happens, correlate the value accordingly.
For (1) to (4), it’s recommended to turn on Full Extended Log (all options enabled: Advanced Trace, Parameter Substitution, Data Returned From Server) in the Runtime Settings (Vugen) to verify the data that is been transmitted between the server and the client (script). Through this, you can find out what and where that could have gone wrong in the replay.
What are the possibilities in Controller?
5. Does it happen at the start of the scenario execution?
We are focusing on the script only and leaving out the load testing setups. If the script/vuser fail at the start of the execution, verify the script again in Vugen, and if it still fails; it’s a script problem where you have to re-work from the previous point (1) to (4).
6. Does it fail on a particular Load Generator (LG)?
This may be caused by inconsistent LG across the load testing environment. As such, ensure the following in order for the script to run properly (but not limited to the list):
a. Folder directories that the script may be accessing to (particularly for file upload and download business processes).
b. Java settings like JVM and CLASSPATHs that can affect Java LoadRunner scripts.
c. All LGs have the same version as the Controller (e.g. LR9.0 across all LGs in the load testing environment).
7. Does it happen at the middle of the scenario execution?
If it is the case, it may be caused by anything but unlikely script problems (since the script/vuser have already started running successfully at the start of the run in (5). When this happens, the Application under Test (AUT) maybe under load and unable to process all requests from the scripts/vusers and therefore returning errors to them. You can verify this with the following.
a. Manually log into your AUT again and verify if it is still running or experiencing any lags.
b. Reduce the amount of vusers being generated in the scenario and re-run the test. If a reduced amount results in an error-free or non-script related errors, you are sure that the AUT is under load.
The above should be sufficient for you to troubleshoot script/vuser problems in Vugen and Controller. However, take note that it may not be limited to those and it will be advisable to carefully work the problem step-by-step to eliminate and possibilities
No comments:
Post a Comment