The below post includes what kind of questions a Performance Engineer may encounter during Analysis session for a detailed oriented and curious client.
Performance Engineer:
I provided the Transaction Response time summary Report and shown different Graphs to the customer
Client:
I would like to have the raw transaction times (number shown in "Pass" column). This means a Transaction passed for 50 times during the test, they want to see the response times of each of the 50 passed transactions.
Performance Engineer:
I provided the Response Times of the two transactions client requested with time stamps.
Performance Engineer:
I provided the Transaction Response time summary Report and shown different Graphs to the customer
Client:
I would like to have the raw transaction times (number shown in "Pass" column). This means a Transaction passed for 50 times during the test, they want to see the response times of each of the 50 passed transactions.
Performance Engineer:
I provided the Response Times of the two transactions client requested with time stamps.
Client:
What is the difference between first Buffer Time and Over All Response time?
Performance Engineer:
The First Buffer Time is always less than the Overall Response time
what we see in the Response times graph is the TIME When Response Completed. It will be obviously more than that of First Buffer Time.
Please refer the below diagram.
Client:
Thank you for the raw data and the above explanation Could you please verify that all data points were included as I cannot match the numbers in the spreadsheet? On the spreadsheet it shows that there were 19 data points for the Transaction1 and 18 data points for Transaction2 yet only 17 data points are available for each in the Raw Data why?
In addition, I cannot see the corresponding entry that matches the value of the maximum number displayed in the summary spreadsheet for either transaction.
The only number that I can currently match is the minimum number. Please let me know if there are other data points that need to be included.
Performance Engineer:
I provided the raw data for the login the Graph. Now I am getting you the Real Raw data from data points.
There is a difference between these two raw data (Real Raw data and Graphed Raw Data)
For Example: In Real raw data, you see two values at 36 seconds. During that 36th second of the Test run two users took 26 sec and 23 seconds respectively.
The Graphed Raw data shows the average of these response times that happened at fraction of seconds. (Because our granularity is only 1 second We cannot go more granular than 1 second)
Average of 26.738 and 23.211 = 24.975. This is what you are seeing in the Graph and But in summary you are seeing 26.738. This value is based on Real Raw data.
Coming to the question, Why we are not able to map the max values to the Maximum data point in Summary? The Web Page component Break down graph is also Average of different users activity at different point. So if you try to map the Max value in Summary with Max value in Web Page Break down Graph we are not mapping same values. Because, they might not occurred at the same time.
What the Response Time Summary shows is pretty much clear. 90 percentile shows how many transactions are below that particulat data point. If the 90 percent column is exceeding your SLA, then we have to correlate it with the Standard Deviation. If deviation is high. Then we need to go to the specific Graph and observe the trend to see where the deviation occurred.