I’ve come to a juncture (read voluntary unemployment) where I’ve had the chance to reflect on my last 6 years as a performance tester, working in the mostly enterprise / commercial test space in and around Australia… Here’s my reflections on what has worked for me as a performance tester:
1. Document templates: Everything from a test strategy through to detailed plans, application simulation models, status updates or summary reports are all based on a template for me now. A template view of things is also a good way of culling back all the unnecessary guff that you often find in these documents. Unless you’re legally obliged to, cut back on the guff and tell them what you’re planning to test, how you’re going to measure it, and what you’re going to do if things fall outside your hypotheses…
2. Look beyond LoadRunner: I’ve gained much self satisfaction and saved employers large sums along the way by looking at tools other than LoadRunner. You may feel like you’re shafting yourself initially as you fumble about in strange GUIs or delve into the command line /code but in reality, you’re making yourself a better tester by understanding what it is you’re trying to test/simulate, and the multitude of ways you can accomplish it. Time to wean yourself off that expensive teat and start looking at the alternatives.
3. Business intelligence: You need to be able to understand machine events and transform them back into business speak. Most managers don’t give a hoot about number of IO’s per second or the vagaries of GC algorithms. You need to transform this data into a language that you can communicate up the chain. Avoid the FUD (Fear, Uncertainty & Doubt) and instead speak in terms of risk (probability, frequency, impact).
4. Monitor and measure: If you can’t monitor then you probably can’t measure. I’ve heard so many times that “you need this one tool because … ” it pulls everything together in one interface … Really, you need to know what you can measure and monitor, what tools support capture (most platforms have native commands to do so) and how you’re going to pull it all together. Timestamps are your friend (for correlating events) and so too are databases for storage. More often than not I’ve employed some form of RRD or RDMS such as MySql and more recently have been looking at the virtues of NoSQL with things like CouchDB or MongoDB.
5. The web server bone’s connected to the, database bone: You need to understand the stack you’re playing in. Most prod environments don’t give you the luxury of prodding around, but VMs are your friend and you’ll probably find most platforms you’re working on have free or developer versions of their products to play with. Chances are if you’ve never installed and configured it, you’re probably not going to have the fuzziest on where to start debugging or tuning it!
6. Adopt a scripting language: You’ll no doubt need to munge data, fake stubs, mock services, crawl for information and the like. I’ve seen some hilarious examples of this using heavyweight tools such as QTP/LoadRunner and some downright painful (although closer to zen) alternatives in shell script. But honestly, adopt a scripting language like Ruby, Python or even Perl, buy the corresponding cookbook, and you’ll have an answer to just about every housekeeping task you might find yourself faced with.
7. Learn from others: Go on, be humble, get yourself an RSS reader and learn from others. My personal favourites to name a few are:
Scott Barber http://www.perftestplus.com/scott_blog.php
Neil Gunther http://perfdynamics.blogspot.com/
Stuart Moncrieff http://www.myloadtest.com/
Corey Coldberg http://www.goldb.org/
Jared Quinert http://www.software-testing.com.au/blog/
Sam Abdelhamid http://mishmashmoo.com/blog/
1. Document templates: Everything from a test strategy through to detailed plans, application simulation models, status updates or summary reports are all based on a template for me now. A template view of things is also a good way of culling back all the unnecessary guff that you often find in these documents. Unless you’re legally obliged to, cut back on the guff and tell them what you’re planning to test, how you’re going to measure it, and what you’re going to do if things fall outside your hypotheses…
2. Look beyond LoadRunner: I’ve gained much self satisfaction and saved employers large sums along the way by looking at tools other than LoadRunner. You may feel like you’re shafting yourself initially as you fumble about in strange GUIs or delve into the command line /code but in reality, you’re making yourself a better tester by understanding what it is you’re trying to test/simulate, and the multitude of ways you can accomplish it. Time to wean yourself off that expensive teat and start looking at the alternatives.
3. Business intelligence: You need to be able to understand machine events and transform them back into business speak. Most managers don’t give a hoot about number of IO’s per second or the vagaries of GC algorithms. You need to transform this data into a language that you can communicate up the chain. Avoid the FUD (Fear, Uncertainty & Doubt) and instead speak in terms of risk (probability, frequency, impact).
4. Monitor and measure: If you can’t monitor then you probably can’t measure. I’ve heard so many times that “you need this one tool because … ” it pulls everything together in one interface … Really, you need to know what you can measure and monitor, what tools support capture (most platforms have native commands to do so) and how you’re going to pull it all together. Timestamps are your friend (for correlating events) and so too are databases for storage. More often than not I’ve employed some form of RRD or RDMS such as MySql and more recently have been looking at the virtues of NoSQL with things like CouchDB or MongoDB.
5. The web server bone’s connected to the, database bone: You need to understand the stack you’re playing in. Most prod environments don’t give you the luxury of prodding around, but VMs are your friend and you’ll probably find most platforms you’re working on have free or developer versions of their products to play with. Chances are if you’ve never installed and configured it, you’re probably not going to have the fuzziest on where to start debugging or tuning it!
6. Adopt a scripting language: You’ll no doubt need to munge data, fake stubs, mock services, crawl for information and the like. I’ve seen some hilarious examples of this using heavyweight tools such as QTP/LoadRunner and some downright painful (although closer to zen) alternatives in shell script. But honestly, adopt a scripting language like Ruby, Python or even Perl, buy the corresponding cookbook, and you’ll have an answer to just about every housekeeping task you might find yourself faced with.
7. Learn from others: Go on, be humble, get yourself an RSS reader and learn from others. My personal favourites to name a few are:
Scott Barber http://www.perftestplus.com/scott_blog.php
Neil Gunther http://perfdynamics.blogspot.com/
Stuart Moncrieff http://www.myloadtest.com/
Corey Coldberg http://www.goldb.org/
Jared Quinert http://www.software-testing.com.au/blog/
Sam Abdelhamid http://mishmashmoo.com/blog/
tim koopmans http://90kts.com/page/2/
8. Write your own blog: Got something to share? Want to improve your writing skills? Go on, write your own blog. I often find that in showing someone else either in person or the written form, I get to understand the subject matter better. So what are you waiting for?
8. Write your own blog: Got something to share? Want to improve your writing skills? Go on, write your own blog. I often find that in showing someone else either in person or the written form, I get to understand the subject matter better. So what are you waiting for?
