Bug Hunting is becoming a common practice form many testing
organizations world wide; yet some test managers wrongly feel they go
Hunting when their testers informally play with the application in order
to find “border-case defects”.
Bug Hunts are Informal Testing exercises; this should not be mistaken with playing with the system without a purpose or objective. In order to achieve something (and not waste your time!!) during these activities you need to follow a specific methodology, perform planning and preparation actions, and monitor and control the process throughout its execution.
Even if there is no written book (at least that I am aware) on Bug Hunts, I follow a process that I caught some years ago during one of the STAR conferences.
I plan Hunts based on Pair-Testing and Soap Opera Scenarios, combined as a tournament between teams.
Pair Testing is when you take 2 engineers and they work as a team on 1 computer (or environment). The idea comes from pair programming with all its known advantages such as knowledge sharing, brain storming, and positive pressure from working with a partner. I’ve had very good results pairing engineers from different teams; especially when taking development engineers and paring them with test engineers, leveraging the advantages and points of view from both sides of the process.
Soap Opera Scenarios are relatively short end-to-end scenarios that take the system from beginning to end of a complex operation on a fast (and sometimes exaggerated) chain of events. Working under extreme conditions we are able to find issues that we missed during our controlled scripts and scenarios.
The Tournament activity is where the human factor comes into play. You want people to be motivated, and what better motivation for an engineer than a price… I keep track of many statistics like the number of original bugs detected per day, the most serious bug detected, the worst system crash, etc; and give a price to the pair who exceed on each category (we usually give away t-shirts and on one extreme case we even gave an iPod!), in any case it is not the price but the spirit that makes the difference.
There are some rules of thumb I give QA organizations before they start planning their Bug Hunts:
1. Make sure you’ve reached a minimal application maturity level; if it is still too easy to detect critical bugs or the system keeps crashing all the time you may be ahead of your time.
2. Work on testing environments with real life data. Your tests need to be far from sterile in order to simulate a real working environment.
3. Create a balanced activity schedule. I work based on 2-day bug hunts, where each day starts with 4 hours of Exploratory Testing around a specific application area (each team or two working on a different part of the application); and during the second half of the day we run our Soap Opera scenarios, where each team receives a user profile and a list of high level tasks.
4. Manage the activity using a tracking system. Any bug tracking system will help you keep track of the issues detected by the teams; if possible use a dashboard to make sure all team members are updated on the position of the rest of their peers.
5. Have a bug referee, she will decide whether the findings are real bugs and also stop duplications from been reported and counted.
6. Create anticipation by having internal teasers and “publicity campaigns” ahead of the activity.
7. During the hunt generate an informal atmosphere by playing music, giving each team bells to signal whenever they find and report a new bug, bringing pizzas and sodas, and using other out-of-the-ordinary things that will create the feeling of a special occasion.
Bug Hunts are as much about the right atmosphere and state-of-mind as they are about the methodology and scenarios you run. Make sure you and your team have fun and it will surely generate the outputs your expect it to produce or more.
http://qablog.practitest.com/2008/05/what-do-you-pack-when-you-go-for-a-bug-hunts/
Bug Hunts are Informal Testing exercises; this should not be mistaken with playing with the system without a purpose or objective. In order to achieve something (and not waste your time!!) during these activities you need to follow a specific methodology, perform planning and preparation actions, and monitor and control the process throughout its execution.
Even if there is no written book (at least that I am aware) on Bug Hunts, I follow a process that I caught some years ago during one of the STAR conferences.
I plan Hunts based on Pair-Testing and Soap Opera Scenarios, combined as a tournament between teams.
Pair Testing is when you take 2 engineers and they work as a team on 1 computer (or environment). The idea comes from pair programming with all its known advantages such as knowledge sharing, brain storming, and positive pressure from working with a partner. I’ve had very good results pairing engineers from different teams; especially when taking development engineers and paring them with test engineers, leveraging the advantages and points of view from both sides of the process.
Soap Opera Scenarios are relatively short end-to-end scenarios that take the system from beginning to end of a complex operation on a fast (and sometimes exaggerated) chain of events. Working under extreme conditions we are able to find issues that we missed during our controlled scripts and scenarios.
The Tournament activity is where the human factor comes into play. You want people to be motivated, and what better motivation for an engineer than a price… I keep track of many statistics like the number of original bugs detected per day, the most serious bug detected, the worst system crash, etc; and give a price to the pair who exceed on each category (we usually give away t-shirts and on one extreme case we even gave an iPod!), in any case it is not the price but the spirit that makes the difference.
There are some rules of thumb I give QA organizations before they start planning their Bug Hunts:
1. Make sure you’ve reached a minimal application maturity level; if it is still too easy to detect critical bugs or the system keeps crashing all the time you may be ahead of your time.
2. Work on testing environments with real life data. Your tests need to be far from sterile in order to simulate a real working environment.
3. Create a balanced activity schedule. I work based on 2-day bug hunts, where each day starts with 4 hours of Exploratory Testing around a specific application area (each team or two working on a different part of the application); and during the second half of the day we run our Soap Opera scenarios, where each team receives a user profile and a list of high level tasks.
4. Manage the activity using a tracking system. Any bug tracking system will help you keep track of the issues detected by the teams; if possible use a dashboard to make sure all team members are updated on the position of the rest of their peers.
5. Have a bug referee, she will decide whether the findings are real bugs and also stop duplications from been reported and counted.
6. Create anticipation by having internal teasers and “publicity campaigns” ahead of the activity.
7. During the hunt generate an informal atmosphere by playing music, giving each team bells to signal whenever they find and report a new bug, bringing pizzas and sodas, and using other out-of-the-ordinary things that will create the feeling of a special occasion.
Bug Hunts are as much about the right atmosphere and state-of-mind as they are about the methodology and scenarios you run. Make sure you and your team have fun and it will surely generate the outputs your expect it to produce or more.
http://qablog.practitest.com/2008/05/what-do-you-pack-when-you-go-for-a-bug-hunts/
No hay comentarios.:
Publicar un comentario