I was visiting friends and family last week in Costa Rica. Even
though there is a growing software industry in there, my non-technical
childhood friends only know that I work in “something related to
computers” and for them that means I do everything from installing
printers to writing the software for the space shuttle program. One of
my biggest entertainment activities was to explain that there is a
discipline called Quality Assurance, and that we work side by side with
the development teams making sure the released products meet the desired
levels of internal and external quality.
During a dinner conversation the father of a close friend asked me a simple question: How come there are bugs in software? From his point of view, if there are teams in charge of testing then there is no reason for the products to have defects. (Was he insinuating something about our job performance…?)
The answer to his question is economic in nature, and one of the biggest complaints of QA teams worldwide: “there are never enough time and/or resources to test the entire product”. Simply put, Organizations know that it is more cost-effective to release “some defects” than to test and fix all of them; especially if the company has “some control” on the number of bugs that will be discovered and the areas where they may appear.
This practice is not some sort of QA-Religious Sacrilege. It is the conscious practice of controlling and managing the number of Escaping Defects of the released application and using this measurement to grade and manage the quality of the product and of the development and testing processes.
Escaping Defects (ED) are all the bugs detected (and reported) by customers, regardless if they were also found as part of the testing process. The Escaping Defects Rate (EDR) is calculated by dividing them by the total number of bugs found in the product (both detected and escaping)
In the same way as different products require different levels of quality to be cost-effective each one should target its own acceptable EDR range. For example an Avionics System used on commercial airliners will aim to have no less than 0.001% and no more than 0.005% EDR after 36 months of been released, while an IM application aimed at high school kids will target between 1.5% and 3.0% EDR after 12 months in the market.
Some general rules of thumb regarding Escaping Defects:
- Enterprise Software will usually require lower EDR levels than Consumer Products
- New products and companies will be expected and allowed to have higher EDR levels than mature and established products and companies
- EDR’s behave like S-Curves over the course of time
Last but not least, since not all bugs are created equal EDR or any other defect statistic will not be able to tell the complete story about the quality of the released product and process; it is also necessary to monitor the nature of the defects been detected in the field. A defect on the main path of the product that renders an important part of the application inoperable will always have a bigger weight than a bug on a side feature. Just keep in mind that not everything resides in the numbers, justice is not always blind…
This means the job of the QA manager does not stop once the application is released. We must monitor the released product in order to evaluate and grade the quality of our product and process, providing feedback for a culture of on-going improvement.
http://qablog.practitest.com/2008/01/process-quality-feedback-and-escaping-defects/
During a dinner conversation the father of a close friend asked me a simple question: How come there are bugs in software? From his point of view, if there are teams in charge of testing then there is no reason for the products to have defects. (Was he insinuating something about our job performance…?)
The answer to his question is economic in nature, and one of the biggest complaints of QA teams worldwide: “there are never enough time and/or resources to test the entire product”. Simply put, Organizations know that it is more cost-effective to release “some defects” than to test and fix all of them; especially if the company has “some control” on the number of bugs that will be discovered and the areas where they may appear.
This practice is not some sort of QA-Religious Sacrilege. It is the conscious practice of controlling and managing the number of Escaping Defects of the released application and using this measurement to grade and manage the quality of the product and of the development and testing processes.
Escaping Defects (ED) are all the bugs detected (and reported) by customers, regardless if they were also found as part of the testing process. The Escaping Defects Rate (EDR) is calculated by dividing them by the total number of bugs found in the product (both detected and escaping)
In the same way as different products require different levels of quality to be cost-effective each one should target its own acceptable EDR range. For example an Avionics System used on commercial airliners will aim to have no less than 0.001% and no more than 0.005% EDR after 36 months of been released, while an IM application aimed at high school kids will target between 1.5% and 3.0% EDR after 12 months in the market.
Some general rules of thumb regarding Escaping Defects:
- Enterprise Software will usually require lower EDR levels than Consumer Products
- New products and companies will be expected and allowed to have higher EDR levels than mature and established products and companies
- EDR’s behave like S-Curves over the course of time
Last but not least, since not all bugs are created equal EDR or any other defect statistic will not be able to tell the complete story about the quality of the released product and process; it is also necessary to monitor the nature of the defects been detected in the field. A defect on the main path of the product that renders an important part of the application inoperable will always have a bigger weight than a bug on a side feature. Just keep in mind that not everything resides in the numbers, justice is not always blind…
This means the job of the QA manager does not stop once the application is released. We must monitor the released product in order to evaluate and grade the quality of our product and process, providing feedback for a culture of on-going improvement.
http://qablog.practitest.com/2008/01/process-quality-feedback-and-escaping-defects/
No hay comentarios.:
Publicar un comentario