Is friendship ruining your testing career???

’m sorry to say that I have some bad news for you today…
Humans are social beings, and as such we have the instinctive and almost unavoidable need to be liked by our peers.
I’m not making this up!  Maslow went as far as listing belonging as one of the primary psychological needs in his famous pyramid, on top of Safety but less important than Esteem.
This means that for you it is (obviously) extremely important to make sure no one is running behind you with a loaded gun; but right after that it is more important for you to feel that your group likes you than it is to know they appreciate you as a professional tester.
If you haven’t understood how this is making you a bad tester, then let me explain a little further…

We are paid to criticize our peers

Our main task as testers is to criticize the work of our peers and to provide visibility into the status of our projects.
Criticising in itself is not a bad thing!
The definition of the word is actually “to consider the merits and demerits of (something) and judge accordingly“.  For example think about the movie critic who writes a positive review of a movie, or the food critic who recommends a restaurant to his readers.
For us as testers this criticism is translated, in the simplest ways, to:
- Test the product developed by our programmers and set our tests to pass or fail.
- Report bugs when something was not done in the way it should have been done.
- Raise flags when we understand the team is not working in a way that will allow the project to be released on time, on scope or with the proper level of quality.
& finally
- Write reports that give an overview of the status of our project:  Good or Bad, and why?
In short we are charged with the job of investigating the work being done by our peers, and reporting to management whether this work is good or bad for the project.
But I don’t want to get my friends into trouble…
When the work is good then we obviously have no problem telling this to our bosses.
But when the work is bad: when tests keep failing all the time, when bugs are raised in large numbers, when the application is not on the state it should have been by this stage of the project…  Then we need to come to our management with bad news that will most certainly get some of our “friends” into trouble.
It is normal to feel remorse for escalating bad news to management, specially when we know that our friends are not completely to blame for all the delays and issues happening in the development process.  So what can you do?

Maybe we need to be “team-players” and help our friends?

How about we stop reporting all the bugs in the Bug Tracking System?  Instead we can send “off-the-record” emails with issues found to our developers, and this way the statistics don’t look so bad for them.
But then what will happen when one of these bugs escapes from the system and is found by a customer?  At that time we will be asked how come we didn’t find this critical bug during the testing process???
Or maybe we can delay some of our tests because we know that the system is not ready and most of them will fail.
Then what will happen when the project manager comes and says that we are the one to blame for the delay of the whole release because we were not able to run our tests on time???
All of a sudden, because we became “team players” and didn’t do our jobs correctly, we became the focus for all the issues in the project!

The best path is to be transparent and professional in our work

It is understandable that you don’t want to get any of your “friends” into trouble, these are the same guys you are having lunch with every day and the same guys with whom you play poker or basketball one night a week.
The best way to help your team mates is by using the same tools that may harm them in order to improve the way they do their work.
For example:
Give them constant visibility into the statistics reflected by your bug findings.  Have a process and a mechanism in place that helps them to see the “story” told by the bugs being found and their trends.  Help them to use this information for their own benefit and to have the ways of proactively informing management of any issues before you get to them with the “bad news”.
Make sure they are constantly aware of your testing approach and schedule.  If you are going to be doing extensive tests on the new functionality give your team the time to have the feature ready on time, share with them the tests you will run so that they are not “surprised” by the bugs that you find.  Remember that you work together so you don’t need to “hide” your scenarios from them in order to find more bugs.
Share your results with the development before you send them management, this will give them time to add their own comments and so help you provide the best possible picture of the situation.

In two words: Working Together

It took me some years to realize this, but as testers we don’t work against our developers, we work for them!
Part of our job (among other things) is to help them write beter software, and to do this as quickly and effectively as possible.
We need to make sure they understand that we want to help, and at the same time to educate them in the ways in which they can help us to do this job better (but that I will leave how to educate them to a different post).
So, the next time you feel that your friendship is getting in the way of your work, remind yourself that you are there to help your friends in their work and not to cause them any harm.
Think about the PROFESSIONAL things you can do in order to assist them in their work.

http://qablog.practitest.com/2013/06/is-friendship-ruining-your-testing-career/

Diferencias entre QA y QC – Parte II

Puntos para seguir discutiendo:

• Todo depende del enfoque que se defina
• Hay que definir un procedimiento de trabajo, ajustarlo y conducir su cumplimiento
• Evitar actitudes defensivas que usan al proceso como escudo en el caso de que los proyectos no salgan bien.
• Hay que luchar para que el procedimiento, proceso o metodología sean herramientas
• Hay que luchar para que el modelo impuesto sea proactivo y nó, reactivo
• Tener presente en todo momento que así como “No Hay Cero Bugs”, “El Proceso No Asegura Nada”

En síntesis, la forma más común en la que nos vamos a encontrar el QA será orientado al procedimiento y a las metodologías, considerando a los proyectos como algo genérico o en el mejor de los casos, definiendo distintos itinerarios en función de una categorización que se haga de los mismos.

No obstante muchos se niegan a pensar en que ésto es el alcance de QA (Aseguramiento de la Calidad)

Fuente de Inspiración: jummp

Lo realmente importante es que exista una comunicación fluida entre todos los implicados con el objeto de buscar soluciones a los problemas y de esta forma conseguir de manera efectiva una mejora continua.

No se trata, por tanto, de eliminar los procesos, sino de que sean racionales y ajustados a lo que la organización necesita y puede soportar y nunca de espalda a la realidad de los proyectos de desarrollo de software que se realizan y el presupuesto con el que se puede contar en los mismos.

Fuente: http://wp.me/ppBOQ-3Lw

El control de la calidad (QC) es reactivo. Se ha llegado hasta el final y se realiza una entrega del producto y otros artefactos documentales asociados. En este punto, a lo más que se puede llegar es a la detección de defectos antes de su puesta en producción.

Estos defectos pueden ser de mayor o menor gravedad, bloqueantes, mayores o menores.

No es malo disponer de esta última línea de defensa, antes al contrario, puede ser muy interesante, pero su efectividad está muy condicionada por el trabajo del equipo de desarrollo, es decir, si se considera que una aplicación es válida porque cumple el plan de pruebas entregado por el desarrollador, estamos dejando toda la responsabilidad en ese plan de pruebas y todos sabemos que, salvo honrosas excepciones, los desarrolladores no somos expertos en eso.

Este último párrafo tiene que ver con que si no hay una línea intermedia representada por el Analista Funcional, puede caber la duda de que el desarrollador haya podido plasmar en un documento, lo que tiene que hacer verdaderamente el sistema.

Indudablemente, las buenas prácticas hacen que al combinar QA y QC tenga incidencia en la calidad y valor del producto final.

Por supuesto y como explica el autor: “Esto es así, siempre y cuando el QA, como he ido insistiendo en artículos anteriores, no se convierta en un simple verificador del cumplimiento de determinados procesos de desarrollo.”

Fuente: Jummp

¿Por qué es más frecuente encontrarnos con QC que con QA (pese a que a ese QC se le llame de manera equivocada QA)? Por motivos económicos, ya que el QA requiere una mayor implicación en el proyecto por parte de los técnicos que realizan este tipo de tareas (salvo que se haya elegido el camino de validar procesos en cuyo caso ese trabajo se simplifica), por lo que la definición de un equipo QA+QC requiere una mayor inversión.

Sin embargo, lo que nos vamos a encontrar con más frecuencia es la combinación de QA y QC pero con una orientación reactiva: entrega y validación.

Se llega a esa circunstancia principalmente por dos motivos: es lo más cómodo, ya que permite definir un modelo con un interfaz definido, desapegado de la realidad de los proyectos que se están ejecutando (al no formar parte de ellos) y por otro lado, por la errónea convicción de que son los procesos, procedimientos y metodologías los que mandan y los que hacen buenos o no el resultado final del proyecto.

¿Pero no es lo ideal que los equipos de calidad y desarrollo sean distintos? Mi propuesta no es incompatible con eso, ya que lo que estoy indicando es que un grupo de personas dentro o no del equipo de calidad colaboren con el equipo de desarrollo (incluso siendo parte de él si las circunstancias lo permiten) para que tanto los trabajos que se realicen en el mismo como los propios artefactos que se vayan a entregar al QC sean lo más efectivos posibles.

URL: jummp

En el caso de un equipo QC, su funcionamiento se puede orientar perfectamente a un procedimiento: recibo una entrega en un formato determinado, hago las tareas de verificación y devuelvo unos resultados. Se puede ser ajeno al conocimiento del producto y del proyecto (se trabaja sobre la documentación escrita por el equipo de desarrollo). Por eso muchas organizaciones que se dedican a prestar este servicios, tienden a crear factorías de testing, ya que el modelo encaja perfectamente, y de esta forma pueden ofrecer precios todavía más competitivos porque “da igual” que la persona que vaya a trabajar sobre la entrega conozca el producto.

Después la pericia de los técnicos QC les puede hacer llegar más lejos, ya que pueden hacer testing exploratorio y aplicar otro tipo de técnicas que les permitan que la detección de defectos sea más efectiva.

Para valorar la efectividad de la implantación de un QC es necesario evaluar sus resultados frente a los costes directos (los del propio equipo del QC) como los indirectos (los que ocasiona en los proyectos).

URL: http://wp.me/ppBOQ-3LA

Hay algo que el autor expresa y que estoy totalmente de acuerdo con él:

“…por lo que no puedo entender el aseguramiento de la calidad si no se colabora en la selección y aplicación de los casos de prueba que permitan verificar el cumplimiento de las expectativas, si no se tratan de establecer medidas que permitan mantener la deuda técnica bajo control y no se mejoran las técnicas de testing que se están utilizando con el objeto de automatizarlas en lo posible y detectar defectos cuanto antes.”

En otra parte, lanza el siguiente comentario:

“…Si el QA solo mira el proceso, se está perdiendo lo más importante.”

Y al final de su artículo, cierra con el siguiente:

“…Tanto el respeto al proceso como la aplicación de determinadas prácticas son elementos instrumentales o de apoyo, no lo olvidemos. El desarrollo de software es mucho más complicado que todo eso porque no debemos olvidar el propósito del proyecto que no es otro que satisfacer un conjunto de expectativas y necesidades y si eso no se consigue, habremos fracasado, en mayor o menor medida, pero fracasado.”

Me pongo a pensar que no todas las empresas tienen dos áreas como las que se han discutido aquí, es decir, QA y QC; y sí me cabe pensar que hay sólo una enmarcada dentro del término “Testing” y que refiere a las anteriores, y que hasta incluso para el área comercial (porque sabemos que es más ‘vendible’) la llaman ‘QA’.

De ahí a reflexionar acerca de la competencia de sus miembros y de las políticas que se adoptan, ya que muy probablemente no tienen los objetivos que el autor de estos artículos ha enunciado.

Fuente: Jummp

http://testingbaires.com/diferencias-entre-qa-y-qc-parte-ii 
http://testingbaires.com/diferencias-entre-qa-y-qc-parte-iii
http://testingbaires.com/diferencias-entre-qa-y-qc-parte-iv
http://testingbaires.com/diferencias-entre-qa-y-qc-parte-v
http://testingbaires.com/diferencias-entre-qa-y-qc-parte-vi
http://testingbaires.com/diferencias-entre-qa-y-qc-parte-vii

Please, stop calling us QA ninjas!

A friend of mine asked me to check the site of a start-up company he is interviewing for as a tester.  The site of the company looks nice, they are in the social media arena and working on yet another way to link between companies and what’s been said about them by bloggers, tweeters, etc.
As I told him, these are the start-ups that will either make it big and noisily, or sink silently into the horizon…  But this is not what made me want to write about them, after all these companies are “a dime a dozen” today.
What caught my eye was that in their “Open positions page” (I was checking the job my friend was applying for) they were looking for a “QA ninja“.

Can someone please ask these guys to stop calling us names?

My first reaction was to ask myself “what where they thinking?!”
I guess the person publishing this position thought:
“We are a special, young, and very cool company; and we are looking for a special kind of tester!  If we call the position a QA Ninja, the candidate will understand his job is “a very cool job” and not the job of a regular tester!”
Even if this is not exactly what he thought, I am sure it was very close.  I mean these guys are not bad or anything, but they surely don’t understand how we testers see our jobs, or what are the things that will make us want to work somewhere or not.

About managers who think testers are anti-bug miracles.

My second thought about choosing to name the job a QA Ninja was that the hiring manager may not really understand what he is looking for in his testers.
To understand what I mean, close your eyes and think about a ninja (and I mean the idea of a ninja based on generation-X movies and TV shows).  You will come up with either movies like American Ninja or shows like the Teenage Mutant Ninja Turtles, or if you are a current parent you may thing of the Ninjago characters by LEGO.

Ninjago by LEGO

In all these images a Ninja is a commando fighter that will enter a dark and dangerous situation and complete his noble objective silently, cautiously, and doing lots and lots of jumps and climbing along the way.
BTW, if you look at the meaning of the real Ninjas (at least based on wikipedia) you will see that they are defined as covert agents or mercenaries who specialized in unorthodox warfare, such as espionage, sabotage, infiltration and assassination…
What this means is that many managers simply fail to understand the concept of Quality Assurance and what they should be looking for in their testers!  We are not anti-bug miracle makers, magicians or ninjas for that matter.
A tester will not get rid of all the bugs in the system, and in the vast majority of projects they will not even find all the bugs in the system by the time it is released.
A good tester, will provide visibility into the project and product been developed.  A great tester will take this visibility and transform it into Testing Intelligence, or the information that will help the management team make educated decisions regarding their products and projects!
But in no case will we take a sword and kill all the bugs (although sometimes we will want to grab a sword and stab a couple of developers…)

If you are looking to attract top quality testers try this…

A good tester likes a challenge.
He will also like to influence not only how he tests the product but also how the system is designed and developed in order to make the testing more effective.  But above all he will like to know he will be heard when he has something to say.
You can advertise that you are looking for a tester that will improve the quality of the product and process, to introduce methodology and technology that will make these efforts more effective and scalable, and that will help the whole team take the the company to the next level regarding the stability, usability and overall performance of the product or system been developed.
Understand that we don’t see ourselves as ninja climbing walls in the night, but as professionals who have a lot to contribute to our projects.

Ps. About my friend and his job…

In the end, I recommended my friend to take the job, after all the company looks pretty exciting and he describes the people there as very open and nice to work with.
But I also told him he needs to have a good expectations-settings talk with his managers before accepting the job, to make sure they understand what he can provide as a tester.

http://qablog.practitest.com/2012/05/please-stop-calling-us-qa-ninjas/

What do you pack when you go for a Bug Hunts?

Bug Hunting is becoming a common practice form many testing organizations world wide; yet some test managers wrongly feel they go Hunting when their testers informally play with the application in order to find “border-case defects”.
Bug Hunts are Informal Testing exercises; this should not be mistaken with playing with the system without a purpose or objective. In order to achieve something (and not waste your time!!) during these activities you need to follow a specific methodology, perform planning and preparation actions, and monitor and control the process throughout its execution.
Even if there is no written book (at least that I am aware) on Bug Hunts, I follow a process that I caught some years ago during one of the STAR conferences.
I plan Hunts based on Pair-Testing and Soap Opera Scenarios, combined as a tournament between teams.
Pair Testing is when you take 2 engineers and they work as a team on 1 computer (or environment). The idea comes from pair programming with all its known advantages such as knowledge sharing, brain storming, and positive pressure from working with a partner. I’ve had very good results pairing engineers from different teams; especially when taking development engineers and paring them with test engineers, leveraging the advantages and points of view from both sides of the process.
Soap Opera Scenarios are relatively short end-to-end scenarios that take the system from beginning to end of a complex operation on a fast (and sometimes exaggerated) chain of events. Working under extreme conditions we are able to find issues that we missed during our controlled scripts and scenarios.
The Tournament activity is where the human factor comes into play. You want people to be motivated, and what better motivation for an engineer than a price… I keep track of many statistics like the number of original bugs detected per day, the most serious bug detected, the worst system crash, etc; and give a price to the pair who exceed on each category (we usually give away t-shirts and on one extreme case we even gave an iPod!), in any case it is not the price but the spirit that makes the difference.
There are some rules of thumb I give QA organizations before they start planning their Bug Hunts:
1. Make sure you’ve reached a minimal application maturity level; if it is still too easy to detect critical bugs or the system keeps crashing all the time you may be ahead of your time.
2. Work on testing environments with real life data. Your tests need to be far from sterile in order to simulate a real working environment.
3. Create a balanced activity schedule. I work based on 2-day bug hunts, where each day starts with 4 hours of Exploratory Testing around a specific application area (each team or two working on a different part of the application); and during the second half of the day we run our Soap Opera scenarios, where each team receives a user profile and a list of high level tasks.
4. Manage the activity using a tracking system. Any bug tracking system will help you keep track of the issues detected by the teams; if possible use a dashboard to make sure all team members are updated on the position of the rest of their peers.
5. Have a bug referee, she will decide whether the findings are real bugs and also stop duplications from been reported and counted.
6. Create anticipation by having internal teasers and “publicity campaigns” ahead of the activity.
7. During the hunt generate an informal atmosphere by playing music, giving each team bells to signal whenever they find and report a new bug, bringing pizzas and sodas, and using other out-of-the-ordinary things that will create the feeling of a special occasion.
Bug Hunts are as much about the right atmosphere and state-of-mind as they are about the methodology and scenarios you run. Make sure you and your team have fun and it will surely generate the outputs your expect it to produce or more.

http://qablog.practitest.com/2008/05/what-do-you-pack-when-you-go-for-a-bug-hunts/

Diferencias entre QA y QC

Este artículo bajo el título “Diferencias entre aseguramiento de la calidad y control de la calidad I”, lo recibí a través del newsletter de jummp (Recomiendo que se suscriban).

El término aseguramiento de la calidad se ha extendido a tareas de prevención en el momento en que se está desarrollando (detección de defectos, adecuado seguimiento de un proceso y/o una metodología) y a las tareas que se realizan una vez que se entrega una determinada versión de un producto o de un artefacto (que puede ser un determinado documento exigido por el proceso de desarrollo).

Y es que aseguramiento de la calidad (QA) vende más que control de la calidad (QC), porque asegurar resulta más rotundo que controlar.

El aseguramiento de la calidad está orientado a la prevención, pretende ser, por tanto, proactivo, lo que requiere que sea una parte más dentro del proyecto y suele centrarse en aspectos relacionados con el proceso de desarrollo y testing.

El hecho de que se considere que el QA está orientado al proceso, en mi opinión, ha restado efectividad a su aplicación porque el producto nunca debe quedar en segundo plano y en el momento en que se transmite el mensaje (aún con la mejor de las intenciones) de que la calidad está en el proceso empieza a interpretarse por los responsables del QA de que lo valioso es el procedimiento y que si se sigue probablemente el proyecto tenga éxito.

Curiosamente si se falla, será culpa de las personas por no haber seguido adecuadamente el proceso. Este doble rasero: las personas son secundarias porque quien asegura la calidad es el proceso, pero si todo falla los culpables son las personas, no lleva a nada bueno porque crea una espiral que irá separando a los equipos de desarrollos de los encargados del QA y que por regla general, si no se corta, agudizará el problema porque la respuesta del QA será hacer todavía más rígidos los procesos, hacer más severos los controles y si se siguen sin conseguir objetivos seguirá siendo culpa de las personas y así hasta el infinito.

Cierto es que hoy en día todavía hay empresas y/o personas que entienden/asocian al Software Testing con QA, ¿Cuánto tiempo llevará hacerles entender que no es así? ¿Cuánto tiempo deberemos anunciarnos a la puerta y decir que no somos QA? Aunque podemos serlo, por supuesto, dependiendo del skill y las ganas que tenga cada uno de nosotros, además de su experiencia.

Según definición: ISO 9000 Definitions
• Quality Control
– The operational techniques and activities that are used to fulfill requirements for quality.
• Quality Assurance
– All those planned and systematic activities implemented to provide adequate confidence that an entity will fulfill requirements for quality.

QC vs QA
Producto -> Proceso
Reactivo -> Proactivo
Encontrar Defectos -> Prevenir Defectos

QC vs QA – Ejemplos
Walkthrough -> Auditoría de Calidad
Testing -> Definición de Procesos
Inspección -> Selección de Herramientas
Revisión de Puntos de Control -> Entrenamiento

Fuente de inspiración: JUMMP

http://testingbaires.com/diferencias-entre-qa-y-qc

Improve your Testing Skills, take a look at your Kids!

I will need to give Rob Lambert credit for planting the seed for this post with a comment he left on a totally unrelated post about Company Politics and QA, when he mentioned learning from his kid how to pick his fights.  This got me thinking that we have many more things to learn from our kids, specially the way to improve how we approach our tests.
I got 2 kids at home; a boy who’s almost 2 years old, and a girl who’s almost 2 months old.  From playing, interacting, and simply from looking at them I’ve been able to grasp some practices and approachesto improve my testing tasks and results.
Curiosity – even if he has seen the same closed box hundreds of times, my son needs to open it in order to know what’s in it.
Skepticism - he doesn’t believe me when I tell him there is nothing in the box, he opens it just in case I might have forgotten something cool inside.
Emotional motivation – regardless if he wants to climb to the sofa, to climb up the stairs, or hold the dog’s leash when we go out for a walk, you can always perceive the motivation in my son’s eyes that make him give everything he has in order perform the task.
Get on your feet fast and keep on going -  Kids are not ashamed to fall down while learning to walk, they lick their bruises (sometimes literally) and get up to continue trying.
Not getting stuck too long in one thing – Children are excellent smoke-testers, they know how to play exactly 15 seconds with each of their toys until they reach the one that is interesting enough to spend more time with it.  And even with the interesting toy, after 5 minutes they know to move along to the next toy in line until they’ve gone over all their repertoire (some times twice!).
Analyze and take stuff apart -  I’ve seen Ariel (my old boy) take apart stuff that I didn’t know could be separated, just give him time and he will understand what pieces can fall apart. Then he will examine each one of them individually to try and find fun things to do with it too.
Watch and learn – They are constantly looking at what’s happening around them and learning, they sometimes are even able to learn from the experience of others (something I wish I was able to do myself!)
Not afraid or ashamed to ask for help – When my kid wants to play with something and he’s not able to get it functioning, once he is done throwing it and expecting it to work he brings it to me so that I can help him.  Just imagine how great it would be having an employee, a boss or even a peer smart enough to try to work a problem by himself, and then if he gets stuck or fails to ask for help…
Use all your senses – Give something new to a kid and he will start by looking at it from all sides, then he will touch it and try to find if there are buttons or stuff to take apart, then he will simultaneously smell and lick it; and only then he will throw it to see if something cool happens when it falls.  Kids use all their senses when trying to understand something.
Personally I think the most important thing I can learn from my kids is to look at all other kids without prejudices, and to give each new person an equal chance the first (and sometimes also the second and third) time they meet them.

http://qablog.practitest.com/2009/04/improve-your-testing-skills-take-a-look-at-your-kids/

Release Criteria – defining up-front when the product will be ready to be released

I make a point of not making important decisions in the heat of the moment. For example, when I disagree with someone on a professional level and tones start getting hot during a meeting, I ask for a timeout or a change of subject and to re-take the issue after a 24 hours period of perspective. In 90% of the cases this time will help me and my colleague to better understand the other’s point of view and reach an understanding that allows both of us to walk out of our argument happy.
Still, there are times during a development project when you need to make a decision on-the-spot with no way to “snooze” your assessment. One of these situations happens around the end of the project, during what many of us call Release Decision Meetings; where representatives from all departments get together to provide their inputs and jointly decide whether the product is ready to be released to the public.
I’ve been part in many of these meetings as a QA Manager. These are the times when on the one hand you are pressured to release the product NOW (or as soon as possible), but on the other hand you carry a big part of the responsibility for the stability and the quality of the final product been released.
How do you make sure to be objective and not to make mistakes at these times of pressure? One ways is by having a Release Criteria list that was defined and agreed up front by all the project stakeholders.
What is a Release Criteria? Simply put, it is the list of Objective Prerequisites that should be met by the product and process in order for us to feel confident about releasing it to the public. The most important aspect of the Release Criteria is that it needs to be reviewed and agreed by all product stakeholders prior to the Release Meetings and preferably during the early stages of the Development Processes.
The release criteria will never replace the Release Decision Forum, as this group of people needs to take into account all objective and subjective aspects surrounding this decision; but it will help you (the QA Manager) to provide an Objective Evaluation based on the parameters that were considered important before we started being influenced by external and maybe inappropriate factors.
What comprises a good Release Criteria list will vary from product to product and from company to company, but some basic items that should be included in it are:
1. Percentage of the original required functionality that needs to be in final the product.
2. Testing coverage level – percentage of the tests that need to be run on the application from the original testing goal.
3. Pass rate for the testing coverage –percentage of the executed tests that need to be successfully run without falling into blocks or failures.
4. Number of critical defects still open on the product.
5. Maximum defect detection rate for the last month, week, or x amount of days– based on the assumption that enough testing is still being done on the product.
6. Minimal load tolerance (if applicable).
7. Minimal amount of time the application has been running continuously on the testing and/or staging environments (if applicable).
8. Minimal amount of time the application has been on a Beta Testing trail (if applicable).
It is always recommended to start updating and publishing the status of the project with regards to the Release Criteria as early as logically possible. If you have Kitchen Monitors in place you should consider adding a page with it the to your running template.
As with all information generated by the QA, using the Release Criteria wisely will increase the amount of value you provide into the process.

http://qablog.practitest.com/2008/04/release-criteria-defining-up-front-when-the-product-will-be-ready-to-be-released/

Code Review Rights and Responsibilities

Code reviews are an important part of any project's health.  They are able to find and correct code defects before making it into the product and thus spare everyone the pain of having to find them and take them out.  They are also relatively cheap.  Assuming your team wants to implement code reviews, it is important to lay out the expectations clearly.  Code reviews can be contentious without some guidelines.  What follows are the rights and responsibilities as I see them in the code review process.
Participants:
Reviewer - Person (or persons) reviewing the code.  It is their job to read the code in question and provide commentary on it.
Reviewee - Person whose code is being reviewed.  They are responsible for responding to the review and making necessary corrections.
Must Fix Issues:
A reviewer may comment on many aspects of the code under review.  Some of the comments will require the reviewee to change his code.  Others will be merely recommendations.  Must fix issues are:

• Bugs - The code doesn't do what it was intended to do.  It will crash, leak memory, act erroneously, etc.
• Potential maintenance issues - The code is not broken, but is written in such a way that it will be hard to maintain.  Examples might be magic numbers, poorly named variables, lack of indirection, lack of appropriate comments, etc.
• Coding standard violations - If the group has a coding standard, it must be followed.  Deviations from it must be fixed when pointed out.

Recommendations:
Other items are merely recommendations.  The reviewer can comment on them, but the comments are only advisory.  The reviewee is under no obligation to fix them.  These include:

• Architectural recommendations - The reviewer thinks there is a better way to accomplish the goal.  Seriously consider changing these, but the reviewee can say no if he disagrees.
• Style issues - The reviewer wouldn't have done it that way.  Fascinating.

Code Ownership:
In my teams there is no ownership of code.  Some people touch certain pieces of code most often and may even have written the code initially.  That doesn't give them special rights to the code.  The person changing the code now does not need to get the initial writer's permission to make a change.  He would be a fool not to consult the author or current maintainer because they will have insights that will help make the fix easier/better, but he is under no obligation to act upon their advice.

http://blogs.msdn.com/b/steverowe/archive/2008/08/13/code-review-rights-and-responsibilities.aspx

Learning from a 5 year old who’s Testing the World

This weekend I caught myself in the car with my oldest son, who’s almost 5 years old, in one of those incredible conversations one can only have with a kid his age who’s discovering the World and its workings.

The subject of this specific conversation is not really important, although in the last couple of months we’ve dissected (there’s no other word to describe it) topics like planes and how they fly, how come the Prime Minister can make decisions for all of us, War, the judiciary system (cops <-> judges <-> jail), Money, what does it mean when someone sings on the radio “you stole my heart” and how come the cops don’t come to take it back, etc.
What I hadn’t realized until this weekend is that my son is actually testing HIS world, and using some simple tools we can all re-use as testers in our daily work.

5 simple testing tools to “re-learn” from our kids

1. Ask all the questions

My kid is constantly asking questions, going on and on and on until he fully understands the point.  His questions are sometimes linked to each other, while other times they seem to be completely unrelated, but in the end you can always see how he uses all the answers to paint himself a complete picture.
The simplest thing to remember as a tester is to ask all the questions.  There really are no stupid questions, but there are plenty of stupid mistakes we can avoid if we are not afraid to “look bad” by asking.
This goes both for cases where we are afraid to ask because we think we will look inexperienced, as well as those cases where instead of asking questions we (wrongly!!) assume we already understand everything.

2. Keep your eyes and ears open

My son is always listening, sometimes I think he even listens when he sleeps, and he will then surprise me with comments or questions about stuff I was not aware he heard.  It is even more amazing when I realize that some of his comments came from understanding something new about topics we last talked about two or tree weeks ago.
What we should re-learn from this is that, as testers, we need to keep our eyes and ears open. We should be able to capture even unrelated information that within the right context may help us make better sense of what is really going on in our projects.
One thing I explain to all my new hires is that our job is not (only!) to find bugs, but to provide visibility to the whole team and specially to our management in order for them to make the best possible decisions.
Good visibility is the result from looking at the same picture from all possible angles and tying together all the strings of information that will help people assess the risks and make a good decision.

3. Don’t be afraid to try, or to ask for help if don’t succeed

As every parent knows, kids love trying things for themselves –  specially when you tell them NOT to do something.  My son is no exception.  But I also saw that he has a quality many of us grown-ups lack, he knows when to ask for help when he doesn’t succeed.
As a manager I love hiring testers who like to try new things and I also like testers who enjoy taking on technological challenges.  But on the other hand, a big problem with these guys is that when they get stuck they usually waste a lot of time trying to solve the issues by themselves instead of asking for help.
So here the lesson to re-learn is twofold.  You need to dare to do more things by yourself, but you also need to recognize on-time when you are stuck and ask for help in order to save valuable time and resources.

4. Work with energy from start to end

I love this in particular about kids, when they do something they put all their hearts and energies on it.  I see it all the time with mine, when they are coloring with crayons, playing “catch” in the park, or simply splashing water during bath time, they are 100% focused on their current task; you hardly ever see a kid who is only partially concentrated on what he is doing right now.
Grown-ups in general and testers in particular should re-learn to put all our efforts in the task currently at hand.  Too many times I see testers (sometimes I even catch myself!) working on  a task while thinking about one or a number of other things in parallel.
Regardless of your personal take on multitasking, there is no way you will achieve the same results on any task if you are at the same time thinking or working on another one.
So the lesson to re-learn here is to fully concentrate on the task you are doing right now.  So simple, yet so hard to achieve…

5. Looking at your work through the eyes of a kid

Kids see all that surrounds them with positive naivety.  For them all people are good unless proven bad, all tasks are achievable unless proven (multiple times) unreachable, and most importantly all situations are positive unless a grown-up comes to spoils their fun.
We testers need to re-learn to look at our tasks through the eyes of a kid.
Let’s start by imagining the product working correctly and only then to go and look for the bugs, let’s understand how the features should work correctly and them to find the flaws in the business flows, and most importantly, let’s provide positive and constructive feedback at the same time that we report about the things that are not working correctly.
Our work can be very negative if we focus only on the bad, and this is why we need to re-learn to inject positive naivety into it.

Dad and mom, help your kids to learn and discover.
Hold their hands and don’t push them back.

I started this blog by describing how my son is constantly asking questions.  Sometimes, specially when it is late in the day and I am tired from all the “challenges” I had to overcome at work, I start loosing my patience and my ability to answer his questions with real and building answers.  On these times I wish I could just ask him to hold his questions until tomorrow, or better yet to go and ask someone else.
But then I remind myself that he is asking me because he trusts me to provide him with the answers that will help him to keep learning and testing the world that surrounds him.
This is why I ask you, as a parent and less as a tester, not only to enjoy these moments but to flourish them, these are the times when you can really help your kids grow; maybe not in a physical way, but surely in an intellectual way, which is no less important in the long run.

Finally…

Some time ago I wrote a blog about learning to test from our kids.  I guess this blog can be taken as a continuation of that previous one, although my feeling is that it stands-up by itself.

http://qablog.practitest.com/2012/05/learning-from-5-year-old-whos-testing-the-world/

Good Project Retrospectives

Retrospectives can potentially be one of the most beneficial tools in software development; then again most retrospectives I’ve participated in where a complete waste of time…
The idea of a retrospective, or postmortem as we called them about 10 years ago, is simple. At the end of the project take the time to define 3 things:
1. What went right and should be kept?
2. What went wrong and should be changed?
3. How to make things better the next time around?
So, how come most projects either don’t perform retrospectives, or carry out postmortems but don’t learn from them and keep doing things in the same unproductive way?
The answer to the first part of the question comes from the statement behind the second part. If a Manager “tries” doing retrospectives on a couple of projects and doesn’t get any added value, why should she continue doing them?
I think the problem is not with the concept of the Retrospective, but with how it is carried out in many Organizations and what is done with the results that come out of it.
There are a number of recurring mistakes and wrong-doings that can be seen in most postmortem failures:
1. Lack of Manager Leadership – If Senior Managers don’t give retrospectives their full attention and don’t communicate (using personal example) their importance, then middle management and the rest of the team will not treat these activities seriously.
Postmortems need to be lead by someone with authority and charisma; this is the only way to successfully guide the team through the complex process of reviewing, analyzing and learning from past activities and results.
2. Not gathering issues and facts throughout the project – Bad retrospectives focus mostly on the issues that happened during the last legs of the process, since these are the things we remember better and the ones that still have an emotional effect in most of us.
It is very hard to keep track of issues if we don’t record them when they happen, and it is even harder to analyze them if we don’t have objective data. The Project or Team manager needs to find ways to record important issues when they happen, and store information that will help the team understand how and why they took place.
A good idea I got from a Group several years ago was to create an e-mail alias that people could send important issues as they happened. At the end of the project the team running the retrospective can use these e-mails as a starting point for their process.
3. Letting postmortems turn into Witch Hunts – In some Young Organizations or after a very stressful project a postmortem can easily turn into a personal Witch Hunt. The most effective way to stop this from happening is by not letting it start in the first place.
If during the analysis and discussion process the team starts taking the subject into personal tones and levels, it is the job of the Manager to stop this on-the-spot and to turn the subject to a solely constructive and impersonal discussion. If necessary she should make a time-out or call a recession until people have a chance to cool down.
4. Lack of clear Lessons Learned, Action Items and Follow-through Plan – The big mistake of creating a large list of things to improve and yet continue working in the same way…Improvement doesn’t happen by itself, it needs to be pushed by people who keep track of the unwanted behavior and constantly make an effort to correct it.
A good practice is to focus on only 3 things to improve from each retrospective, and then during the following project to implement a way to measure these 3 things and constantly track their improvement.
I personally like to take my targeted improvements and start reviewing them on the weekly (or periodical) management and team meetings, 10 minutes before each meeting and 2 minutes during the meeting itself is usually enough to ensure these action items stay in-mind and on-track.
Last but not least, as I was looking more material for this article I found something really amazing from a group of people I was not taking seriously enough. The Game Development Industry, or at least the New Engineers specializing in this area have been using and publicly posting their retrospectives more than any other groups that I know of. I suggest you take the time to review this site, and learn both from the way the can seriously analyze their wins and mistakes; you might even read something that can be leveraged to your team or project.

http://qablog.practitest.com/2008/07/good-project-retrospectives/

Interviewing the Experienced

This week there was an interesting conversation over on Slashdot.  The subject of the post is an age discrimination suit against Google.  However, the discussion has gone to other interesting places.  The question is being asked if there is a difference in the way you should interview experienced people vs. those just out of school.  It reflects something I've come to understand after years of interviewing.  The software/IT industry is relatively young.  This is true both in terms of the workforce and the maturity of the work ecosystem.  The result of this is that we don't tend to have very sophisticated interview processes.  To be sure, they are good, but they are not flexible enough.  They are aimed at hiring the young hotshot recent graduate.  They are not usually designed to find someone with experience.  That will have to change as the median experience in the industry raises.
What is it that biases interviews toward those recently in school?  The questions we ask favor those who have graduated in the near past over those who have been in the industry for a decade or two.  It is pointless to ask a newly minted graduate about his/her experience.  They will likely have a few group projects and perhaps a summer internship to talk about, but the projects are all limited in scope.  Even someone who has been in the workforce for a few years will likely have been implementing someone else's design.  Assessing non-trivial design skills is hard and still harder in an hour-long interview.  So often we don't evaluate this.  Instead, we turn to the easily measurable.
We, as an industry, tend to ask a lot of questions that are more about particular knowledge than about problem solving skills.  Sure, we couch them in terms of problem solving skills, but they are really about particular knowledge.  Asking someone to describe OS internals (processes, memory, garbage collection, etc.) is biased toward someone who has recently taken an OS class or the few who work on the guts of operating systems.  It's not that most experienced people can't understand them, it is that they don't need to.  Many coding questions are about syntax or simple tasks no one faces in real life.  Asking someone to swap the words in a string without using extra memory isn't about problem solving as much as it is about knowing the trick.  Expecting someone to be able to characterize the running time of some algorithms is a test for how long it has been since they last took an Algorithms class.  I've even seen people take the simple question of "sort this list" and turn it into "can you implement quicksort in your head?"  Why would anyone need to?
As developers grow in experience, their work changes.  While the specifics of an implementation are still important, the harder work is about the interaction of objects.  Design becomes more important than syntax.  Strategy more important than tactics.  This has an impact on their specific knowledge.  Everyone knows that most of what we learn for a CS degree is of little value in the industry.  Why then should it be surprising that those whose graduation date has a different 3rd digit than that of the current year don't have fast recollection of such facts?  There is only so much information a mind can hold at a time.  It is thus more important to know where to find an answer than to be able to recall the answer off the top of one's head.
If the test of tactical information is not what we need on bigger projects and is not what experienced programmers are best at, how do we change the interview process to dig out the strategic skills we should be looking for?  Unfortunately, I don't have any great answers.  I'm hoping some of you can help here.  My best advice is to think about the questions you are asking.  Ask yourself if the skill required to solve it is something an experienced programmer would use every day.  If not, don't ask the question.  Instead, look for questions that are more central to work people actually do.  Also, make sure to expand the repertoire of questions to include design questions.  These take more time to think up than "code a fifo queue" but are a better judge of the utility someone will bring to the team.  Don't be afraid to ask softer questions about a person's experience.  What were the difficult problems the candidate ran into on their projects?  How did they handle them.  These questions will elicit low-value responses from a recent grad but a cornucopia of information from someone who has been around the block a few times.

http://blogs.msdn.com/b/steverowe/archive/2007/10/12/interviewing-the-experienced.aspx

Ejercicios Diagrama de flujo - Propuestos 3

Tercera serie de ejercicios propuestos para realizar diagramas de flujo.
Los demás ejercicios Parte 1   Parte 2   Parte 3   Parte 4   Parte 5   Parte 6   Ejemplo 1   Ejemplo 2   Ejemplo 3   Ejemplo 4   Ejemplo 5



 1) Calcular el promedio de las 7 calificaciones de un alumno en la materia de Diseño de Algoritmos.

2) Leer 10 números y obtener su cubo y su cuarta.

3) Leer 10 números e imprimir solamente los números positivos.

4) Leer 20 números e imprimir cuantos son positivos, cuantos negativos y cuantos neutros.

5) Leer 15 números negativos y convertirlos a positivos e imprimir dichos números.

6) Suponga que se tiene un conjunto de calificaciones de un grupo de 40 alumnos.
Realizar un algoritmo para calcular la calificación media y la calificación mas baja de todo el grupo.

7) Calcular e imprimir la tabla de multiplicar de un numero cualquiera.
Imprimir el multiplicando, el multiplicador y el producto.

8) Simular el comportamiento de un reloj digital, imprimiendo la hora, minutos y segundos de un día desde las 0:00:00 horas hasta las 23:59:59 horas.
 

9) Una persona debe realizar un muestreo con 50 personas para determinar el promedio de peso de los niños, jóvenes, adultos y viejos que existen en su zona habitacional.
Se determinan las categorías con base en la sig, tabla:

CATEGORIA y EDAD
Niños 0 - 12
Jóvenes 13 - 29
Adultos 30 - 59
Viejos 60 en adelante

10) Al cerrar un expendio de naranjas, 15 clientes que aún no han pagado recibirán un 15% de descuento si compran más de 10 kilos.
Determinar cuánto pagará cada cliente y cuánto percibirá la tienda por esas compras.
 

11) En un centro de verificación de automóviles se desea saber el promedio de puntos contaminantes de los primeros 25 automóviles que lleguen.
Asimismo se desea saber los puntos contaminantes del carro que menos contaminó y del que más contaminó.

12) Un entrenador le ha propuesto a un atleta recorrer una ruta de cinco kilómetros durante 10 días, para determinar si es apto para la prueba de 5 Kilómetros o debe buscar otra especialidad.
Para considerarlo apto debe cumplir por lo menos una de las siguientes condiciones:
- Que en ninguna de las pruebas haga un tiempo mayor a 16 minutos.
- Que al menos en una de las pruebas realice un tiempo mayor a 16 minutos.
- Que su promedio de tiempos sea menor o igual a 15 minutos.

13) Un Zoólogo pretende determinar el porcentaje de animales que hay en las siguientes tres categorías de edades: 

de 0 a 1 año,
de mas de 1 año y menos de 3 y
de 3 o mas años.
El zoológico todavía no esta seguro del animal que va a estudiar.
Si se decide por elefantes solo tomará una muestra de 20 de ellos;
si se decide por las jirafas, tomará 15 muestras,
y si son chimpancés tomará 40.

14) Una compañía de seguros tiene contratados a N vendedores.
Cada uno hace tres ventas a la semana. Su política de pagos es que un vendedor recibe un sueldo base, y un 10% extra por comisiones de sus ventas.
El gerente de su compañía desea saber cuánto dinero obtendrá en la semana cada vendedor por concepto de comisiones por las tres ventas realizadas, y cuánto tomando en cuenta su sueldo base y sus comisiones.

15) En una empresa se requiere calcular el salario semanal de cada uno de los N obreros que laboran en ella. El salario se obtiene de la sig. forma:
Si el obrero trabaja 40 horas o menos se le paga $20 por hora
Si trabaja mas de 40 horas se le paga $20 por cada una de las primeras 40 horas y $25 por cada hora extra.

16) Determinar cuántos hombres y cuantas mujeres se encuentran en un grupo de N personas, suponiendo que los datos son extraídos alumno por alumno.

17) El Departamento de Seguridad Publica y Transito del D.F. desea saber, de los N autos que entran a la Ciudad de México, cuántos entran con calcomanía de cada color.
Conociendo el ultimo dígito de la placa de cada automóvil se puede determinar el color de la calcomanía utilizando la sig. relación:
DÍGITO COLOR
1 o 2 amarilla
3 o 4 rosa
5 o 6 roja
7 o 8 verde
9 o 0 azul

18) Obtener el promedio de calificaciones de un grupo de N alumnos.

19) Una persona desea invertir su dinero en un banco, el cual le otorga un 2% de interés.
¿Cuál será la cantidad de dinero que esta persona tendrá al cabo de un año si la ganancia de cada mes es reinvertida?.

20) Calcular el promedio de edades de hombres, mujeres y de todo un grupo de alumnos.

21) Encontrar el menor valor de un conjunto de N números dados.

22) Encontrar el mayor valor de un conjunto de N números dados.

23) En un supermercado un cajero captura los precios de los artículos que los clientes compran e indica a cada cliente cuál es el monto de lo que deben pagar.
Al final del día le indica a su supervisor cuánto fue lo que cobró en total a todos los clientes que pasaron por su caja.

24) Cinco miembros de un club contra la obesidad desean saber cuánto han bajado o subido de peso desde la última vez que se reunieron. Para esto se debe realizar un ritual de pesaje en donde cada uno se pesa en diez básculas distintas para así tener el promedio mas exacto de su peso.
Si existe diferencia positiva entre este promedio de peso y el peso de la ultima vez que se reunieron, significa que subieron de peso. Pero si la diferencia es negativa, significa que bajaron.
Lo que el problema requiere es que por cada persona se imprima un letrero que diga: “SUBIO” o “BAJO” y la cantidad de kilos que subió o bajo de peso.

25) Se desea obtener el promedio de G grupos que están en un mismo año escolar; siendo que cada grupo puede tener N alumnos que cada alumno puede llevar M materias y que en todas las materias se promedian tres calificaciones para obtener el promedio de la materia.
Lo que se desea desplegar es el promedio de los grupos, el promedio de cada grupo y el promedio de cada alumno.

Do you have Information or Garbage in your Bug Management System

I’m amazed that some people still think of their Bug Databases as “Garbage In – Garbage Out” systems where to STORE bugs as they become available.
When I review the subject objectively, I think these people are both Right and Wrong at the same time.
Since I am a positive person I will start by explaining why they are RIGHT
If you treat your bugs like garbage then that is all they’ll ever be.
If you don’t take the time to define a defect life-cycle that fits your specific needs, if you don’t think ahead about the information you want to capture from each issue when it is reported and handled, and if you don’t do something with the data that “sits waiting” at your fingertips; then don’t expect it to be worth anything at all.
As my father thought me: “nothing good in life is free, if you want to get value you need to work for it”.
Now, let’s review why these people are also WRONG
A well defined and managed Bug Database can be one of the most important assets of a Development Organization.
1. If you use it to manage not only bugs but tasks, enhancements, and all other action items of your team (someone said Agile?); it ensures that everyone is working on the most important things for the project all the time.
2. By keeping track of the rates for bug detection and resolution you are able to understand whether your project is on-track or slipping out of control for its expected release dates.
3. If you think ahead of time about the information you will want to analyze during your project retrospective you’ll be able to collect the data in real-time.
4. If you keep track of your escaping defects and perform root cause analyzes of the reasons they were released (remember to stay clear of witch hunts!), they become an Internal Engine pushing forward your On-Going Improvement Efforts.
5. When you use your Defect Database in conjunction with your CRM (customer relationship management) system – and most importantly when you bring in the issues and enhancements reported from the field – you are able to leverage this information in order to plan, design, develop and deliver the features that will help you to advance your product in the eyes of your customers.
And the list goes on and on… so I will stop in here.
I think it is time we realize that we (testers) are in the business of Process and Product Intelligence. Our bugs are an essential part of the data we use in order to reach concussions and do our jobs better.
Treat your bugs like garbage and that’s all they will ever be, it’s up to you.

http://qablog.practitest.com/2008/11/do-you-have-information-or-garbage-in-your-bug-management-system/

Test For Failure, Not Success

We recently went through a round of test spec reviews on my team.  Having read a good number of test specs in a short period of time, I came to a realization.  It is imperative to know the failure condition in order to write a good test case.  This is at least as important if not more important than understanding what success looks like.
Too often I saw a test case described by calling out what it would do, but not listing or even implying what the failure would look like.  If a case cannot fail, passing has no meaning.  I might see a case such as (simplified): "call API to sort 1000 pictures by date."  Great.  How is the test going to determine whether the sort took place correctly?
The problem is even more acute in stress or performance cases.  A case such as "push buttons on this UI for 3 days" isn't likely to fail.  Sure, the UI could fault, but what if it doesn't?  What sort of failure is the author intending to find?  Slow reaction time?  Resource leaks?  Drawing issues?  Without calling these out, the test case could be implemented in a manner where failure will never occur.  It won't be paying attention to the right state.  The UI could run slow and the automation not notice.  How slow is too slow anyway?  The tester would feel comfortable that she had covered the stress scenario but in reality, the test adds no new knowledge about the quality of the product. 
Another example:  "Measure the CPU usage when doing X."  This isn't a test case.  There is no failure condition.  Unless there is a threshold over which a failure is recorded, it is merely collecting data.  Data without context is of little value.
When coming up with test cases, whether writing them down in a test spec or immediately when writing or executing them, consider the failure condition.  Knowing what success looks like is insufficient.  It must also be possible to enumerate what failure looks like.  Only when testing for the failure condition and not finding it does a passing result gain value.

http://blogs.msdn.com/b/steverowe/archive/2008/06/04/test-for-failure-not-success.aspx

What to think of when defining your Company’s Bug Lifecycle

As software development organizations grow and evolve part of their internally defined processes start loosing their effectiveness and/or efficiency; this means that once in a while they need to re-evaluate these processes in order to adapt and improve them. This is the case of the Defect Lifecycle, that in most organizations needs to be reviewed every 2 to 3 years.
Even if this process appears trivial at first (“Open > Fixed > Closed” right? – WRONG!) the lifecycle of a bug is directly linked to multiple aspects of the organization and it needs to capture these constraints in order to achieve a correct flow definition (I wrote “a” and not “the” correct definition since there are usually multiple alternative flows, each with its pros & cons).

First Stop: What is a bug lifecycle?
The simplest definition of a lifecycle (in our case a bug lifecycle) is a “flow of states” that is followed by every bug in the system. This flow defines the initial (or entry) states, intermediate states, and final (or exit) states; it also defines the paths that may be followed by each bug, intrinsically defining the paths that cannot be followed.
The simplest bug lifecycle I can think off would look something like this:
In addition to the states and paths, some lifecycle definitions also include “transition rules” describing who can move bugs between each state and under what conditions.
For example, we may define that only the development manager can move a bug from NEW to OPEN and that in order to do so the field “Responsible Developer” should be filled with the developer in charge; or we may define that a developer can only set a bug to FIXED if she has filled the “Comments Field” with an explanation of her fix and suggestions to the testing team on how to test it.
Now for the main course: what should we consider when defining our lifecycle?
1. Who logs bugs into the system?
In some cases only the testing team logs bugs into the system, in other cases we provide the customer support team with the possibility to log bugs originated in the field, while some companies may have a strong quality culture where each developer logs bugs detected not only on his peers code but also on his own.
The flow for each of the cases above would be somewhat different: Bugs from the QA may go to a “Dispatcher” that assigns the issue to the engineer who is both qualified to tend the defect but also available based on the urgency of the fix. Bugs from customer support may be reviewed first by the QA in order to assure that the issue has not been already submitted into the system (avoid duplication), that it can be reproduced in-house or in case that it cannot be reproduced that we have the means to understand the issue and validate a fix, and that the description has all the information required to handle the bug. Finally we may decide that bugs that a developer logs to himself can be fixed directly by him, but defects detected on the code of other developers need to pass through the “Dispatcher” for approval and prioritization.
2. What information needs to be entered into a bug?
As people from different internal organizations start reporting defects and as more information is required by your developers you may be tempted to create flows specifically for each internal organization; try to avoid this behaviour.
Paths should be simple and kept to as few as possible; in many cases we may place an internal validator, such as a product manager or a test manager to filter bugs and reject those that do not meet reporting standards or are missing the required information.
3. How many teams/groups handle issues?
Similarly to the influence of multiple organizations reporting bugs, we may have multiple teams handling them. In such cases you want to assign a dispatcher that understands who should handle the issues and what information will they require; consider also the cases where multiple teams may need to handle a single bug and how will you manage the transition between the development teams.
4. How does your company handle communication around bugs?
Some people try avoiding formality in their organizations, I know I do; but we need to differentiate between formality and order when we define our bug lifecycle.
A project will have hundreds if not thousands of bugs reported throughout its lifecycle, and there will be a time when we will review a bug that was handled 2 or 3 years ago in order to understand what was done with it.
When defining your bug flow, make sure to think about logging not only the initial and final information for the defect but also any correspondence or communication around it. The best way to do this is by keeping a comments log where developers and/or QA engineers communicate around the bug (avoiding the use of offline e-mail messaging) and implementing a history tracking mechanism logging the changes done.
5. Is your project running multiple product lines? Can bugs be shared?
If your company has multiple product lines you may need to fix the same bug on multiple places or products. Does this mean one defect report or multiple defects that need to be cloned? Think about who will fix and verify the bugs when making this decision.
6. What do developers do with bugs that they don’t understand or cannot reproduce?
Many companies use a REJECTED status that gives developers a way to request more information about the issues.
The catch with this status is that some engineers abuse it when they are to lazy to try and reproduce the problem by themselves – I worked in a project with a developer would not fix any bug unless the tester sat with him and reproduced the bug on his machine, he lasted about 2 weeks before he got fired!
7. What happens to bugs that we decide not to fixed? (Now or Ever?)
Think about how to handle the defects that are been postponed or delayed.
Many companies create a “bug limbo” where issues get lost once we decide they will not be handled on the current version, should this be the case?
If you decide a bug will not be fixed on the current version then make sure to review it for the next version. Avoid treating a POSTPONED status as a way to avoid rejecting a bug. When rejecting a bug write a comment indicating the reason in order to keep track of your decision (see point 4).
8. What happens to bugs after they are fixed?
Who will validate the bug, the person who reported it or someone who is currently in charge of the feature? How will we communicate this to the tester? Is bug verification done as soon as we receive the fixed build or as part of a concentrated effort in the project lifecycle? How will we return to development bugs that are either not fixed or that were only partially fixed?
These are only part of the questions to answer when looking at the verification part of our process.
9. How do you handle bug fixing updates to the field (Release Notes or Read-Me’s)?
It is important to close the loop with customers who report defects. If a customer opened a support ticket with a bug you want to inform him of the outcome of his request even if the bug was rejected for some reason (this is another reason for keeping track of your rejection considerations).
Additionally, some organizations release a list of important fixes as part of their documentation. How will you select the issues to publish?
For dessert: Technical Considerations
The following 2 considerations are external to the process but they are not less important, and in some cases they will dictate how you work more strongly than many other factors.
10. What information are you looking to get from your defects?
We all use defects statistics to understand the quality of our product, but we can also use them to measure the quality of our process.
For example, do you measure rejection rates to qualify the work of your testers?, or the time it takes to fix and release a bug from the field to measure your support levels?
You can only measure the data that you are keeping track of; but on the other hand if you log every piece of information your database will become un-supportable.
Define up-front what information is or may be relevant and keep track of it; in the worst-case scenario you can always start keeping track of data in the middle of the process and reach conclusions by extrapolating the information.
11. What defect management tool(/s) do you have in place to handle the process?
Every respectable defect management platform lets you define your own automatic bug flow but some are more flexible and offer more advance functionality than others. If you already have a tool in-house then understand what it can do (and how much effort will it take to do it!), and if you don’t have a tool yet define your requirements and then look for one that will be able to handle them (and hope it doesn’t break your cost constraints!)
Many companies have multiple tools that are directly or indirectly involved in this process.
For example some have separate bug tracking tools for their testing and development teams, others integrate the flows between their CRM and bug tracking systems. Even if your tools support an automatic integration mechanism, it will always come at a cost and with multiple constraints; make sure you understand these constraints too.
As I started this post: there is no best approach, specially as the number of people and teams involve in the process grows bigger.
The best advice I can give you is to try to keep your process as simple as possible. In many cases the best way to approach a complex problem is by giving it a simple solution (and training your users on how to adapt to your simplicity constraints).

http://qablog.practitest.com/2008/03/what-to-think-of-when-defining-your-companys-bug-lifecycle/

How to build your Killer SaaS Application?

Hello all!
Usually this blog focuses on QA methodologies, test processes, testing tools, platforms and QA best practices; but from time to time, we will also talk about related topics such as: technology, IT, SaaS (Software as a Service or On Demand solutions) and more.
Many people asked me what are the key things to keep in mind when developing a SaaS application. These are the main points that guide us when we build the PractiTest platform:

• Scale, Scale, and how to scale - that’s one of the main issues to think about BEFORE starting with any piece of code
  How can I make it scale?
  How much will it cost (per additional customer / project / server)?
  Will it add additional maintenance tasks?
  How can I make my servers Highly Available?
  What will happen if the platform will require more than one DB?I know that Agile development is all about how to start the development before you make too many design documents; and not starting and fixing things that aren’t broken yet (and maybe will never be!). So maybe I’m a little old-fashioned, but in my opinion to start and code before taking into account any ‘scale thinking’ is irresponsible. I think that regarding scaling we chose the middle way. Some examples:
  
  • We did not decide before writing the code what would be the indexes in the DB (for example), but when we create the tables, we put some relevant indexes, even when our DB is already working very fast.
  • We made sure that scaling can be achieved by adding relatively low cost machines.
  • We use opensource technologies to make sure scaling by adding machines will not require many $$$ for licensing.
  • We made our code ready for S3 (for file storage) if required, so we will not need to spend many bucks on storage solutions.
• Security – unlike traditional SW, where the DB is located on client’s premises, and all the communication is usually only inside the LAN of the organization, in the case of SaaS systems everything is over the web.
  Security is one of the most critical items, since you are storing and transmitting your clients’ data. Your code needs to forbid ANYONE IN ANY CASE to get or update data unless he’s identified as part of that specific account. The key here is strict standardization within your code.
  We built PractiTest in a way that even WE can’t manipulate or see any client’s data if we don’t have the required permissions.
  Before building any web application, I think anyone should learn about (and how to avoid) things like SQL Injection and Cross Site Scripting.In addition, security also means respecting the users’ privacy and sensitive data:
  • Hashing users’ password (MD5), before saving it to your DB.
  • Filter sensitive information from the default logs (credit-card info, password).
    Build it in a way that even you (or the system admin) will never be able to find users’ sensitive data.
  • Get an expert to try and manipulate your data.Special Tip for testers: try to find a user password on the DB or logs, or try to manipulate/get data that you’re not allowed to access, using http post or get. If you succeed it means there are issues with the product. These issues are showstoppers for every SaaS products, they are also important to fix in any SW product.

• Fast upgrades – SW upgrades shall not take more than few minutes. This should include DB & files backup, latest code version backup (in addition to version control), modifications to the DB schema (if required), upgrade of the code, and sanity testing.
  All the above should be automated. As a rule of thumb, the update should last no more than 10 minutes. If there is a failure, make sure you can rollback within 1 minute. Fast rollback is the main reason we backup DB, files, and code version, before we start the upgrade.
  As a SaaS solution you are expected to quickly solve every problem that may occur; also adding modules is something that you should be able to do easily.
  For example, in PractiTest, we usually do bi-weekly updates since we are adding features all the time.
• Make It Work Fast! The last thing your users (and ours) want is to wait (while you load your application). Actually some customers choose us mainly due to the fact that PractiTest works much faster than the products they used to run in-house.
  How could that be if they run it on their local network!? These are the things we keep in mind:
  • Small javascript files, small download pages.
  • We use only one JS framework. More than one can make the browser ‘become heavy’.
  • We make sure we have enough network capacity to support our clients.
  • We constantly check DB queries on production. Making sure all run fast.Our platform already works very fast for our clients, and this is even without (yet) things like:
  • Javascript and CSS compression (such as YUI Compressor or Packer)
  • Browser image cachingFor additional techniques you can look here: http://developer.yahoo.com/performance/rules.html
• No Installs for the clients – When you are developing a product that is installed on the customer’s premises, you can sometimes get away supporting only IE with an ActiveX based product. But as a SaaS your platform needs to support all main browser without instaling ANYTHING.
• Adjustments as needed – this is only one of the SaaS advantages – if there is a problem, fix it on the fly.

As I mentioned before, this is obviously a partial list of our key points. I mainly wrote what to do and not so much how to do it properly. If you want to know more about how to do it (or how we do it in PractiTest) drop me an email or post a comment.

http://qablog.practitest.com/2008/11/how-to-build-your-killer-saas-application/

A good tester asks good questions

Testing a new application or product is always challenging.
There’s the challenge of working with new technologies, new languages and even new platforms; forcing us to investigate and learn how to cope with these new paradigms.  But in a sense all these technical issues can be easily solved by turning to the best-friend of most geeks (like me).

Chances are that if you are asking yourself what tools to use in order to automatically test a new technology, how to write tests for a specific scripting language, or what are the weak spots on an operating system or mobile platform; the answers will already be available on the Internet, written by some early adopter or tester who took on the challenge and posted a paper or blog explaining what he did and how.
But the real challenge is coping with the things that cannot be searched on the Internet, and the ones you will need to figure out by yourself. I am referring to the functional and business aspects of your new application and the way your users will work and interact with your product once it is released.

Who do I  talk to in order to get information?

The first question to ask is “Who do you need to talk to in order understand what to test?” and the answer, at least in the beginning of the process, is simple – TALK TO EVERYONE YOU CAN.
When you start a new project or when you’re asked to test a new application you need to start by mapping all the people who can provide you with information.  So make a list of everyone you can talk to – Marketing, Development, Sales, Support, the CEO – and try to talk to each of them in order to understand what you need to test (and why)!!!???
Even if you don’t manage to talk to all of them (sometimes it can be tricky to get a 30 min session with your CEO…) at least try to make the list and cover as many different people as possible in order to get different perspectives and testing inputs.

What questions to ask?

Here is the hard part of the problem…
Imagine the following scenario:
You finally schedule the meeting with your CEO in order to get her inputs into what should be tested in the system. You arrive at the meeting and ask her: “Mrs. CEO, I have been given the task of testing the new product, and I wanted to ask you what do you think should be tested?”.
She stops and looks at you for a couple of seconds before answering: “Well, I don’t know, aren’t you the testing expert here…?  I guess you better test everything, right?  We don’t want any bugs slipping out the door, do we?”
What’s wrong with the scenario above?  Well, basically we came to the person and asked him the wrong questions…
If you pay attention to what the CEO said she was right, we are the testing experts.  She expects you to come up with what should be tested in the system, but on the other hand she is also living under the illusion that everything should or even can be tested (something we all know is impossible and even when possible, not economical in the long run).
The problem here is that we asked someone else to do our work for us, to come up with what to test, instead of asking for the input we need to understand by ourselves what needs to be tested and how.
So going back to our CEO meeting scenario, what questions would have been valuable to ask?  Well, you need to ask for her inputs on the system, without even talking about the testing operations.  Try to understand what areas are important from a user perspective, or based on our competitive advantages, or based on what makes our application unique, etc.  You should focus your questions on what is important for them.
Here are a couple of examples you can ask your CEO or VP Marketing or even Director of Sales:
- What are the most important aspects of our Product?  The things that make us unique?
- What areas in the product are the most widely used by our customers?  What type of things would make our users angry and make them choose not to work with our product?
- Where is the market focusing on today?
- How are we better than our competitors?  What areas are the ones that are the most problematic in our competitors, the same areas where we want to exceed?
- Are there any risks you think we should be specially aware off?  Risks in our technology, risks in the product?
Notice that we didn’t ask them what to test, but we did ask what’s important in their eyes (based on their experience).
In the case of the CEO, Marketing and Sales functions we will want to talk about stuff that relates to the functionality of the product.
If we were to talk to our Support Team we would ask them questions related to the areas in which our users find bugs, focusing both on the places where there are the largest amount of bugs as well as where the most critical issues are found.
Finally, when talking to our development peers we will ask them about technological risks, as well as places where they are making the most changes, or where the product is relatively weak or complex and so where we should be putting more testing efforts.

The art of listening (and putting together the puzzle)

So what do you do with all the information?  Basically you need to take the stuff you got, and process it in order to get a 360-degree reading of your application.  What do I mean by 360-degrees?  I mean from all the different angles that matter: Technology, Usability, Supportability, Competitiveness, etc.
After all, your work is to test and to provide visibility into whether the application is meeting the quality standards of your users and stakeholders.  The only way to do this is by understanding what’s important to all of them and creating a test plan (or work plan!) that will effectively cover it.

http://qablog.practitest.com/2011/04/a-good-tester-asks-good-questions/