Testing Wisdom
A test is an experiment designed to reveal information or answer a specific question about the software or
system. Stakeholders have questions; testers have answers.
Don’t confuse speed with progress.
Take a contrary approach.
Observation is exploratory.
The narrower the view, the wider the ignorance.
Big bugs are often found by coincidence.
Bugs cluster.
Vary sequences, configurations, and data to increase the probability that, if there is a problem, testing will find it.
It’s all about the variables.
Don’t confuse speed with progress.
Take a contrary approach.
Observation is exploratory.
The narrower the view, the wider the ignorance.
Big bugs are often found by coincidence.
Bugs cluster.
Vary sequences, configurations, and data to increase the probability that, if there is a problem, testing will find it.
It’s all about the variables.
Ubicaciones/Archivos
- Nombres Largos (>255 caracteres)
- Caractéres especiales en el Nombre (espacio * ? / \ | < > , . ( ) [ ] { } ; : ‘ “ ! @ # $ % ^ &)
- No Existente
- Ya Existente
- Sin espacio
- Mínimo Espacio
- Protegido contra Escritura
- No Disponible
- Bloqueado
- En una Máquina Remota
- Corrupto
Tiempo y Fechas
- Timeouts
- Time Difference between Machines
- Crossing Time Zones
- Leap Days
- Always Invalid Days (Feb 30, Sept 31)
- Feb 29 in Non-Leap Years
- Different Formats (June 5, 2001; 06/05/2001; 06/05/01; 06-05-01; 6/5/2001 12:34)
- Daylight Savings
- Changeover
- Reset Clock Backward or Forward
Números
- 0
- 32768 (215)
- 32769 (215 + 1)
- 65536 (216)
- 65537 (216 +1)
- 2147483648 (231)
- 2147483649 (231 + 1)
- 4294967296 (232)
- 4294967297 (232 + 1)
- Scientific Notation (1E-16)
- Negative
- Floating Point/Decimal (0.0001)
- With Commas (1,234,567)
- European Style (1.234.567,89)
- All the Above in Calculations
Cadenas de caracteres
- Long (255, 256, 257, 1000, 1024, 2000, 2048 or more characters)
- Accented Chars (àáâãäåçèéêëìíîðñòôõöö, etc.) Asian Chars (漢字 )
- Common Delimiters and Special Characters ( “ ‘ ` | / \ , ; : & < > ^ * ? Tab )
- Leave Blank
- Single Space
- Multiple Spaces
- Leading Spaces
- End-of-Line Characters (^M)
- SQL Injection ( ‘select * from customer )
- With All Actions (Entering, Searching, Updating, etc.)
General
- Violates Domain-Specific Rules (an ip address of 999.999.999.999, an email address with no “@”, an age of -1)
- Violates Uniqueness Constraint
Web Tests
Navigation
- Back (watch for ‘Expired’ messages and double-posted transactions)
- Refresh
- Bookmark the URL
- Select Bookmark when Logged Out
- Hack the URL (change/remove parameters; see also Data Type Attacks)
- Multiple Browser Instances Open
Input
- See also Data Type Attacks
- HTML/JavaScript Injection (allowing the user to enter arbitrary HTML tags and JavaScript commands can lead to security vulnerabilities)
- Check Max Length Defined on Text Inputs
- > 5000 Chars in TextAreas
Syntax
- HTML Syntax Checker (http://validator.w3.org/)
- CSS Syntax Checker (http://jigsaw.w3.org/css-validator/)
Preferences
- Javascript Off
- Cookies Off
- Security High
- Resize Browser Window
- Change Font Size
Heuristics
Variable Analysis
Identify anything whose value can change.
Variables can be obvious, subtle, or hidden.
Variables can be obvious, subtle, or hidden.
Touch Points
Identify any public or private interface that provides visibility or control.
Provides places to provoke, monitor, and verify the system.
Provides places to provoke, monitor, and verify the system.
Boundaries
Approaching the Boundary (almost too big, almost too small), At the Boundary
Goldilocks
Too Big, Too Small, Just Right
CRUD
Create, Read, Update, Delete
Follow the Data
Perform a sequence of actions involving data, verifying the data integrity at each step.
(Example: Enter → Search → Report → Export → Import → Update → View)
(Example: Enter → Search → Report → Export → Import → Update → View)
Configurations
Varying the variables related to configuration (Screen Resolution; Network Speed, Latency,
Signal Strength; Memory; Disk Availability; Count heuristic applied to any peripheral such as 0,
1, Many Monitors, Mice, or Printers)
Interruptions
Log Off, Shut Down, Reboot, Kill Process, Disconnect, Hibernate, Timeout, Cancel
Starvation
CPU, Memory, Network, or Disk at maximum capacity
Position
Beginning, Middle, End (Edit at the beginning of the line, middle of the line, end of the line)
Selection
Some, None, All (Some permissions, No permissions, All permissions)
Count
0, 1, Many (0 transactions, 1 transactions, Many simultaneous transactions)
Multi-User
Simultaneous create, update, delete from two accounts or same account logged in twice.
Flood
Multiple simultaneous transactions or requests flooding the queue.
Dependencies
Identify “has a” relationships (a Customer has an Invoice; an Invoice has multiple Line Items).
Apply CRUD, Count, Position, and/or Selection heuristics (Customer has 0, 1, many Invoices;
Invoice has 0, 1, many Line Items; Delete last Line Item then Read; Update first Line Item; Some,
None, All Line Items are taxable; Delete Customer with 0, 1, Many Invoices)
Constraints
Violate constraints (leave required fields blank, enter invalid combinations in dependent fields,
enter duplicate IDs or names). Apply with the Input Method heuristic.
Input Method
Typing, Copy/Paste, Import, Drag/Drop, Various Interfaces (GUI v. API)
Sequences
Vary Order of Operations Undo/Redo Reverse Combine Invert Simultaneous
Sorting
Alpha v. Numeric Across Multiple Pages
State Analysis
Identify states and events/transitions, then represent them in a picture or table. Works with
the Sequences and Interruption heuristics.
Map Making
Identify a “base” or “home” state. Pick a direction and take one step. Return to base. Repeat.
Users & Scenarios
Frameworks
Judgment
Inconsistencies, Absences, and Extras with respect to Internal, External – Specific, or External –
Cultural reference points. (James Lyndsay, Workroom Productions)
Observations
Input/Output/Linkage (James Lyndsay, Workroom Productions)
Flow
Requirements
Users/Functions/Attributes/Constraints (Gause & Weinberg Exploring Requirements)
Nouns & Verbs
The objects or data in the system and the ways in which the system manipulates it. Also,
Adjectives (attributes) such as Visible, Identical, Verbose and Adverbs (action descriptors) such
as Quickly, Slowly, Repeatedly, Precisely, Randomly. Good for creating random scenarios.
Deming’s Cycle
Plan, Do, Check, Act
Fuente: Blog Test Obsessed Test Heuristics Cheatsheet Permalink
