- Take your testing up a level from test cases to techniques
Test cases are a common unit of measurement in the testing world. we count the number of them that we run, and when we find a bug, we proudly present the test case that we used to find it.
I hate it when we do that.
I think test cases - and most discussions about them - are generally meaningless.
I propose we talk in more high-level terms about test techniques instead. for example, when one of my testers finds a bug, they often come to my office to demo it to me (i am a well-known connoisseur of bugs and these demos are often reused in lectures i give around campus).
Most of them have learned not to just repro the bug by re-running the test case. They know i am not interested in the bug per se but the context of its exposure. there are the type of questions i like to ask during bug demos:
1. what made you think of this particular test case?
2. what was your goal when you ran that test case?
3. did you notice something interesting while running the test case that changed what you were doing?
4. at what point did you know you had found a bug?
At Microsoft, we have begun using the concept of test tours as a way to categorize test techniques.
- It is our task to use testing as an instrument of improvement
- This is the true job of a tester: to make developers and development better; we don't ensure better software - we enable developers to build better software; it isn't about finding bugs, because the improvement caused is temporal
The true measure of a great tester is that they find bugs, analyze them thoroughly, report them skillfully and end up creating a development team that understands the gaps in their own skill and knowledge
Making developers better, helping them understand failures and the factors that cause them will mean fewer bugs to find in the future
The real value of tests is not that they detect bugs in the code, but that they detect inadequecies in the methods, concentration and skill of those who design and produce the code (Tony Hoare)
- Testing without innovation is a great way to lose talent
Testing sucks. now let me qualify that statement: running test cases over and over - in the hope that bugs will manifest - sucks. it's boring, uncreative work.
What is interesting about testing is strategy, deciding what to test and how to combine multiple features and environmental consideration in a single test. the tactical part of testing, actually running test cases and logging bugs, is the least interesting part
