This blog post aims to address a controversial topic within the QA tester community: test cases. When I first read about this topic, I was wondering whether test cases are actually an asset or a hindrance to the QA testing process. The following paragraphs are meant to help us distinguish between the advantages and disadvantages of different test cases.
On one hand, some people say that it is essential to design a test plan and have the whole list of test cases before starting to test. These people consider test cases an asset. On the other hand, some other people say test cases are actually a hindrance to the QA testing process since the tester would then be biased, thus limiting creativity by focusing on the test cases.
First, I am going to present the information supporting the idea of test cases being an asset.
Test cases are an asset
As we know, there are some projects which lack documentation for different reasons; given the circumstances, these teams cannot provide new teammates with the proper information to understand the product. In this case, new people on the team can use test cases as the documentation they need in order to understand the project.
Other people consider test cases important as it shows stakeholders and/or clients the parts of the system which have been tested; in other words, what testers have worked on. From this perspective, test cases are such a nice tool for the QA testers to justify their work.
Moreover, writing test cases before running any tests is also considered good practice so that future testers are able to run the necessary tests once the person who designed the tests is no longer in the project.
For other testers, creating the test cases before even getting their hands onto their new product is great to learn the product. By executing a set of tests, they will get a pretty good hands-on experience without having to use their peers’ time to give them a tour of the product.
Test cases are a hindrance
For instance, according to James Bach and Aaron Hodder in their article “Test cases are not testing: towards a culture of test performance”, testing cannot be predicted for two reasons, on the one hand, there is always more testing to do than what we can afford to do, and on the other hand, we don’t know where the bugs are until we find one. Creating test cases before testing may bias the tester.
Similar to what happens with recipes, which are not cooking, we need to bear in mind that a test case is not a test, and should avoid using an artifact as the basis for human performance; also keep in mind that with tacit knowledge and skill, the artifacts are not central, and they might not be necessary whatsoever. So the test case may have a role, but the tester is indeed the center of testing.
Taking the example of the recipe into account, the performance of the tester should have substantial freedom to make her own choices from one moment to another. Having said this, if a tester has a test case, it is hard to make decisions at the moment of testing because the tester is then biased by the test cases being followed.
Test cases do not cover as many scenarios as a tester would like, for several reasons. A test case, most of the time, can cover the happy path and the most common paths for a feature, but they lack focus on elaborate scenarios, tricky paths a normal user might follow. Having test cases biases the tester. Testers are then tricked into focusing on the scripts, rather than focusing on learning about the product, or finding bugs. They may execute the test cases to the letter, and they can perfectly carry out this activity without finding any defects, which is the main objective of a tester, not to have bugs on the product.
Based on my experience regarding test cases and whether they are convenient or inconvenient for the QA testing process, I do have my own take. In the end, I believe it all comes down to being open to the needs of the product, meaning there will be some times when test cases will be quite useful, but some other times, it’ll be better not to stick to a document. After all, testing is more about how testers perform, rather than the test cases per se.
For instance, if regression testing will take place, the best approach might be to use scripts so the tests are carried out straight forward. If the testing is not about checking the main functionalities in a product, the following scripts might not be the best approach since the tester should be free of biases, the product should be explored in great detail. Important to take into account the fact of keeping documentation once testing is done, even though there are no test cases, the tester should keep notes of the results, for further situations.