Many projects are connected with such situations when bugs develop some kind of “immunity” with each test, while QA engineers try to find these errors using their advanced technologies and methodologies of bug searching.
When the same tests are reproduced one-by-one all the time, after a while, they stop finding new bugs and defects. In such a case, even a well-done test case for some functionality won’t allow performing proper testing, and as a result, the client can receive a product, full of undiscovered bugs and errors.
Like in an insect’s life, in the field of software testing, all bugs can develop some “resistance” to the same testing method or approach (in the agricultural field, it’s pesticides).
Such a system phenomenon is called the pesticide paradox in the testing.
Almost 30 years ago, Boris Beizer, American engineer and the author of this concept, proved the existence of such technological phenomenon in practice.
The nature of such a paradox is that multiple usages of similar testing methods become non-effective in a due course because “survived” bugs develop a special immunity.
Why Doesn’t the Time Spare Even the Most Qualitative Test Case?
Naturally, the simplest things can be, for example, changing the project, adding some paid functionality or beginning the work on optimizing the previously developed functionality.
The updating of the test case is very important in such a situation, even without the pesticide paradox.
But even if the project wasn’t updated for a long time, it’s necessary to add new improvements to the test cases in order to find new bugs that naturally can be met in any software.
Practically, there’re a few reasons, why the same test suite stops being useful and effective in time:
- Nobody can ever foresee all actual and potentially possible scripts for testing in the beginning. Even if a developed product is really simple and doesn’t have a confusing functionality, it’s impossible to create enough amount of tests, which can help to find all mistakes. And we don’t even say about difficult systems with a large number of incoming data, development scenarios and created dependencies. That’s why the tester should search for some new ideas and solutions, which in the future can help to deal with the unknown bug;
- After some time, the tester gets used to the project particularities, remembers its weak points, and there’s a risk that he/she can miss bugs or take a defect as a functional feature, that was added some time ago. When QA specialist has been working on a project for more than 1 year, he/she must perform a regression testing and “go through” the same test suite in order to save a fresh look at the product – and this is a really difficult task.
How to Support Created Test Cases?
It’s very important to use different sets of approaches and methodologies during the performance of any testing. It is done in order to look at the created software in various ways: from the sides of project architecture, its functionality, and data processing (incoming and outgoing).
Also, it is important to understand how the clients will interact with this software, what the problems can be and how to deal with them.
For example, there’re several mnemonics that help any tester to raise a number and efficiency of tested projects.
One of them is SFDPOT (San Francisco Depot) by James Bach. Every letter of SFDPOT stands for some aspect of the tested product, that should be thoroughly checked:
- Structure – means the structure of software;
- Function – what software does, its basic functions;
- Data – how the data is processed;
- Platform – what platforms support created software;
- Operations – how the software will be used;
- Time – how the software will act after some time.
There’s another mnemonic I SLICED UP FUN, by Jonathan Kohl, that was developed to test the productivity of mobile apps. It reminds what exactly should be tested:
- Input – potential ways of transferring data to the app;
- Store – a specification of apps stores;
- Location – a location of potential clients;
- Interactions – functioning interactions;
- Communications – communications;
- Ergonomics – ergonomics;
- Data – processed data;
- Usability – its easy usage;
- Platform – supported systems and platforms;
- Function – available functionality;
- User scenarios – scenarios of usage;
- Networks – network.
The other mnemonics can be used, such as:
RCRCRC (Recent – Core-Risky-Configuration sensitive-Repaired-Chronic) – for effective assistance during the regression testing.
COP FLUNG GUN (Communication-Orientation-Platform-Function-Location-User Scenarios-Network-Gestures-Guidelines-Updates-Notifications) – for testing the mobile app productivity.
Taking into account various insights into a product, we can create a group of really useful and multifunctional tests.
Usage of mnemonics can manage the risk that at some point, a tester can forget to pay his/her attention to the important detail of the created software.
Performing the tests only with some mnemonics, the tester realizes that all important components of the project were held and nothing was missed.
Of course, mnemonics aren’t the universal mantra of creating a full test plan, but a helpful assistant in ideas and methods generation.
The created test suite should be constantly added with new tests, in order not to face the situations when this suite doesn’t align with tested software and isn’t actual anymore. It should be done every time when new functionality is added to the software.
These tips will help QA specialist to support the applicability of created test cases:
- Do the constant editing of test cases, when the architecture and internal structure of the software are being changed. When you begin creating some test case, you need to take into account that it may be edited in the future. You or your future colleague will thank you for this;
- Remove the old test cases from the actual test suite. If you see that functionality (that your tests were created for) was deleted from the software – you need to archive such test till this or similar functionality won’t be added to the project (current or future) again. You don’t need to delete the test suite, because there’s a chance that functionality may be added by client’s requests, software user’s wish or after other factors coming;
- Create new test cases, that allow testing the functionality of new improvements. Always when the QA team has to test new functionality, testers should take care of writing actual test suite, that can be expanded and improved in the future. Of course, it can happen that the task should have been done yesterday and the project should be tested as soon as possible. Even in these situations, you have to record the names of the scenarios. It can help to deal with test cases totally thoughtfully when filling its attributes;
- Create the test cases for scenarios testing, connected with bugs, that were found by testers or a client previously. Such a practice allows adding new improvements to the list of test cases, which the tester can forget while he/she designs the main base of test suites.
Constantly improving the test suites, the tester can find new and “interesting” bugs, look at the tested software in different ways, and always be completely sure that project was released as a really qualitative and working product.
As a result, the QA specialist never gets bored on the project, because he/she doesn’t need to run the same test suites anymore, which will decrease the profit from testing.