“Quality is never an accident. It is always the result of intelligent effort”
The emphasis on quality has never been more focused than it is today. In a world where the consumer is spoiled for choice, quality is what separates a successful product or service from an unsuccessful one. The emphasis on quality in the software development space has been increasing as well. However, software developers are not only tasked with the role of developing high-quality products, they also have to deliver the same in the shortest possible timeframe. Functional testing often is the elephant in the room when it comes to continuous delivery since most of it is still manual testing. Hence the role of test automation as a critical enabler of software quality has become firmly established in the industry.
With development methodologies such as DevOps, where Continuous Testing lies at the heart of the development process, it becomes imperative to ensure that all the code components and functionalities are operating as desired. In order to do so, software testing experts have to take a close look at their test design. They must ensure that the tests cover a significant portion of the software, paying especially close attention to those components that have a high impact on the user as well as the business. Unless this is done, the testing effort risks falling short of the desired impact.
The Fundamentals of Test Design
A number of testers take the requirement-driven approach to start their automation testing initiatives. This entails taking a stock of all functional and non-functional requirements that are expected of the software, discussing these with the development team and then designing the test cases. This approach is absolutely useful for ensuring no functional gaps are left in the test coverage. At the same time, a comprehensive test design should also take into account the required code coverage, as well as the effort required for testing.
“One characteristic of good test design is the achievement of maximum testing with the minimal number of tests and minimum effort in the shortest possible timeframe.”
Two of the most commonly used approaches for test design are -
Black Box Testing techniques
Black Box testing techniques are input and output based. Black box testing implements specification and experience-based techniques to design test cases taking into account specifications, requirements, software descriptions etc. Black box tests can be both functional and non-functional however, the tester does not need to have any knowledge of the internal structure of the code or the software under test.
Some Black Box test design techniques include,
- Boundary Value Analysis
- Equivalence Class Partitioning
- State Transition Diagrams
- Orthogonal arrays
- Decision Trees and Decision Tables
White Box Testing Techniques
White Box testing techniques are structure based software testing techniques that focus on the internal structure of the code component under test. The aim of this technique is to write code to test the code by acquiring intimate knowledge of the code under test. These techniques hence demand information regarding how the software is composed. This testing technique calls for the developers to design the test cases in the code itself and then call for code inspection and code peer reviews.
Some White Box testing techniques include,
- Condition testing
- Statement testing
- Loop or LCSAJ testing
- Decision Testing
- Path Testing
Test Design is all about optimum risk coverage
As mentioned, the focus of test design must be on maximum coverage with minimal effort. The first step is to determine the appetite for quality risk. How is the balance between the schedule risk v/s the quality risk? A calibration of various parameters in alignment with the risk appetite must then follow. These parameters include the percentage of functional coverage for various parts of the application, code coverage, and priority order of test areas among others.
One crucial parameter is Code Coverage. To achieve the right code coverage, testers need to run the test cases over the software under test and run a code coverage tool in parallel. The code coverage tools provide insights into all the test cases that have run and report the lines of code that were tested. This visibility helps greatly in reducing quality risks as testers gain a quantified insight into the amount of code that has been tested. This means they can tweak their test cases as per need. For example, in test automation initiatives, an objective could be achieving at least 90% of code coverage for non-core components and 100% coverage of the core components of the software under test. The report from the code coverage tool can reveal the gaps in testing if these numbers are not achieved and raise the red flags for areas that need greater testing focus.
Both developers and testers should do a constant review of the code coverage report to identify the gaps that need to be addressed.
By taking a more considered approach to test design, testing and development teams can ensure more effective testing and thereby deliver high-quality products faster. Clearly, while testing, areas such as exploratory testing are an art that demands creativity from the testers, test case design is a science that is exact in nature. For testing and automation testing initiatives to be successful, there has to be an orchestrated synergy between the two. Perhaps exploring this coming together of art and science in testing could be the subject of a future blog?