How to Plan for a Testing Project

Harshani Perera, 29 April 2021

When you are presented with a project to test, it can be overwhelming at first in deciding what to test to ensure a quality product will be delivered to the customers.

There are an infinite number of tests that can be run but it’s not possible to test everything. You are also bound by two big factors: budget and time hence you need to determine where to concentrate your testing effort. How to determine this in a way that delivers a quality product within the given budget and time?

Reading through the book ‘Managing the testing process: Practical Tools and Techniques for Managing Hardware and Software Testing’, I came across a good testing approach described by Rex Black that provides a satisfactory guide in answering the above question.

Before doing ANY testing effort, you need to first figure out the answers to the following:

1. What you might test

2. What you should test

3. What you can test

What you might test: Extended Test Effort

Some testing effort can be the responsibility of another team hence it would be a waste of time and would add little value to test something that is already being covered. Therefore, what you might test are all the untested areas that fall within your scope for testing. t

Tests you might carry out are:

· White Box testing (Structural)

· Black Box testing (Behavioural)

· Alpha/Beta testing (Live)

· Unit Testing

· Integration testing

· System testing

· Usability testing

· Acceptance testing

· Regression testing

What you Should Test: Considering Quality

The next step is to identify what we should test from those untested areas.

To do this, find out what quality means for the company and the client then identify the risks that exist for this quality. From these risks, you need to then identify the critical quality risks. Critical quality risks are the risks that relate to how the customers will actually use the product.

The following example adapted from Rex Black’s book strives to explain how to identify these critical quality risks.

A good test system

A good test system will allow the tester to understand the users’ experience of quality.

This test system allows the tester to achieve a good coverage of the usage experienced by the customer.

A poor test system

This test system does not allow the tester to understand the users’ experience of quality. The customer usage and the critical quality risks are not covered. However, if the tester’s coverage aligned with the customer’s usage, this would have been a good test system.

The following graph adapted from the book compares Test coverage vs the Quality risk coverage.

We should be aiming to be more towards the right-hand side to ensure customer use coverage. The higher up we go, the more quality risks we will cover hence a better test system. If all the quality risks are covered with no customer use coverage, then this is a poor test system.

In summary, what you should test are the areas that affect the users’ experience of quality. Testing should focus on finding the critical defects that prevent users from getting their work done with the product.

At this step, you should have a prioritised list of the critical quality risks prepared.

What you Can Test: Schedule, Resources, and Budget

A project is limited by time and budget hence it’s not possible to test everything. We need to make some difficult choices on what we will focus on by selecting the highest quality risks from the list.

Hence, what you can test are the untested critical areas using the limited time and budget you have been given.

If the project needs to be completed earlier and/or on reduced costs, you can eliminate test cases from low priority to high priority to meet this target. If you know that this will create risks to the project, you should inform the management of later discoveries that could cause overruns in budget and time so that the team are aware of the risks and can either accept this and go forth or make changes. It is difficult to make changes to the budget and/or schedule towards the end of a project so it’s better to inform the team as early as possible.

Alternatively, if you have been given a fixed schedule and a budget, then the only thing you can agree to do is the best testing possible with the given limitations.

I hope this provides you with some guidance on planning for a successful test project that will deliver a quality product and create a positive experience for your customers.