Estimation Techniques for Software Testing: Which to Choose for Your Project

Time is money, and finding out how long it will take to test your product and how much it will cost is one of the first steps in your cooperation with a QA team. Nowadays, there are dozens of methods for estimating projects. In our article, we describe the most common estimation techniques along with their advantages and limitations. We also reveal techniques preferred by the RubyGarage QA team when assessing new projects. 

#1 PERT

The program evaluation and review technique (PERT) relies on three variables to predict project duration: the most optimistic, most pessimistic, and most likely cases.

The most optimistic case (O) is an assumption as to the length and cost of the project under the best conditions possible. Such conditions imply that all team members perform their duties on time and without schedule slips, force majeure circumstances, or a need for rework.

The most likely case (M) represents an estimate considering the team’s usual workflow, schedule slips, and rework.

The most pessimistic case (P) is the worst scenario possible. To calculate P, estimators suppose how much time it will take and how much it will cost to test a product if everything goes wrong at every stage of testing.

Once all three variables are defined, the estimate (E) is calculated in the following way:

Benefits

The PERT model works according to a what-if technique that allows for estimating possible risks. Thanks to this technique, an assessment team can provide a realistic estimate.

Another benefit of this model is that it gives a clear understanding of various scenarios that may take place during product testing. Thus, a QA team can be ready to meet possible obstacles and mitigate them in time.

Limitations

This technique isn’t suitable for large projects with a lot of variables. It may take quite a lot of time to estimate such projects, and there’s a higher possibility of inaccuracies.

Additionally, the variables used for PERT estimation are subjective, so there’s still a chance of an error.

#2 UCP

The use case points (UCP) method is based on a project’s use cases. 

UCP estimation is the calculation of different variables that influence project development and complexity. These variables are actors, adjusted and unadjusted use case weights and points, technical complexity factors, and environmental factors.

Actors are users or programs that interact with tested software. Unadjusted use case weights are things that influence a project’s size. Technical complexity and environmental factors influence the project at the technical and environmental levels. According to Gustav Karner, there are more than 10 factors that determine the technical complexity of a project and eight that influence its environmental complexity.

Estimation starts with each variable being assessed in accordance with its complexity and influence on a project. Then these variables are calculated with the help of special formulas. The overall size of a project is determined according to the following formula:

Once a QA team knows the size of a project, it’s time to find out the duration of testing. There are two ways of calculating how long it will take. It’s possible to use Karner’s approach and regard each use case as 20 person-hours. Another way is to rely on your company’s experience and estimate project length based on the time use cases usually take in your company.

Benefits

UCP estimation can be performed at the early stages of a project. It helps you plan testing activities and approve your budget in advance.

Moreover, it’s possible to automate estimation with the help of use case management tools so an assessment team spends less time estimating.

Limitations

The UCP method can be applied only to projects where specifications are expressed in use case points. Otherwise, the assessment team needs to opt for another technique. 

Further, quality results from the UCP method depend on clear use cases. If use cases are poorly written, the resulting estimate is likely to be inaccurate. 

#3 WBS 

The work breakdown structure (WBS) method relies on splitting the whole scope of work into smaller tasks. Then, QA team members assess how much time it will take to accomplish each task. After each task is estimated, the time is added up to understand the duration of the project.

In the image below, you can see an example WBS estimation carried out by the RubyGarage team for a whole project, including QA activities.

Benefits

The WBS technique ensures a detailed estimate. By dividing a large task into smaller ones, there’s a lower chance of missing things, and the assessment team will each task that influences project completion. 

Clarity is another advantage of this technique. WBS results are represented in a table to ensure transparency and make it easy to track progress.

Limitations

Being resource-intensive, the WBS technique takes a lot of knowledge, usually from different areas. Thus, a WBS assessment involves all stakeholders.

Also, WBS estimates might become outdated over time since project requirements can change at different stages. If a client decides to change some functionality, an assessment team will have to re-estimate parts of the project.

#4 Wideband Delphi

The idea of the Wideband Delphi technique is to gather an expert team that usually consists of three to seven people: a moderator, a project manager, and other team members who can contribute to project evaluation. Here is how it usually goes:

1. An expert team meets for the first kickoff meeting to learn the main information about the project ‒ its goals, requirements, the client’s preferences, etc. During this meeting, each team member provides an estimate of the time needed to complete the project.
2. During the second meeting, a moderator presents the results of the fist estimate anonymously so team members don’t know whose ideas are represented. The team members then analyze the results, taking into account the ideas of other participants. Thus, they have a look at the project from different points of view. Then they re-estimate the project.
3. In the third meeting, new estimates are presented and team members try to reach an agreement once more. Now the estimation timeline is reduced, since team members share similar ideas.
4. Meetings continue until all team members agree on a single estimate.

This is the traditional way of performing Wideband Delphi estimation. However, companies can alter the procedure. For example, it’s possible to carry out all estimation rounds during one meeting or hold an open discussion instead of presenting estimates anonymously. 

Benefits

Wideband Delphi estimation is easy to carry out. It’s possible to start assessing the project once a QA team gets requirements from the customer. Moreover, such estimation doesn’t require any special preparation or tools.

Expert participation increases the chances of a precise estimate since the project is looked at from different perspectives. 

Limitations

Applying this technique is time-consuming. It can be difficult to come up with a single estimate during the first round. That’s why such an assessment requires two, three, or sometimes more rounds. 

Additionally, the results of such an estimate are impossible to reuse. This estimation technique can’t be applied to other (even similar) projects since it’s hard to document the assessment process and use it as a basis for future estimates. Every time it should be applied, it must be carried out from scratch.

#5 Function points

Function points (FP) are the units used to measure project size that consequently influences its duration. Finding out the size of a project is possible on condition that a QA team knows the following variables: unadjusted function point, total degree influence, and value adjustment factor. Once all the variables are calculated, a team identifies adjusted functional point that equals the project size using the following formula:

To carry out an accurate evaluation, it’s important that a QA team decides on the type of count (there are three counts for this estimation technique), its scope, and boundaries before performing the estimate.

Benefits

There are no limitations as to the way project requirements are expressed, and this means that using the function point technique, it’s possible to assess the size of any project.

Function point estimation results are easily comprehensible not only for a QA team but also for clients. 

Limitations

It takes a lot of time to estimate a testing project using the function point technique since the estimate needs to take into account a lot of variables.

Additionally, a function point estimate is difficult to perform because an assessment team needs to analyze a lot of input data and produce a lot of documentation.

#6 Percentage distribution

When applying this technique, the software development life cycle (SDLC) is divided into stages (project management, requirements, design, coding, testing, documentation, installation and training) and each is assessed in percentage terms of the time required for the project.

The whole scope of testing tasks is also distributed among 100 percent of the total testing time. The QA team decides how much of their testing time it’s reasonable to spend on different testing stages (component testing, independent testing, system testing, etc.) and expresses this time as a percentage.

Benefits

This technique is easy to apply. It doesn’t require any specific tools or have any specific requirements ‒ you only need a team that has enough expertise to estimate a testing project.

Limitations

To get the most precise results, a percentage distribution assessment should be carried out by highly experienced experts. It may be difficult to find all the experts who might be helpful in estimating a project in one company.

Since estimation is done without specific requirements and input data, the results might not be very accurate.

#7 Experience-based

The idea of this approach is to estimate a project by relying on the experience of a QA team. A team might already have tested a similar project, in which case they can take it as a basis for estimation. The experience-based approach includes two sub-approaches ‒ analogous and expert judgment. 

Analogous estimation is when a QA team takes the estimate of a previous similar product and creates a new estimate based on the existing estimate. In case there are new requirements in the new project, they’re assessed based on the experience of an expert group inside the company. 

Benefits

Using this approach, a mature quality assurance team can provide a precise estimate. When a team has performed dozens of similar projects, they can express their estimates with confidence. 

It’s also fast to conduct expert-based estimation, since an assessment team has estimates and documents from previous projects at hand that can be used as a basis for the estimate.

Limitations

The experience-based approach is difficult to apply to unique projects, as it takes more time and effort to estimate new features a QA team may encounter for the first time.

Moreover, this technique isn’t suitable for young teams who are at the beginning of their careers since they might lack experience.

Software testing techniques the RubyGarage QA team uses to estimate your project

There are even more ways to estimate a software testing project, and it’s even possible to combine some of the ones described above to provide a more precise estimate. That’s exactly what we do at RubyGarage ‒ our QA team uses WBS and experience-based methods to estimate new projects.

If we get a project that’s similar to one our team has done before, we use ready-made estimates based on the work breakdown structure. 

If we’re dealing with a unique product, our senior experts gather to estimate it using the experience-based technique. To make our estimates more realistic, we usually include two options ‒ minimal and maximal. The minimal value is an optimistic prediction, while the maximal is a pessimistic prediction that includes some risks. 

By combining two estimating methods, we can provide our clients with the most accurate results possible.