Test Estimation

My manager once came up to me to ask if I was willing to take up a mammoth project and deliver it in a month's time.

It was a simple e-commerce wallet according to him that requires to be built from scratch and deployed as a beta for government regulators to review.

An e-commerce wallet as simple as it sounds has so many integration pieces that it takes months to develop and I was not sure if we would be able to deliver on time. Nevertheless, I agreed to review the scope and come back to him if it would even be possible to deliver within the required timeline.

The 200+ pages requirement document took about a week to read and I was intimidated by the enormity of the requirement. A wallet simply put is a savings bank account equivalent. It has a number of business rules, conditions and government regulations to be followed, to be operational.

The next step was to estimate the number of test engineers required to complete it within the deadline.

Having estimated similar projects before using a linear timeline, the challenge here was to test it as and when a module is available while the fully integrated workflows may come to us a lot later.

The test estimation had to be module-wise and not feature-wise. For simplicity, let us state a module as a part of a feature.

---

Why is Test Estimation important?

As controversial as it may sound, Testing is a subjective task. While you can measure the outcome objectively, coverage is a factor that will continue to be subjective. Testers consume a product, they assess, analyze, and break-down a product from several different points of view to finally give a verdict.

Let me illustrate with an example. Let us say you have invited your friends for dinner and you will have to cook for them. What are the factors that will you keep in mind before you start cooking.?

You will take into account the number of friends who may join for dinner, then what they prefer to eat and drink. You will buy the ingredients and start cooking a few hours early so that you can complete a full course meal?

If you assess the situation the dinner timing, the number of people and the number of ingredients required, time to cook each item can be measured objectively, however, the end result, in this case, a full course meal, how it tastes, how it is presented is subjective to the individual who finally eats it.

Coming back to Test Estimation, here are the parameters I took into account before coming up with the estimates

1. High-level modules: Breakup of very high-level modules. For example in the case of wallet: Create a Wallet, Connect bank account, Credit/Debit, Refund, and Withdrawal

2. Low-level modules: Breakdown each high-level module to a very low level. For example in the case of Create Wallet, break it down to New Customer Sign-up, Existing customer migration to the wallet, Migrate pending refunds to the wallet, etc...

3. Module Risk score: Based on the business criticality of the assign a risk score. A simple question to ask is: How irate your customer will be if this particular feature did not work and assign a score accordingly

4. The experience and skillset score of a test engineer: This is a heuristical score. Based on the level of experience and the knowledge a test engineer might posses. Testing a wallet requires knowledge of accounting. A high score for 5/5 for those test engineers who can understand and test accounting software while the most experienced who can do performance testing may be given 4/5 since wallet may be required to huge transaction volumes.

5. Story points to complete (1 story point = 8 hours): A heuristical assessment of how long it would take to complete testing a module