When it comes to QA, we are often asked, “What should be the testing strategy?” But, the answer to this question is not that easy as this varies with the software’s maturity.
Think back to the early 2000s. Social media was a new and upcoming phenomenon, and websites such as MySpace, Orkut and Bebo did not gain much popularity back then. Early adopters were mostly youngsters who were willing to look past a bug or two, understanding that social media represented a developing and innovative way to engage with others.
But within a few years, the scenario changed; social media became mainstream. The late adopters consisted of older people, with parents and grandparents signing up on social media sites such as Facebook, WhatsApp, and Twitter.
As the number of people using these platforms increased, the statistics of users changed. Companies had to change their ways of testing their products as they realized the importance of testers to suit their new user base.
Here’s why product maturity needs to be taken into consideration when developing your own testing strategy.
The 3 stages of product maturity:
- The validation stage: Flexible testing
Users at this stage are new and excited, and they forgive software hiccups as the product rapidly changes.
Developers focus on creating a basic product and building an app that customers actually want. Software changes are frequent, as the product goes through many challenges and opportunities, until the company either finds a promising prototype.
The testing method that we should use:
At this stage, the focus is on ensuring cost-effectiveness and user friendliness. Your organization is working hard to find a product-market fit, and so your QA approach should be limited to testing core user flow and unblocking critical user issues.
For this reason, ad-hoc testing is the most helpful method. With this approach, tests can be conducted randomly, without any formal planning/procedure. This gives you the freedom to upgrade your strategy as the product develops.
Automation is not a primary concern at this stage. Manual testing methods are the best as they allow for more flexibility. Manual unit testing could be crucial to this QA strategy.
- The Predictability Stage: Software Stability
Once you find your product-market fit, your priorities change. The focus shifts from developing an application fit for the market to making sure its core elements are working fine.
At this point, your product has a larger number of users, and these users may include people who are less forgiving of software bugs.
The testing method that we should use:
The QA strategy must include exploratory testing and automation to ensure full coverage. When King, the maker of the global game phenomenon, Candy Crush, reached the predictability stage in one of its games, it implemented a mixture of different tools and strategies.
It increased the degree of test automation, and the company deployed a large number of testers that included both internal and external testing partners that used crowdsourcing. This enabled the company to discover and fix software bugs that occurred during the previous stage.
Now, we must be able to prioritize which test cases must be automated and which must use crowdsourced testing, but the main focus at this stage is using a combination of exploratory and automation testing.
This way, you can discover bugs which the developers may not have considered before.
- The Scaling Stage: Minimizing negative impact and accelerating the growth
At the scaling stage, companies focus on advancing growth through new user adoption and increased engagement with existing users. The main QA concern at this stage is that even a small bug can impact a huge number of people.
In the prior stage of validation, a bug that affects 1 percent of users wasn’t even on the QA team’s radar. In the predictability stage, a bug like this is “low priority.” But at the scaling stage, 1 percent of your users could include more people than your entire user base in the first two stages.
For example, a bug that affects just a small percentage of Google Maps users might lead to over 10 million people getting affected. Further, the more your business is intertwined with any application, the higher the stakes.
The testing method that you should use:
At this stage, software testing is of major importance. A well-planned and extensive QA strategy is critical to ensuring that a bug does not affect even a small percentage of your user base. Due to this, companies normally invest in different types of testing.
For example, King uses this time for performance testing. They consider factors such as battery, network, and CPU consumption to ensure that the application is genuinely meeting and exceeding customer expectations. Again, automation testing can also be extremely useful at this stage.
An effective QA strategy is crucial:
As your product changes, your QA strategy must change with it. There might not be any fixed method of testing for each stage of your product’s evolution, but there are common challenges that all organizations need to overcome. Making use of different testing methods that suit the stage of your product’s maturity will help you continue to scale.
Adopting a domain-specific approach to testing and continuously modifying your strategy as per specific scenarios is the real answer to the question, “What should my testing strategy be?”