How much testing is enough testing for mobile applications, especially Android, is a question which is often difficult to answer. A good mobile app testing strategy is as important as having a good test plan. As a lead, if we have an effective mobile app testing strategy,  then we can get maximum efficiency and keep the cost low.

Lets see some pointers below on effective mobile app testing strategies.

What is a Mobile App Testing Strategy?

In answer to the above question, planning and execution plays an important role in any testing and development. In the case of mobile app testing, strategy plays a key role as it lists out the things to do as part of mobile app testing and to achieve good quality with maximum coverage in the available time. As a part of the testing strategy, we need to select correct device models (which are mostly used by the customers in real time), customer use cases, and customer requirements. Apart from that we need to do performance, usability, security, and compatibility testing. 

We will go through each factor one by one and see how can help in testing.

1. Device Selection

Selection of devices is highly dependent on your target market.You should select those test devices that your target customers are most likely to use and you want to distribute your app in.There are different criteria to find suitable devices for your testing. We might have got the question whether to use real devices or emulators for testing. Below is the comparison that will help you to select the best option for your application.

Real device vs. device emulator

Real devices

Pros:

• Provides you more accurate results
• Testing can be done on the same device which customers are using and on the same OS which manufactures modified (For e.g. One UI for Samsung devices, MIUI for Xiaomi devices, Oxygen for Oneplus devices etc.)
•  Testing can be performed on real time interruptions like getting calls or getting SMS etc.

Cons:

•  No early access to verify products on new OS. If it’s available also, it’s limited to 1-2 devices.
• Few devices are too costly so we have to spend more money.
• It is not easy to downgrade or upgrade the OS on the device.

  Mobile device emulators

Pros:

• We can simulate both hardware and software requirements.
• Generally emulators are free and open sources so they is easily available 
• It will help you get unexpected results.
• Emulators can be easily connected to an IDE for the initial stage of development.
• It is easy to upgrade/downgrade or change the OS.

Cons:

• Generally emulators are slow in speed.
• We can’t test applications on factors like battery overheating/battery drainage or conflicts with pre-installed system apps.
• Setting up a good emulator might take time and is sometimes expensive.

2.  Network connectivity

Strategizing your testing based on network connectivity is important for the applications which deal with sharing data, online connectivity or end-to-end tracking over the internet.

So during testing we should cover all the types of networks with different signal strengths so that even with low internet connectivity the application works smoothly.

3. Performance

After selecting device and network connectivity, we should assess the performance of the application and for that we need to do performance testing.As we know, devices have limited storage capacity and RAM to run many tasks together, below are the testing we need to perform as a part of performance testing

1. Load testing
2. Stress testing
3. Endurance testing
4. Spike testing

Server/API Performance

Performance is equivalent to the response time from the Server. In this, tests are run to validate how efficient the app is in handling data transfers from the server 

 and at what rate data is transferring.The faster the rate of data transfer, the more efficient the application. The number of API calls generated by applications should be less and in case of ‘server down’ where and how the data is saved should be covered in this testing.

Mobile application testing tools help QA to troubleshoot any error that comes up.There are different sets of mobile app tools available for different OSes, like for Android, we can use Monkey Runner, JMeter, etc. and for iOS, we can use Automater.

4. Security

Including Security testing in your strategy becomes a game changer for your application because security of users data is very important and many attackers can easily attack the applications. Generally, penetration testing is done  for mobile apps.

5. Automation Testing

Automation testing will help you to speed up the testing process. All the stable modules of your application will be tested in minimal time so that QA is more focused on the stability and new requirements of your application. To understand more about automation testing and manual testing, here’s a simple comparison.

Manual vs. Automated Testing

Automated Testing:

Automated testing should be used to test those use cases which we need to test again and again,use cases which we need to run for validating basic functionalities. It helps to test strict requirements and those features are stable and unchanged. Automated testing also helps to test repetitive testing like clicking on some module multiple times or to perform monkey testing for finding crash and ANR related issues in mobile applications.

Manual testing:

Manual testing is helpful to test new features or to reproduce bugs. For testing complicated use cases also manual testing should be done.

For improve mobile testing, we should perform manual and automated testing in ¼ ratio to test comprehensively and save time.

6. Usability

The look and feel of your application also should be good. If it is difficult to use the application then customer’s might not use the application. So adding usability testing in your testing helps you to analyse the application in terms of usability. Here, we will cover  how user-friendly the mobile application is in terms of the navigation, how easy it is to us, how flexible the app controls are, etc.

7. Compatibility

After covering all the major points in your testing, we should cover compatibility of the applications in different operating systems, devices with different screen sizes/resolutions, internal hardware (memory size, processor speed, and button/input differences), and different languages. It is a very critical task for QA to cover this part.

Below are the few general questions we should cover while adding compatibility testing as a part of your strategy.

1. What happens if the disk space is completely occupied?
2. Whether my application is compatible with different languages or not?
3. Is my application smoothly running in different timezones?
4. Can my application  be installed on an external SD card?
5. Is my application able to work smoothly if the OS is upgraded to the latest version?
6. · Can I update the app when there are multiple updates available?
7. · What happens if I don’t update?

Conclusion

All mobile applications need customised testing approach based on their use cases and target customers.Selecting the correct approach for testing helps to ensure high quality end products.So if you are planning to strategize your testing then first understand the product and what the customer or client expects from it, and then cover all the aspects as needed.