Once a company creates its web and mobile app testing strategy, they often lock it away as though it is the secret formula to create Coca-Cola. It should not be though. A web and mobile app testing strategy is a guide to your organization’s testing. It is also a living document that requires updating from time to time. Use your results as a way to tell which aspects of the strategy are and are not working.
We have discussed at length the unique challenges of mobile testing. Developing a solid strategy takes the right leadership, project, and testing managers. There are so many factors involved that are under your control and even more that are not. Take into account your app’s complexity which in theory should change over time. Also, consider your industry and how it changes over time.
So let’s say you have been working using the same strategy for a few months even years. When is the right time to review it? How are these reviews done? Let’s take a look.
What Every Web and Mobile App Testing Strategy Needs
The decisions that go into a mobile testing strategy include functionality, compatibility, and data. When creating a vision for your mobile testing process you need to develop a strategy that contains the following steps.
- Cross-Platform Testing – If your app interacts with others, and let’s face it in 2020 it probably does, then you need to understand all the business objectives, project requirements, languages, different platforms tests will run on, and user needs when developing your web and mobile app testing strategy.
- Feature Functions – You must test the functions of an app, this goes without saying. You also need to test the features built into devices that affect your app.
- Application type – Which type of mobile app will you develop? Native, mobile-web, or hybrid. When building a strategy you need to understand each type of app and which one yours is going to be.
- Front-end Testing – Testing anything visible on the client-side (GUI). This type of testing also needs testers to understand business goals.
- Back-end testing – Also known as database testing. This step tests the server-side of your web and mobile apps.
- Multiple network compatibility – It might seem impossible to test your apps on devices of all brands and platforms but it is essential. It might seem impossible to test your apps on devices of all brands and platforms but it is essential. Try it as we do it.
- Storage – This will help your app in that you will discover how your app will affect users on a device vital basis. Try out monitoring tools for more
- Performance – Your strategy needs to include testing network conditions and geolocation so you can use your app the way your users will. We have a solution for performance testing as well.
- Application Flow – Testing the visual flow will ensure that your users can navigate the visual elements of your mobile app.
Hopefully, these are the items that you have included in your testing strategy. It would serve your organization well for a long time. With changes to your app(s) and developments in your industry, it seems like you need to revisit the strategy. So let’s see how to do that.
How to reduce the bugs in your mobile testing strategy.
When Derek Zoolander and Hansel have a breakthrough when they realize that the “files are IN the computer”. Never mind that to retrieve said files they smashed the computer to bits. That said there are probably bugs IN your mobile testing strategy. Not only code related, but rather in the processes and practices, you employ during the SDLC.
So let’s take a look at some best practices that will help you discover what parts of your strategy might not be delivering value and how we can fix that.
QA Testing and Business Practices
This might shock you but Business Users and QA Testers are both cut from a very different cloth. QA Testers look at a problem with an almost arithmetic eye. They identify the problem, based on a parametric context, and solve it.
On the other hand, product managers and UX architects (business users) have knowledge that stretches beyond simply finding bugs. They have knowledge that stretches across different aspects of a product.
These teams both likely gave their blessing to your testing strategy when it was first implemented. At the point where you are reviewing the strategy, you should also take a look at the relationships between these two teams. How have they been working together? Are they communicating productively? The people involved are as important as the process itself. They are capable of sinking the entire project and by extension your app.
Cross-team Test Case reviews
Organizations create winning mobile testing strategies with plans that eliminate siloed teams. Testers, developers, and everyone in between should review all of the cases involved in developing a new feature. In regards to developing a testing strategy, reviews should happen as early in the process as possible.
As we have seen in many cases these groups, siloed as they are not, become siloed as they go because these are the teams that spend the most time together. For example, some say that organizations should eliminate their QA teams. You might see an inefficiency in your testing that would seem to corroborate that extreme theory.
This is a result of a non-siloed strategy finding their silo again. As the development team sends more code to production there are a huge number of potential failure points that can show up.
As you review your strategy keep in mind that your non-siloed teams might not be communicating well. Make sure that from business to QA everyone reviews the requirements, designs, and test cases early in the process so that all teams understand the flow. If you can keep everyone working on the same strategic page you won’t need to change any of these flows when reviewing your mobile testing strategy. Silos only are a detriment if teams do not communicate or work on the same processes.
Cross-Functional UI Testing
UI testing is simple in theory and one of the more important cross-functional aspects of a testing strategy. Basically per your strategy, once the product team creates and approves the UI designs the developers implement them. The next step is where it gets tricky. Once implemented everyone examines your UI designs from the stakeholders back to the designers to ensure that everything aligns as it should be.
When reviewing the original planning of your UI testing you must view this aspect of your mobile testing strategy from the business perspective rather than the QA perspective. They are used in this process more for their input while the product owners and business leaders create the designs.
One of the aspects to examine is to take note of where small changes have been made to the initial plans and to see if the handover from business leaders is working. A high number of concessions or changes in the final product will let you know where there might be a breakdown in this process.
Making sure that your UI testing processes are clear and well oiled will give you a final product that is created by one team, implemented by another, and show little to no alteration between the two.
Supported Devices and Operating Systems
Chances are, when you first created your testing strategy, even if it was simply a few months ago, the device models OSes, and versions your app supported have changed. Beyond that, we know all about how fragmented devices are these days.
When examining your strategy you will need someone to look at your internal data and compare that to the best available industry-wide data. Armed with that information they will be able to make a judgment call on which devices OSes and versions to continue to support and which to deprecate.
It makes sense from a testing strategy standpoint there are a ton of specific elements that go wrong on the device and OS level. You must make a decision when creating and reviewing your strategy to determine what to support. This aspect of your testing strategy will probably need to be reviewed more often than the rest.
This might sound harsh but it is preferable to leave a few users behind rather than to try to support every OS and version available. Once you have the final list in place an automated testing tool will help manage your tests and the devices they are executed on.
Testing on Real Devices
This is an extension of the previous point. We encourage you to have real device testing as part of your mobile testing strategy. It makes real devices and browsers easily available to more DevOps teams, wherever they are. It also makes introducing new devices and operating systems fast and efficient and helps streamline collaboration and sharing.
However, you might find as you review your mobile testing strategy, that some devices are no longer relevant on a global market scale. With the proper internal data, you will be able to see which devices are tested on less than others. Potentially, removing them from your efforts which could increase your automated testing efficiency.
The opposite is also true, you must keep your eye out for new devices coming and decide which ones you should add to your testing efforts.
Performance testing matters
In 2014 the British research company GMI reported that 89% of people they polled said an app’s interaction with their battery life is the most important factor when using an app.
Monitoring and testing battery life is an essential piece of your Performance Testing strategy but it is only a piece. We recently wrote a checklist for Performance Testing. There are many factors when testing performance, such as load and stress testing. There are also external matters to consider like testing under different network conditions. This is especially important if you have a wide user base located in far-flung markets worldwide.
As you review your testing strategies you also need to make sure that your performance testing is effective. Making sure that the devices you are testing on are working properly, and be sure to test against the system and native apps to show a baseline for your performance monitoring.
Performance testing should be something that everyone in your organization agrees on from stakeholders to QA this includes UX performance testing. Issues can arise however if you don’t set realistic performance benchmarks. As you review your testing strategy it may not be your team failing to meet their performance goals. It might simply be that your performance testing expectations are too high and need to be adjusted.
Regression Testing Again?
Every mobile testing strategy has space in it for Regression Testing. If yours doesn’t then what are you waiting for? Go now and develop one.
Regression testing exists to make sure that when you take one step forward you are not taking two steps back.
In examining this aspect of your mobile testing strategy you need to be sure that legacy functionality remains just that functional. Sometimes older app functions will need to be deprecated in favor of new ones and all of that should be viewed and with no emotional attachment. The aim is to make the best functioning UX not load up on incompatible features.
Automation Testing Too
There is no simpler or quicker way to test your features than automated testing. The best part is that it does not require your QA testers to manually check use cases.
No doubt this is part of your mobile testing strategy already. The effort in setting up automated testing requires your teams to build test cases that will catch all of the issues related to a new app build. As you move forward these tests can be rerun.
As you review your test strategy it pays to review your automated test scripts. To stay on top of all scenarios, you might miss tests that are run on deprecated features. You must review these tests to make sure that not only do they run properly but that they also relate to the app, and its functionality. This is not a situation where you would want to say “We have a test for that.” Write and manage your automated tests in a way where if something is less efficient or unrelated is still included it should be removed.
How often should you review your mobile test strategy?
This certainly depends on your team. As long as your teams are working towards a common goal and vision the results will reflect that. It is when there is a breakdown that you might need to take a step back and see if the breakdown is caused by inefficiency in your strategy.
In that regard, you might not review your strategy all that often, but the opportunity needs to be present.
If you work with a third-party SI, for example, you might need to review more often. Some QA teams will review every time they run tests as they find the context changes with each release. Others only review when they have the time, which is likely not often, or perhaps only when they are in a period of knowledge transfer either to new team members or managers.
We have reviewed all of the most important aspects of your web and mobile app testing strategy. What was important was not only looking at each part of the strategy. Also provide some ways to review these as not every strategy is set in stone.
Successful companies use different tactics in regards to keeping their app bug free. Included in that is reviewing the very processes and means that they employ.
Once you develop your web and mobile app testing strategy the result will be reliable and bug-free app releases. However, without examining your core strategy from time to time your processes might get stale and become less effective. You must review these strategies so that your SDLC functions as expected as you prepare your apps and updates for production.