In many respects, performance is more important than function. Performance is making sure that your app works seamlessly across multiple devices, versions, service providers, and real world network.
Performance issues are when an app doesn’t perform to the expectations of the customer.
One of the big challenges to smooth mobile performance is the load. You want to test interruptions, app response time even freezes as more data is requested from the server due to more users, or more complex mobile applications. The challenges in mobile load testing are many:
- By its nature, mobile generates more load. There are more users on mobile devices than on desktop computers and their web browsers.
- As the global internet audience shifts from the web to mobile, expect more users coming from mobile devices. As the cost for a Smartphone continues to go down, this trend will accelerate.
- Mobile applications are replacing web applications. Mobile applications, which go beyond the browser and tap into the native functions of the hardware’s (mobile device’s) operating system, are more complex and require the server to work harder.
All of this is increasing the load, requiring app developers to make sure that the servers have the capacity to continue to provide smooth and fast service to their mobile customers.
What Makes Mobile Apps Thick
Mobile apps are similar to desktop applications. However, they are “thick” clients in that the complexity of today’s mobile apps are rapidly becoming more like desktop apps, rather than web apps.
A web app will work on the web browser, but not use the native functions of the computer itself.
Native or hybrid mobile apps will use the abilities of the mobile web browser, but also the native abilities of the mobile operating system, which manipulate the functions of the mobile device itself. These mobile apps interface with the browser, operating system, and the local device, to make them more complex.
The fact that a mobile app offers more is a key reason why so many users are shifting from desktop to mobile. It is accelerating the pace at which new users are on mobile, and increasing the time users spend on mobile. All of this creates load issues that require mobile testing.
Mobile Load Creates a Bigger Effect on Your Servers
One of the main causes of load on your server is the number of concurrent sessions, and since mobile apps are more complex, they take more room from the servers. The same number of mobile users compared to desktop users can result in 20% more load on your servers.
Network conditions also affect load. As a mobile app runs through a bad mobile connection, it will take longer to process, increasing a number of users on the server as more requests are held up at various points in their digital journey.
Traditional Load Testing is Synthetic
In traditional load testing, the application under test is being removed and a synthetic network traffic is generated.
But first, we need to see what does load look like?
Load is the process of a mobile application asking for information from the application server. The process begins when your application opens up a TCP connection to the server. Then it sends a request to the server. Then the server sends a response to the app, and the process completes when the application then closes the connection.
What are typical things mobile applications request?
- Information – to either receive it from the server or to send it that it will be recorded on the server.
- Session ID
- Transaction ID
But typical requests can have dozens or even hundreds of fields. One of the big challenges of the load engineer is to understand the nature of all those fields.
When you create synthetic users, and synthetic requests there are risks. Quite often a request will be made to the server, but the fields are not correct. The information transfer will be recorded between the server and the app, but because the request was faulty, the server sends back a 0 value. In essence, you can have a test where the server records performing 1,000 operations, but not notifying you that 500 of them required no work – it just sent back a 0 value because the request wasn’t sent right or the form fields weren’t properly configured.
This can lead you to a false positive – that the application can handle increased load. The truth is it can only handle the load if all the added users are requesting zero information. In the real world, where users are all executing functions by entering in the right information, the real load will overrun the server causing crashes, freezes, the slow functioning of the app, and interruptions.
Performance teams need to adapt methodologies to validate that the generated load is valid. Some approaches can be performing database and UI validation to verify that the transaction generated the expected effect, looking at the server log files to find the “transaction complete” log messages, monitoring the server vitals to verify the load effect on the server’s CPU, memory, the number of threads, and so on.
Monitoring Your Load using Real Device Transaction
As you generate the synthetic load you would like to generate a few interactions with the servers based on real devices running your real application.
This provides the validation you need from doing the synthetic load testing. You can get response time references that you can cross reference with the synthetic load, and you get real transaction response times. Using real devices for your synthetic load gives you more accurate data, and tells you whether or not your application server can handle heavier loads.
The Pleasant Surprise of Heavy Loads
Enjoy a FREE one month trial of SeeTest Automation along with a live demonstration on developing test automation scripts!
Check it out at Automation Made Simple!
QA Makes it Work!
Load testing answers the good questions. Heavy loads come from better-developed apps and a higher level of traffic. Unlike a function breaking down, load testing addresses issues we look forward to encountering.
Heavy loads can be unpredictable. There are several scenarios when they come out of nowhere. The key to load testing is that when your finance and marketing team are breaking out the champagne, you want to be prepared so you can join the party. To do that you need to be ready for:
- Certain times of the day that have higher amounts of traffic.
- Certain events like holidays, weekends, special company events, or sales.
- An advertising campaign aimed at generating more downloads of the app.
- A successful piece of content can “go viral” resulting in waves of traffic downloading and using your app. This doesn’t have to be big. It can be a localized audience reading a piece of content or seeing someone in the business appear on the news, and a small amount of your total market suddenly downloads or uses the app. That small amount can double your traffic overnight, putting stress on the server.
- At the same time, old content can “suddenly” be discovered by a large group of people, raising its ranking in the search engine results, generating more traffic. Search engines can suddenly tweak their search algorithms, resulting in a surge in traffic.
- Lower network coverage means functions perform slowly, and each user stays on the server longer than anticipated. This can cause more users to be on the server at the same time more often. Bad environmental conditions can spring up in densely populated cities causing more people to be “stuck” in your server longer.
The SeeTest Solution for Load Testing
Experitest has developed an add-on to HPLoadrunner where you can use HP Performance Center like you always have, but extend it to measure the performance of your mobile application under large amounts of traffic. You can generate users with Vugen, and SeeTest will create scripts based the functions you wish to execute on your mobile device.
You can connect real devices and see how they perform under the load you generate with VuGen. SeeTest will measure response times and device vitals like CPU, memory, battery, and GPU as each function executes. With the SeeTest Add-on, you are extending the capabilities of LoadRunner.
SeeTest gives you seamless integration into HP Performance Center as you can create a VuGen load script directly in LoadRunner, or in using SeeTestAutomation to create the script, then take the code and simply copy and paste it into LoadRunner. The code is perfectly compatible with Performance center.
This enables you to get screen shots of every step, and receive the monitoring results directly from LoadRunner.
Mobile applications create a wide range of new opportunities for every business. They also come with a lot of challenges unique to mobility. Unprecedented levels of global traffic and network virtualization are two issues that must be overcome to deliver a mobile application that will perform smoothly and quickly over any device, operating system, version, and local network. Load testing with the SeeTest Mobile Addon is a surefire way to crush these challenges and deliver the best user experience in your market.
Get Updates on Software Quality Assurance at
our Experitest LinkedIn Page