Experitest Blog

How to identify you’re testing on a real device, and why is it even important?

Superb UX comes with flawless testing

In today’s world, end users have become accustomed to a superb user experience when using their mobile applications. They expect everything to run smoothly, quickly and usefully, on every device they have and under almost every network condition.

 

This has made mobile and web application testing a critical phase in the app-release lifecycle. Functional problems that were not detected while testing, may result in loss of customers, reputation and revenue to the organization. Therefore, companies spend a significant amount of time and effort on testing applications on all types of devices, operating systems and browsers, assuring nothing gets broken.

 

The testing challenge of the digital era

In the digital era we live in, the speed with which new mobile devices and operating systems are released is hard to follow. Testers need to have access to numerous devices and operating systems, and the different combinations they might have. So instead of acquiring and maintaining such a large array of devices, more and more organizations are transforming to testing on devices hosted in the cloud – either real, emulated or simulated. The problem is, that when connecting to a cloud-based device, it is hard to determine whether the allocated device is a real one, an emulator or a simulator.

The importance of testing on a real device

Although emulators and simulators might seem like a good enough option for testing, there is actually great importance given to testing applications on real devices. The problems with emulator/simulator testing are a bit different between iOS and Android-based devices:

 

Android devices:

  • Testing apps on Android emulators is tricky since every device manufacturer performs some changes to the operating system – what is known as “Android fragmentation”. However, all Android emulators run the original Android OS, and therefore cannot emulate anything changed by the device manufacturer, which can, of course, affect the application.
  • Another issue is that actual physical components on the device make it behave differently than the emulator. The emulator’s software cannot accurately imitate the behavior of real hardware such as GPS, camera, CPU, USB connection, etc. so every test that interacts with hardware will not exactly reflect the real-life behavior.
  • Real device vitals (CPU usage, temperature, battery consumption, and memory) cannot be measured and monitored using an emulator and might be affected in real life. Measuring these parameters is crucial for testers, as end users will not accept applications that affect their device’s overall performance.
  • Emulators are jailbroken by nature. This means that any application security features (such as jailbreak detection, fingerprint authentication etc.) will not work on an emulator.
  • Another surprising fact is that despite what people tend to think – emulators are actually slower in their interaction speed and in the initialization of a new session. The reason is that they’re using virtualized hardware, whereas today real device hardware is very strong.

iOS devices (or Apple devices):

  • iOS does not have any emulators, only simulators, which simulate software only and not hardware. The iOS simulators don’t run the actual iOS operating system, but rather a simulation of it that mimics the responses expected from the OS. This means that the application is not even tested on the real operating system. Additionally, the actual application that will be installed on the real device cannot be installed on the simulator. It has to go through a different compilation in order to run on the simulator, and the result of this is that the application’s behavior
    on the simulator and on the real device can eventually be significantly different. This makes simulator-based tests untrustworthy.
  • Similar to Android, device vitals are extremely important to test, and cannot be measured and monitored using a simulator.
  • iOS simulators, similar to Android emulators, are actually slower to interact and initialize compared to real devices.

A real, legitimate device, not jailbroken or rooted

So at this point, it seems quite clear that performing tests on real devices is a must. Another issue to consider is testing on real licensed devices, and not jailbroken or rooted ones. The issue with breaking/rooting the device is that such an activity changes the operating system. Some processes will work well on a jailbroken/rooted device, but not on a real one. Another issue is security – devices that were jailbroken or rooted are not secure and enable protocol cracking and reverse engineering. Therefore, organizations that want to stay on the safe side should not enable their app’s installation on jailbroken or rooted devices, let alone test their applications them.

How do you identify a simulator, emulator or a jailbroken/rooted device?

 

iOS devices:

 

Apple, who like to be in full control of their products, have applications that cannot be deleted from the device. If you find a device that has no “Settings” or “Clock” applications – the device is either not real, or jailbroken.

 

real ios device 1 real ios device 2
Photo application found on a phone Clock application not found on a phone
indicating this is a simulator
or a jailbroken device

 

Trying to use the phone’s camera and checking whether you can take an actual image is another way of knowing it’s a real device with a real physical camera.

 

Another hint that the device might be jailbroken is the iOS version. Apple continuously releases iOS versions and updates to overcome security issues that may be exploited to jailbreak the device. If the device is on an early version of iOS (e.g. iOS version 10.3 while 10.4.1 has already been released), and the software update is blocked, there’s a high probability the device is jailbroken.

 

Android devices:

Android is a bit trickier, as Google allows performing changes to the operating system. In order to assure the device is real you must go into:

 

Settings-> System

 

Within that path, you must verify there’s a utility called: “Developer options”. If you cannot find this utility, you’re running on an emulator.

 

real ios device 1 real ios device 2
Real Android device with
Settings->System->
Developer options utility
Emulator – with no
“Developer options”
under Settings->System

 

Another option for identifying an emulator is opening the camera application, and checking whether it’s possible to take an actual image. This way you’ll know whether the device has a real physical camera.

 

Identifying a rooted Android device is a little more difficult, but there are a couple of tools available for installation that can perform that check for you, see links at the bottom of this post.

 

Another option for checking if it’s a real device, for both Android and iOS, is verifying the device has a serial number, which assures it is real.

real ios device 1 real ios device 2
Android serial number iOS serial number

 

Always check before testing

 

To achieve the best and most reliable results in testing, be sure that you test your application on real devices. When testing on a cloud platform, use the techniques provided above to verify the device allocated to you is real, so you can send your results and recommendations with peace of mind and full confidence. Check out the links below for additional information and tools to tell a simulator/emulator-jailbroken/rooted-real trusted device apart.

 

Identify an Android emulator:

https://gist.github.com/espinchi/168abf054425893d86d1

Identify an iOS jailbroken device:

https://github.com/thii/DTTJailbreakDetection

https://developers.tune.com/sdk/detecting-ios-jailbroken-devices/

https://medium.com/@pinmadhon/how-to-check-your-app-is-installed-on-a-jailbroken-device-67fa0170cf56

https://www.trustwave.com/Resources/SpiderLabs-Blog/Jailbreak-Detection-Methods/

Identify an Android rooted device:

https://github.com/scottyab/rootbeer

https://promon.co/security-news/root-detection-android/

https://medium.com/@scottyab/detecting-root-on-android-97803474f694

Test your application on hundreds of real mobile devices.
Start your free trial now.

Follow our Digital App Testing page on LinkedIn

Lital Grossman – Product Marketing DirectorTom Ben Simhon – CEO SeeTest.io

Comments are closed.