Mobile app testing is a crucial factor and various factors must be considered while you process with it. Now what holds utmost significance is whether mobile app testing should be done on real mobile devices, emulators or both. The answer is quite simple as it depends on what you are keen to test.
For instance, emulators work well for user interface testing and initial quality assurance, whereas real devices are vital for performance testing while device cloud testing is useful in increasing the number of devices and operating systems.
We have come up with the factors you must consider in order to decide upon real devices or emulator-based quality assurance (QA) testing.
Real V/S Emulator
Vice president of product at cloud-based testing platform provider Sauce Labs say emulators are just the best to start with as they are less expensive as compared to mobile devices. “They’re faster, and the catch the most common bugs,” he averred.
“Emulators allow you to debug flows as you’re testing the app,” says Ahmed Hassan, senior QA and test automation engineer for collaboration platform provider Huddle. It allows delving into code, file structures and databases in real time, thus you can finely inspect whatever feature being tested. Meanwhile, mobile technical evangelist Eran Kinsbruner at Perfecto Mobile, which is a mobile app testing platform provider, asserts that testing mobile apps on emulators is not safe as the users are running apps on real devices, not emulators.
Coming to real devices, they are certainly good for testing performance and Parobek says, “How much of the CPU are you going to use for your app? How much memory? How responsive is it?”
However, device performance should not be your only concern. “Real devices are good for testing your application under less than optimal network conditions,” he says. “How does your app respond if the network is dropping in and out, for instance?”
Physical inputs tend to be challenging. “With all these sensors such as GPS, proximity sensors, life sensors, force touch gesture, make sure that your application responds properly,” Kinsbruner says. “Many of these gestures cannot be simulated. If they can be emulated, it’s probably going to be only on one flavor or one emulator, not on all the LG, Samsung, and Sony emulators, and so forth.”
Meanwhile, emulators many a times may fail to test your app. “There are a bunch of third-party libraries, for example, that require a specific type of hardware, and that can’t be run on an emulator,” says Andrej Kostresevic, CEO at the mobile app development company Nomads.
What to choose?
By now it must be clear that a choice between emulators and real devices for testing must not be an either/or proposition. You should aim to find the right balance of emulator and device-based testing supporting your QA standards all through constant integration and deployment.
As per Huddle’s Hassan, the emulator versus real device testing decision is taken on a case-by-case basis. For instance you can test the changes impacting user interface and mobile app architecture using both physical devices and emulators. At the same time, the choice between emulators and real device also depends on your position in the development cycle.
Ultimately, an emulator must be used to test the user interface of your mobile app and to debug flows during the testing process, however you can’t test gestures on an emulator, as for that you require a real device, which is also useful in testing app performance under less than ideal network conditions. Thus, it is as per your needs that you should make a decision on whether to run mobile app tests on an emulator or a real device on a case-by-case basis, taking into account factors exceptional to your organization, such as budget, staffing, project time lines and various other factors.
How do you combine emulator and real device testing in your mobile app testing efforts? Add your comments below.