There certainly are no shortage of mobile app testing devices out there today. At Sofy, we’re well aware that the development landscape is in a period of transition. In turn, it’s easy to see how shifting definitions of common terms and the introduction of new concepts can lead to confusion and bewilderment among development teams (check out Sofy’s ever-expanding glossary of terms to know here).
In the blog post preceding this one, we discussed the hot topic of manual-code, low-code, and no-code approaches, highlighting the strengths and weaknesses of each. Yet whatever approach you decides to pursue, your team must inevitably choose between the use of emulators and physical devices. Like the choice between manual coding, low-code, and no-code solutions, each path presents potential benefits and drawbacks that must be carefully considered. First, let’s start with a top-down view of the matter at hand:
What is mobile testing in QA?
Mobile testing is the process in which a quality assurance testing team assesses the performance of an app over a range of mobile app testing devices, whether they be emulators/simulators or physical devices. Mobile QA testing is an exceptionally important processor ensuring that your app performs as intended on physical devices used by your target audience.
What exactly are emulators?
In the world of mobile testing, emulators (similar to—but not quite the same as—simulators) are digital representations of physical devices. In other words, emulators/simulators are programs designed to replicate the behavior of mobile phones. Quality assurance testers use emulators to avoid the cost of acquiring, housing, and maintaining physical devices. In turn, when a team identifies a need to test on multiple devices, testers can conceivably set up a testing environment solely employing emulators, all without paying a penny.
Emulator risks
However, emulators have downsides. For one, they’re all too often a representation of an ideal rather than reality. The core issue associated with emulators is the fact that emulators simply can’t offer a perfect one-to-one match for the devices for which they are intended to represent. Ultimately, emulators can only offer an approximation of an actual device.
And that means that the use of emulators comes with risks. Consider a common scenario among mobile device users: Emma depends on her favorite app for her everyday schedule: She utilizes it to wake up, to stay in touch with her friends, and to check her email. One cold morning, she opens her favorite app and, at the same time, she receives a phone call. The app suddenly crashes.
Scenarios like these are quite typically experienced by smartphone users. You’re dependent upon this app, and yet this is a difficult scenario to replicate—and therefore to test for! —in an emulator-based testing environment.
Other challenges
Replicating user environments is hardly the only challenge that comes with pursuing emulator-based testing. Working with locators presents other complexities, leading to significant challenges for testing teams. Identifying controls often makes for a particular problem. In general, testing platforms require ramp up to allow testers to set up their system, develop associated code and processes, and become familiar with the strengths and weaknesses of the results. And emulators must be consistently up to date.
All of these factors equate to a significant expense that manifests in development and testing resources.
What is a physical device in mobile testing?
In the context of mobile testing, a physical device is a non-simulated, real device, such as an actual mobile phone. Testing with physical devices resolves most problems associated with emulator-based testing because it eliminates the guesswork—there’s no need to attempt to imitate a device by way of an emulator. All of the mystery surrounding emulator-based testing vanishes, becoming a non-factor. There’s no need to wonder about the accuracy of a simulation.
And yet physical devices present their own problems. For many testing teams, the biggest issue with setting up a physical device testing platform is cost. Purchasing up-to-date representations of particular physical devices on the market comes with considerable expense, and the tremendous variety of form factors and configurations in use today only complicates matters further.
Adding to this, just as with emulator-based testing environments, setup and maintenance often prove to be complicated: Quality assurance testers can expect a significant investment in building necessary code to interface with the devices.
The result is that building a physical device testing platform is simply too resource intensive—in terms of both time and money—for most teams to consider, leading many teams to go directly to emulators. They then simply attempt to work around the above-mentioned emulator problems, and hope that the results are what they expect.
Which mobile testing devices to choose?
You should identify any and all devices that your user base is likely to use and test on those devices. This means that if you’re developing a mobile app that is only available on iPhones, you should test on all iPhones currently supported by Apple. However, this can quickly become overwhelming: Picture adding Android devices to the mix and the number of devices you will need to test rises dramatically.
What do you recommend?
In the past, compromise was necessary for many teams who pined for well-maintained and up-to-date labs of physical devices. Such an approach was far beyond the budgets of many teams. Today this isn’t the case. Your team can easily access Sofy’s Device Lab through the Sofy platform, bringing the big-budget experience to all, and at a fraction of the cost. We invite you to try out Sofy today and start experiencing the benefits of the No-Code Revolution.
