The mobile device ecosystem is growing exponentially. It’s hard to keep up. With billions of devices in circulation and hundreds of new mobile products released annually, ensuring our applications are adequately tested across devices, versions, and operating systems is becoming more challenging. A real mobile device lab helps.
What is real device testing?
Real device testing involves testing mobile applications on a real mobile device, as opposed to an emulator, simulator, or code-based unit testing. Real device testing is increasing in popularity as cloud-based device labs make it easier to virtualize device testing. The goal is to test on as many real devices, versions, and operating systems as feasible to ensure the most coverage.
Why real device testing?
Real device testing has several benefits. While not meant to replace other forms of testing, real device testing can help provide much better test coverage.
While emulators can validate user interfaces and simple actions, it takes the connectivity, hardware, and external factors of a real device to get the most accurate representation of how a mobile application will perform. Many use cases can only be property tested on a real device. Consider the following:
- Network: The real world consists of connectivity, different bandwidths, 4G, 5G, and WiFi, competing signals, and background downloads. A real device interacts with these network changes in a much more organic way than an emulator or simulator can. The tester can find that a certain process or behavior is considerably slower under specific network conditions and they can work to remedy it.
- CPU and Hardware: Real devices have real components and serve as a better indicator of real-world environments. As different processes run simultaneously on your mobile app, the CPU will flex to meet its needs. The real device is the only effective way to make sure your applications aren’t blowing up a device’s CPU.
- Battery: Really, the worst app is one that drains a battery dry. We know how important conserving energy can be in our busy world. Customers have a huge range of products to choose from and they will not choose yours if their battery is being drained. The real device can monitor battery usage much more effectively. (Read here for more on this topic.)
For more on the many benefits of real device testing over approximations like emulators, see this article.
The physical device lab
So how do we manage a real device testing lab? One way is to physically acquire the devices — at least the most popular ones — and have QA perform testing. This physical device lab has several benefits, most notable among them sensor testing, like gyro (what happens when I physically turn the phone at different angles), and GPS (can the app accurately track me and perform location-based actions? ). This would be especially useful in AR- or VR-based applications that rely heavily on location. In these cases, we would need to take the physical device with us.
The cloud device lab
Today, there’s a different way to build your device lab—a better way—and that is virtualizing the device lab in the cloud. Note that this is different than emulation—this is not replicating or imitating the device. This is using an actual, real device that is stored and accessed from the cloud, just as you would interact with a remote desktop.
A real device lab lets testers access hundreds of different devices with their unique combinations of screen size, OS, version, RAM, and more. With access to these devices, testers can perform manual tests or run their scripted test using the framework of their choice.
What are the benefits of a cloud-based real mobile device lab?
So, what do you choose: Real device testing with the physical device on-site or real device cloud? We’ve mentioned some of the benefits of testing on the physical device (physical devices are more suitable for location-based testing with GPS and other sensor use cases). Here are some reasons to consider taking advantage of a real device cloud.
Cost
How many mobile devices are released each year? Imagine the cost of procuring even the most popular devices, not to mention the cost of labor spent updating, managing, storing, and maintaining these devices. After a short period, these devices will be rendered obsolete and will need to be replaced by the next and best.
Beyond covering the cost of the physical device, you’d also be constrained to the actual location of the device, so a QA team that resides in multiple locations, or perhaps a remote setting, would have to be sent the devices to test, or would have to travel to them. In today’s flexible work environment, this setup would add considerably more time, energy, and ultimately cost to keep a physical device lab.
A cloud-based real mobile device lab, where access to devices is very real, allows you to rent or virtualize physical devices hosted in the cloud. You can access them whenever and wherever. For the most part, the host worries about the expense of maintaining the device and you get the benefit of using them for testing. The only real cost added would be the cost of the cloud service. Sofy provides access to hundreds of iOS and Android devices so you can test your apps at scale.
Security
Keeping expensive devices with potentially private information around is never a good idea. With a private real device cloud, you receive private access to a pool of secured devices that can only be accessed by you. What’s more, you can tailor device configuration to meet your needs, control access, and use IP restrictions.
Scale
This goes without saying but it bears highlighting: Hosting real devices yourself isn’t feasible at scale unless you’ve got a dedicated group who procures, configures, patches, updates, troubleshoots, and everything in between. On the other hand, a real device cloud can scale to your needs. Android only? No problem. Plan to grow your offerings in the future or need to quickly add storage and capabilities? Great! You can leverage the device cloud to do just that.
Time
With the device cloud, you can run your tests concurrently on multiple devices at the same time and receive real-time feedback. Plus, with Sofy, you can write all of these tests completely codeless (which is, after all, the future). You also don’t have to worry about time-sucking activities like installing updates, reconfiguration, and managing storage. These time-savers can lead to faster time-to-market and better outcomes.
Collaboration
Your QA team can access the same private environment and collaborate on testing in real-time. With virtual access to real devices, you and your team can replicate bugs quicker and share testing results to make adjustments and pick up where you left off.
Moving forward
There’s little question that real device testing is a must-have for a solid mobile testing strategy. Despite the advances and application of both emulators and simulators, much remains untested without real device testing. A real device cloud, like the device lab from Sofy, offers a scalable testing environment that doesn’t require the hands-on setup and maintenance that hosting your device would. Do yourself a favor—save the headache and reap the benefits.
Disclaimer: The views and opinions expressed above are those of the contributor and do not necessarily represent or reflect the official beliefs or positions of Sofy.
