Whether it’s the digital alarm that wakes us up in the morning, a ride share app we use to summon a vehicle to pick us up, or a phone game to pass slower hours of the day, apps play a fundamental role in most of our lives today. And chances are that, today, each is a product of intensive mobile automated testing.
Each app requires extensive testing throughout the software development lifecycle (and throughout an app’s lifespan). Considering how complex everything is today and factors such as room for error and efficiency, it’s no surprise that automated testing has come to dominate the testing world.
With or without the most extensively researched test plan imaginable, manual testing is simply no longer a reasonable (or reliable) primary method for modern teams. (And if you’re confused by any of the terms used below, have a look at the Sofy Glossary!)
Mobile automated testing challenges
That certainly isn’t to say that automated testing is without its own problems. As an example, let’s take a quick look at five mobile automated testing challenges:
1. Platform
All major app stores present users with their own testing platforms, such as Google’s Android Studio and Apple’s Xcode and sandbox. Appium is the most commonly used platform for cross-OS support. Each of these platforms supports both emulators and physical devices, but the majority of their users utilize emulators.
When it comes to mobile automated testing, figuring out exactly how to use a platform is a significant challenge. Wouldn’t it be nice if one could simply open up an app and create an automation to run it, smooth and hassle-free?
Set up and configuration also takes time and resources. Compounding this matter is working around the eternal matter of compatibility (such as ongoing issues with Apple’s popular M1 chip systems). Unfortunately, it’s all too rarely the case that automated testing apps are truly intuitive and straightforward, or prepared for shifts in compatibility.
2. Locators (Ability to identify controls to create automation)
Dealing with locators presents another significant challenge. It’s tough to determine where to click to, for example, create login information. Before getting started with locators, an engineer must first identify control names and, with dynamic content, sometimes there are no uniquely identifiable control names with which to work at all.
Indeed, whether a button, an icon, or an image, how does one identify a given control? As apps and hardware grow increasingly complex—there’s that important word, complex, again—and diverse, so does the matter of dealing with locators. This adds another challenge to our list.
3. Form factors
In the United States alone, consumers use thousands of different types of devices. This makes testing for all these combinations and permutations of hardware, operating systems, and accompanying form factors nearly impossible.
In turn, supporting numerous devices and their associated forms presents a major challenge. Consider how much shapes and sizes vary, and other significantly impactful factors like variations in device configuration. Hardware comes in all shapes and sizes, and so do their accompanying apps. For truly thorough testing, each possible form needs to be considered during the testing process.
4. Dynamic content
Hand-in-hand with the above-mentioned matter of locators is the matter of dynamic content. As its name implies, dynamic content is always changing, and this presents a consistent challenge for testing environments. With every change, the testing environment also needs to be adjusted, adding to a need for increased time and resources throughout testing stages.
5. Language
Another major issue is the matter of language, specifically coding language. From iOS’s Xcode to Python and Java, there are probably more coding languages out there in the digital wild than ever before. Whatever language you choose, producing code for automated testing is a significant time investment, and the quality of the results you can expect to receive is directly tied to the quality of the code you produce. The room for error here is always a concern.
6. Maintainability
For our final item, let’s consider the eternally important matter of maintainability. On average, many thousands of apps are published every month through the world’s app stores. Each app must be maintained: Developers add updates and new features to each app on a regular basis, often quite frequently. For each update or modification, testers need to modify and recreate testing automation environments very frequently.
In short, quality assurance professionals face a few sub-challenges here:
- Product changes
- Flakiness of tests due to different form factors or dynamic content (as discussed above)
- Underlying framework issues, like operating system changes. As an example, consider the ongoing fallout of those now-famous iOS 14 privacy changes, breaking for example primary aspects of Facebook’s data collection strategy, which also caused all associated automation tests to fail.
Solution
All of these issues really boil down to a primary issue: Time. Bandwidth is really the most valuable resource any of us have, and it’s the primary reason to turn to automated testing for mobile environments in the first place.
We’ve built Sofy to save you time. Sofy not only allows you to test real devices with ease, the platform provides an integrated development environment that allows you to focus on testing, rather than putting hours into setup.
Additionally, Sofy standardizes test data (so you’re using the right data all the time), and identifies types of content, employing machine learning to allow you to easily work with otherwise tricky matters like dynamic content. Sofy also automatically identifies form factors and adapts to them.
