Complex test automation can be the worst. Test automation is a must for any organization that develops and supports an application stack. A well-rounded test automation suite saves time, delivers better applications, speeds up deployments, and frees up valuable resources for more important work.
However, as the code base grows, devices improve, and technology changes, the risk of technical debt and complexity compound. Does the test automation you’ve grown to rely on serve you? Or is your test automation simply too complex?
Like everything else, there’s always a point of diminishing returns. Overkill. As complexity increases, technical debt is sure to follow. Eventually, overly complex test automation could lead to the very problems it intended to solve. Testing should be where bugs are caught, not where bugs are made!
So what is overly complex automation? How do you know if your automation is too complex? More importantly, what are steps you can take to avoid overly complex automation? Let’s take a look.
What exactly is overly complex test automation?
If you feel like you’re spending more time updating test classes than you are running tests, then I’m sorry, your automation is too complex.
When your test automation takes a few hours to set up, it’s probably overly complex. Good test automation should run well whenever it’s run, and with little to no need to intervene with the code base. When a single, complex test tries to do too many things, or test too many assertions, the greater the chances of failure. If your developers have to make constant updates to the scripts to account for small changes in the application then the code is too complex.
Overly complex automation may try to do too many things at once or make assertions that are too granular and don’t serve the actual need. Overly complex automation is validating things that don’t need to be validated and that regularly change.
Symptoms to look for
Consider the following symptoms of overly complex automation:
- Too much time spent setting up test automation before it’s run: Do you have to give the dev team a warning before running test cases? Are set-up tasks preventing your team from adopting a CI/CD tool?
- Inconsistent testing results, despite little to no changes being made in the code base: Hold on…ok…try again! Overly complex automation can cause unreliable results and test flakiness. (Want to learn more about how to avoid flaky tests? Check out our post on how to avoid flaky tests.)
- Test automation takes a long time to run: Does the entire suite take hours to run? Do some tests take way longer than others? These are signs that your automation may be too complex.
- Tests that cover too many scenarios: Does a single test cover dozens of scenarios? Do you find yourself constantly waiting on one particular test class to pass? Or do you have to dig in to discover what exactly failed? This is a telltale sign of over-complexity.
- Different tests for different devices: Do you have to tinker with tests depending on the device they’ll run on? Or maintain large test libraries for each device? With code changes, regression testing can be a nightmare. Some frameworks support certain devices.
- They’re hard to support: You have to rely on the same person to update, improve, or debug your test automation – and they’re very busy. What if they left? Complex automation makes it difficult to maintain and few people have the skill sets to do so.
How do I fix overly complex automation?
The first step to fixing overly complex automation is to take it high level. Is your testing aligned around a strategy? Are you employing the test pyramid or another testing framework? Does your testing strategy align with your development goals? You may find that some of your complex test automation is no longer needed. And that’s an easy win.
Once you locate the complex automation, document its behavior. Begin to decouple the automation and break down the test into smaller pieces. Can they run successfully without being coupled together? Are all parts of this test needed? Do I have any dependencies in here that are causing issues?
Choosing the right automation tools
There are a host of testing automation tools, from free open source frameworks like Selenium, Appium, and Espresso to no-code automation suites. As the organization grows, so do testing needs. It takes resources to manage a test automation suite, especially if you’re going the open source route. Overly complex automation could be a sign of the wrong tool.
If an organization doesn’t have the resources to support the tool, it can expect technical debt, flaky tests, and unnecessary or redundant complexity. Open-source tools are free but rely on a deep understanding of code and testing best practices.
Tools like Sofy help manage technical debt by providing a no-code way to easily implement test automation. With Sofy, you can digitally grab a real device, record a test, and automate it. The platform writes automation that is scalable, reliable, and efficient. The platform also evaluates code changes and suggests updates to test automation. Sofy also helps you understand the debugging failures, erasing the need to scour complex test scripts for answers.
With Sofy, you can simplify and scale test automation, without creating an overly complex collection of scripts.
Conclusion
Like any technology in your stack, your test automation suite can add technical debt, cause deployment delays, and affect the health of your applications.
Automation is supposed to make things easy, not complicated.The more challenging it becomes to test your application, the less you can truly say that it’s automated. If not managed carefully, over complexity can disrupt your development lifecycle.
Is your test automation too complex? It may be time to evaluate your testing strategy, your code, your tools, or all three.
Disclaimer: The views and opinions expressed above are those of the contributor and do not necessarily represent or reflect the official beliefs or positions of Sofy.
