There’s more to testing than automation, here’s why.

4 min read

Since joining SOFY.AI, I am getting a better appreciation of role of automation in software testing. While automation may be the nirvana for any software QA team, 100% automation is impossible.  This also the pragmatic view we took in developing SOFY.AI.

To get a holistic perspective, let me start with different dimensions of Testing. You can find in many articles on this topic in medium website.

1. Functional & Non-Functional Testing

Testing can be broadly classified into functional or non-functional testing. Functional testing is app-specific and validates app functionality. So if you have a shopping app, functional tests validate your core functions like product catalog search, shopping cart, shipping and tax calculation, checkout experience, etc. Non-functional testing is app-agnostic and is about behavior of the app independent of the app’s functionality. It tests for app’s performance, UX design, usability, crashes, globalization, localization so that these issues are detected before the app is released to production. Much of non-functional testing is performed manually using exploratory testing given the non-deterministic nature of these tests.


  • Unit Testing
  • Scenario / Integration Testing
  • User Acceptance Testing
  • Regression Testing
  • Smoke Testing


  • Performance
  • Security
  • Reliability
  • Usability
  • Scalability
  • Globalization / Localization
  • User Experience
  • Failover
  • Penetration

2. Manual & Automation

One of the most important decisions a testing team has to make is to determine which of the above types of testing is needed for their app and how much to automate. Manual testing, as the name suggests, is performed by a QA tester. It offers the benefit of minimal upfront investment and provides an easy path to get started. However, it comes with cost, lack of transparency, need to provide devices each of testers, etc. On the other hand, automated testing solves these problems providing scale and repeatability. Automation requires use of testing frameworks (Appium, selenium, cucumber, etc.), expertise of QA engineers with deep technical skills, and use of multiple tools. However, one must constantly modify these automation scripts to adapt to UI changes, device and OS compatibility.

Types of Automation:

There are essentially 2 ways to automate.

  1. Automation using scripts: This is a do-it-yourself way of automation using frameworks like Appium, Selenium, Cucumber, etc. This is the dominant way automation is done today. Most of the commercial automation tool for mobile testing (Sauce Labs, Browserstack) are designed for this scenario. This approach is much better than custom scripting solutions as these frameworks try to abstract some of common functions needed for automation. When considering automation tool for mobile testing, one has deal with variations in device form factors, differences in behavior across OS version within a OS family (Android or IOS) as these are not always handled by the frameworks.
  2. Nocode or low code (NCLC) automation is a way to creating automation without writing code. It takes users intent and auto-generates automation scripts based on the frameworks. Some of the automation tool for mobile testing in this category also use AI and ML to address the above challenges that addressed by the frameworks.


Manual testing is still significant part of any project testing. A very high percentage of testing is performed using manual approach in early stage products. As a product matures, percentage of tests that are automated increases 50% or 60% of total testing.

Today, automation is primarily about writing code/scripts using frameworks such as selenium, Appium, etc. to perform functional testing. Most of the tools in the market target this scenario. No code automation is still far and few.

Exploratory testing is heuristic in nature and is performed predominantly using manual approach. There are very few tools that support automation of non-functional UI testing.


Leave a reply

three × four =

You might also like

4 min read

What’s new in Android 12 and what should the developers expect

  Google I/O, the yearly developer conference of the organization, came back after the COVID-19 pandemic which was canceled last year. In this year’s I/O 2021 keynote, Google has launched the latest android version called ‘Android 12’.   Google upgrades the Android operating system with a full-scope hardware and software overhaul, a cornucopia of technology […]

4 min read

Why You Should Adopt Automation of UI Testing?

The primary goal of any business is to increase its sales and increase the growth of the business. UX/UI Design plays an essential role in achieving this goal.

4 min read

There’s more to testing than automation, here’s why.

Since joining SOFY.AI almost a year ago, I have been immersed in understanding software testing and role of…

1 2 3