Want a quick way to destroy your company’s reputation? It’s quite simple—just release your into the wild with a bunch of bugs and issues. Of course, it’s never going to be perfect, no matter what the expectations, but when your app doesn’t work at launch, well, don’t expect a lot of forgiveness from your user base. With that in mind, let’s talk about ad hoc testing.
Shifting left, QA testing, and the ad hoc approach
Testing is clearly important and, before diving specifically into ad hoc testing, it bears highlighting exactly why testing in general will always be crucial for the success of your mobile app. Building testing into the earliest days of your software implantation framework—commonly known as shifting left—can save you from having to face the wrath of your customer base, who are more than capable of spotting the bugs and defects that you should have caught before your software went out the door.
Today, game releases are probably the most obvious and vocal example of this, with companies nearly destroying their reputations by releasing games that clearly weren’t in a completed state (think for example Polish developer CD Projekt Red’s disastrous launch of Cyberpunk 2077 in 2020).
Intentional testing and software failures
Test-driven development can ensure that there isn’t anything that will break a game or app at launch, and testing methods—like ad hoc testing—are one way you can ensure that you don’t end up baking bad code into your software’s core. This approach works the same with any other type of software, with early bugs and defects becoming nearly impossible to untangle when they’ve been a part of the code since development began.
A large percentage of software failures ultimately originate from treating testing like an afterthought. All too often, disastrous launches and sour updates directly result from dev teams throwing together test plans at the last minute. When this happens, it can result in a type of tunnel vision where missing obvious mistakes becomes much more possible because issues aren’t what testers are looking for as they go through test documentation and procedures.
While there are a variety of testing methods available to modern QA teams, ad hoc testing aims to specifically allows you to see things from the end user’s perspective, catching things that are glaringly obvious to an outsider. In fact, sometimes thinking about testing from the perspective of an outsider is exactly what you need to get your software into an ideal state for release.
Ad hoc testing defined: A quick overview
Alright, so what exactly is ad hoc testing? First, let’s start with refreshing our memory on what ad hoc means:
Ad hoc. Adj. “Created or done for a particular purpose; that answers a specific need or demand, rather than in accordance with a general policy, rule, etc.” (“ad hoc, adv. and adj.”. OED Online. September 2022. Oxford University Press.)
Most testing requires a build up before testers can get started. A testing team must learn enough about what is happening to create a test plan and follow procedures so that changes during development are tested within the scope of the work. It’s all too easy to think that you know exactly what a change in the code is going to do and how it affects the software, but there are virtually always components impacted that you don’t consider.
That’s just the nature of testing.
Ad hoc testing and code
All of that code is connected in some way, so even if you think that something is too isolated to affect anything else, it shouldn’t come as a surprise that something is broken after a change. It’s possible that a change was merged in the tree that shouldn’t have been, or a typo in another section was mistakenly added. And then it gets committed.
Ad hoc testing takes off the blinders of organized testing and allows you to find the problems that you didn’t realize were possible with the latest round of changes. Instead of documenting what you need to test, ad hoc testing is more like how an end user is going to feel coming into the project. Users have certain software expectations – the selling points of that software – and they expect it to work from the time they boot it up. It’s random and unplanned, giving you a lot of leeway to do whatever you want without focusing on any one element.
Typically, ad hoc testing is performed after formal testing is complete so that the obvious problems that may not be seen during focused testing are more likely to be detected before release. However, ad hoc testing can be performed any time there is limited time to test and you don’t want to entirely skip testing at a particular phase. Like other testing types of testing, once a defect is detected, it’s wise to test until you’re confident that you’ve resolved the issue.
Ad hoc testing types
Since there really aren’t any specific steps to ad hoc testing – that would defeat the purpose of this particular testing method – you can choose a type of testing that fits a specific need. Consider the following three examples:
Monkey testing
This type of testing features the distinction of a.) having the most fun name and b.) being the most chaotic. Monkey testing features no specific module or section, with the tester just working the software to try and break it. Think of this as trying to come out from the end user’s perspective without knowing how one should do something. No one can break software like an end user, and this type focuses on that approach to see the things you almost certainly won’t find through planned testing based on how they should use the software. Instead, you are looking for how end users will think things should be done.
Buddy testing
In this type, a buddy in development works with another buddy in testing in tandem once unit testing is done. The pair will work on the same module to see if they can find any defects from their respective perspectives.
Pair testing
Not to be confused with buddy testing, pair testing consists of two testers assigned to their respective modules. The two usually possess different skill sets. Both stay on the same machines to test for defects. This type can actually be more like a team where each tester has a different job, with one member of the pair usually actually performing the testing, while the other makes notes, detailing when defects were found and what triggered them.
In practice, the method you use will generally be dictated by time constraints and available personnel.
If you’re working with little time and have at least three team members available, you could complete one of the two types that use two people and have a third person conduct monkey testing. And a tip: In my experience, monkey testing may be something that you can complete with a technical writer. Usually, technical writers aren’t heavily involved in the majority of the development and testing, giving them a much closer perspective to an end user. When the writer finally gets started documenting the software, this individual can be your monkey tester, too.
Considerations and further tips on ad hoc testing
The beauty of ad hoc testing is that you don’t have to document it beforehand. (However, that doesn’t mean that you shouldn’t expect associated documentation.) You’re free to skip some early documentation, but you don’t have to. Here are some considerations and tips to make your ad hoc testing more effective:
Basic software comprehension
At a minimum, your testers should know what the software will do and how it is meant to work. They should have working knowledge of the system under testing so that they know what sources are likely to trigger problems.
Make basic plans
When two testers work together, they need a rough idea of what they plan to do. At most, they can create an outline so that they can make sure to cover those elements, but there shouldn’t be anything extensive to that plan. It definitely should not follow any existing documentation. This is not system testing or focusing on just a single set of changes, but in looking for things that these types of testing won’t find.
Focus under time constraints
You can select specific modules for testing if time is very limited. This should always include your core elements, anything what is business critical, so that you can build your customers’ confidence about the quality of the product.
Track everything
Every issue you find should have a corresponding defect number and built-out tracking so that you can ensure that it is fixed. Each should be treated like a defect found through any other testing method so that you don’t release a product with a known issue – or if you do for some reason have to release it with an issue, you can note that in the release.
Ad hoc testing is largely informal, but it serves the same purpose as every other method of testing. Since it is unscripted, it’s just a little faster and looser.
A little bit of chaos to keep finding those glaring errors
Even if you have time for all of the other methods of testing, I recommend building ad hoc testing into your project because, in my experience, it helps combat tunnel vision. Test plans and designs are made based on what the developers plan to do, and these can work great when not tested on a larger scale. Ad hoc testing ensures that you take a much more holistic approach to testing to catch the things that are far more likely to be missed by other types of testing.
If you’re looking for a platform that allows for a variety of testing types, including ad hoc testing you can get a demo with Sofy today. You’ll find that allows for ad hoc testing and a lot more (like testing on real devices)—and all in a single platform, helping you keep your team as organized and as efficient as possible.
Disclaimer: The views and opinions expressed above are those of the contributor and do not necessarily represent or reflect the official beliefs or positions of Sofy.
