Over the last twenty years that I’ve been involved in product development, I have seen tens of thousands of bugs created for different product lines ranging from websites, backends, APIs, and SQLs, to compilers and languages. The goal is always the same – to release a great product. But, in my experience, teams who are not focused on producing quality bugs with the right data waste a lot of time following up on the priority, steps, and coordinating between the developers and QA engineers. Data provided while creating bugs with the correct set of information is extremely critical in the whole bug lifecycle and saves a lot of time and effort for triage teams and engineers.
Lifecycle of a bug
There are five steps in a bug’s lifecycle which directly impact the efficiency of the engineering team:
- Creation: QA engineers, product managers, or developers create a bug
- Triage: The triage team looks at the bug in daily scrum
- Assignment: The triage team decides if the bug needs to be fixed and assigns it to the developers who work with QA engineers on reproducing the bug
- Fixing and Checking: The developers fix the bug
- Regression testing: The QA engineer determines, with help from the developer, how to perform regression testing
What Makes a Good Bug Report?
- Focus on Customer/User Scenario: All bug creation must begin with the goal- the customer. I have seen bug reports that talk about everything other than the impact on the end-user of the software. A good product bug or report is important from the user perspective – it should provide the personas, core scenario, and impact (a scenario impacting 100% of the users is more impactful than an edge scenario affecting 2% of the users).
- Detailed Steps on Reproducing: Good QA engineers always spend time and effort upfront to identify critical steps with all the supporting artifacts like screenshots, logs, trace, detailed steps, and customer impact that will help provide specifics. In some complex products, QA engineers also look at the code to determine what other problems may be hidden and identify them.
The New Mobile Test Reports
At Sofy, our goal is to make QA engineering extremely productive so engineers don’t waste time on repetitive tasks like explaining test results, reporting detailed bugs, and reproducing issues for developers.
To better understand the process, we studied thousands of test runs and reports generated at Sofy. Then, we used this data to create new reports that are focused on three key pillars of effective test reports and bugs:
- Single Click Bug Report: Seamlessly create bugs while performing testing with a click of a button and automatically create in your bug systems (JIRA, etc.).
- Aggregate The Right Data at the Right Time: Sofy uses a patented platform to capture all the details about your test run automatically. There is no need to aggregate data from different tools to get the right information when the bug happens. If during testing a crash happens, Sofy automatically creates a crash dump, previous steps to reproduce, environment details, and logs so that the developer and triage teams have everyone in one place
- Single Click Environment Setup for Developers: Developer and QA engineers can create specific devices or environments within minutes using Sofy’s lab environment.
- Smart Regression: Once a bug is fixed, Sofy automatically suggests what to test and how.
By significantly improving the quality of bug reporting, Sofy reduces the friction between QA engineers, triage teams, and developers on whether a bug can be reproduced, where the logs are, and environment setup. Everything is taken care of with high-quality bugs. Setup a use case for your app and start automation in minutes.
