New feature: SofySense – Manual test case & test results AI generator to help you speed up your testing process. Try it today.  

Sofy launches product feature suite to deliver effortless test maintenance and ensure continuous precision – learn more.

sofy logo

Webinar Ep. 1

No-Code & Mobile App Testing: Current Insights

Video Transcript

Cassandra Schwartz: Welcome to the Sofy Webinar No-Code & Mobile App Testing: Current Insights. Today, we’re going to be taking a look at contemporary developments in mobile app testing, this landscape, including the challenges that folks are facing, solutions, and what we expect to see in the near and distant future. I’m really excited to be here with Syed, our CEO and founder of Sofy.  

Syed Hamid: Thank you for having me, Cassandra. Great to be here. 

 Cassandra Schwartz: Yes. A few housekeeping items first. We are going to be taking questions. There is a chat box as well so feel free to submit those. We’ll save time at the end, though I’m going to try and keep my eye on it. If there’s something that’s super-relevant that I can slide into our conversation with Syed, I will add those in so we can get them addressed. If we don’t answer all of the questions during the chat today, we will also follow up. I want to make sure that we get everybody’s questions answered. And so if there’s some that are super-specific to your use case or things like that, we will make sure to take it offline and give you some good insights there.  

Oh. You know what? And I forgot to introduce myself. I am Cassandra Schwartz, the VP of marketing here at Sofy as well, you should probably know that. [Laughter] So, okay, we are all super-familiar with apps. They are on your mobile phone. We now live off an app. I remember more than a decade ago, it was like, “Oh, there’s an app for that,” and now, we don’t say that anymore because there is an app for everything. Our mobile phones are attached to us. We give them to smaller children I feel like every year. They are the first thing we see when we get up and the last thing. And so we’re using these apps throughout our day and we’re really relying on it to live our lives. And frankly, it’s pretty incredible. I am old enough to remember cellphones without apps on them and living our lives around that. It’s pretty cool what’s happened over the last few decades. 

Syed Hamid: That’s very correct, Cassandra. It’s amazing how we have transformed everything in life and not only from the Gen Z but also every aspect of software in our life from healthcare to e-commerce to delivering food. There isn’t any task in a day that people don’t use apps. And to a certain extent is that not only we use apps all the time, but our expectations are very, very significant. We want the latest and greatest installed on our application without even talking to us. I mean, that was incomprehensible just a decade or two back, right? When companies like Microsoft that were shipping products every two years. Today, if you are not shipping every week with the latest feature in my hand, I get disappointed. So, you’re totally right. It’s so amazing to see how everybody’s life has evolved and transformed. 

Cassandra Schwartz: Yeah. And that specific, the expectation that it’s always being updated, I don’t even think about the updates, right? I remember when you had to update manually and you had to go in and push buttons to make it happen. And now, I have an expectation that my apps will always work and that I’m always going to get the best from it. I imagine for engineers, that creates a lot of stress. 

Syed Hamid: I think [Laughter] I’m under a bit of stress. I can remember that we had back in my days at Microsoft, we used to release apps every two years, right? And people were happy, “Oh, I release pretty good. I’m releasing every year.” Today, the engineer needs to write and release products literally on a daily basis. And the expectation, I was reading an interesting article that, hey, good teams [Phonetic 00:04:14], almost 125,000 apps are released every other week and 70% of those are at least shipped every two weeks and on average, people release their apps in seven days.  

Imagine you’re writing a code and you need to release to not only one country but worldwide and expect the same quality every time that you release. It does create a lot of stress on the engineering teams. And efficacy, and that’s why engineering efficacy is becoming of paramount importance for any company. Not just a tech company but any company in the world. And every company’s effectively becoming a software company. 

Cassandra Schwartz: That’s so true. When I first got into tech, it was like, “Oh, you’re in tech.” And now all of the people I know, whether they work in retail or healthcare, they’re all in tech. We’re all in tech no matter really what you’re building because apps, software, is part of really making everything function these days. And creating that quality experience that also then comes down to testing, right?  

Syed Hamid: Yeah. It’s a thing that I’m passionate about and I see that pain point literally every time. And I was also reading quite a few reports and one of the marketing reports talks about almost 70% of the engineering managers or engineering leaders says that testing is one of the biggest impediments for their faster releases. Which effectively means engineering efficacy. And it’s happening probably [Phonetic 00:05:55] because we haven’t evolved the testing per se. Like we just talked about, hey, people were releasing in years. Now they are releasing their apps in days and hours, right? We are building products faster; we have cloud computing that enables us to release products faster.  

Yet at testing level, it still takes the same manual approach of, hey, give me a device. I need to test an iPhone. Okay, I need to buy a $1500 device to test on it manually. And if an issue comes then I need to actually report that on skilled people like software engineers. And that’s why it’s becoming more complex. So, your metrics and if you have to release in 10 countries, it’s significant because every county is using different apps or devices. You need to release that. Every country is a different network configuration and setup, so you have to validate for that. And then you also have to validate for all the UX and UI and different languages, and that takes time.  

And most of the people still continue to do either by manually testing it or hiring software engineers to write code and validate it. Both of them are not only very time-consuming but most, most important, it’s actually you can’t find people to do it. So, you have a real double whammy situation which is causing additional stress on developers because they can’t effectively test on all their target personas on a fast and an efficient way and be able to successfully release the application. And that is a direct implication on your bottom line.  

I read another report that anybody who’s installing a new application, 70% of the people who download an application, before downloading they actually look at the customer feedback. And if feedback is bad, they would not install the application. So, now there’s a lot of things needs to be come together in order to make it a successful app release. 

Cassandra Schwartz: Yeah. It’s incredibly complex and you just said that. Do you think this complexity was predictable? Like has this happened before? Have we seen it in other areas that we should take lessons from?  

Syed Hamid: I mean, it has happened effectively every time, right? I remember, I mean, I’m old enough to remember the ’90s development in [Inaudible 00:08:21], right, when everything was becoming like… Actually, Java was coming in, right? And people were talking about abstraction layer. So, it’s gradually if you see that the productivity of engineers is improving year over year because now, they can… From the DOS world to the Windows to IDE to Cloud, all this enables us to release products faster, better, and efficiently across the world, right?  

So, I mean, obviously we can’t predict the future but historically we see that the demand for engineers or software engineer continue to increase. As an anecdotal point, one of the conferences in the US represented that, hey, there is a demand for like 60, 70 million developers by 2030 and we are not producing enough. So, having the ability for engineers to produce more better and faster with high quality is actually the highest [Phonetic 00:09:21] part of it. And I think that’s where people see the progression happening, right, and that’s where you see the APIs and the advancement in software development which basically helps us release products faster. 

Cassandra Schwartz: So, looking at that, we’re not going to have enough people. What’s the solution here? What’s the shift that’s going to solve that? 

Syed Hamid: I think if you kind of look at kind of a progression, right, if you just go 10 years back, big data was huge, everybody was talking about big data, and nobody talks about big data today. And the reason for the big data was, hey, now we are generating so much data across applications, like companies like DataDogs and New Relics of the world. If you want to know what your customer experience is, you have all the data that you can see. Slice it and dice it every way that you want it to be.  

And I see it’s that now you know what’s happening in your app application. Previously you didn’t have the data. Now you can understand your testing data, you can understand your production data, you can understand your customer data. So, that data is actually of the essence [Phonetic 00:10:33] to help us take the productivity to the next level.  

What do I mean by that? It’s that today if you look just at the performance testing as an example, you have to write or use a specialized tool to understand, “Hey, what’s happening on my IOS performance versus Android performance?” Right? Now with advancement in the…at the [Inaudible 00:10:57] level from New Relic and other APM solutions, you can actually get that information almost real-time. So, I see the same progression that must happen, right, from the manual testing to coded testing to literally a no-code platform where we shield all the complexity from people so that they can field test faster, better, and efficiently. So, we have to really simplify the problem in a consumable fashion. 

 Cassandra Schwartz: So, I mean, we know no-code. If you’re not aware of Sofy, that is the foundation of what we have built. Can you talk to me more about this shift into no-code and the future you see there? 

Syed Hamid: Yeah. So, I think if you see it like we see it it’s that no-code is that what problem we are solving first. The problem we are solving is that if you look at a QA resource which is testing an app, what do they have to do? What’s the day in a life for a tester looks like. They get up in the morning obviously, and then they get their app deployed through CICD pipeline on any number of devices. Then they install it through TestFlight, or any other tools out there and then manually test it. Actually go through. We have a lot of customers who are just doing this in all the places and if you have a team in like international location, then sometimes you have to ship physical devices for them to manually test it. So, you test that.  

Then you find issues, and when you find issues, you have to report it to your developers and the information going back and forth, “Oh, you didn’t repro the problem properly,” or “I need more information about how to reproduce that problem. And the cycle goes every day. Every time a new build comes in and do it. So, we believe that the new no-code is literally transforming an existing, repetitive, mundane task, that almost 70% of the individual QA time is spent on that, transform that and abstract it into no-code.  

What does it mean? It means that you should be able to get your environment up and running under 10 seconds versus 30 minutes. What it means is that in order to create automation that takes you two hours to create and run it on different platforms, Android versus IOS, it should take you 15 to 20 minutes to do that. And third, in order to validate as I mentioned on hundreds of devices, that requires you to write custom code. You can do it with a click of button.  

So, our goal is not to create a no-good solution. Our goal is to transform a QA who’s spending 70% of manual repetitive tasks to transform into highly productive where your release, if it’s taking days and weeks, how do we transform your application, do the high-quality release in hours. And that’s where the no-code is an actual step forward, right? Look, we are just abstracting you from writing all the coding in Appium or C# or Java or Xcode. Instead of doing that, anybody can do it. Because that’s one problem of the development lifecycle.  

And the second problem is you just can’t hire people of the highly skilled software engineers and put them to write test. Because that’s not providing value to the end user. You need to create feature and we should build an environment where anyone can create test. Anyone can run on real physical devices in minutes and hours and not in days and weeks. And that, we believe, is what our distinguished thing is. And that’s why I left Microsoft, to actually focus on this problem because this is not a specific problem to a particular company or other. It’s a problem that industry is facing, and we have like over 5 million QA resources worldwide who’s spending almost 70% of their tasks doing just repetitive manual tasks and we want to be the forefront of that revolution. 

Cassandra Schwartz: So, I mean, I have to ask the little elephant in the room is you’re talking about replacing some of those repetitive tasks. As a QA person, I feel like I would be like, “Oh, are you trying to replace me?” What do you stay to that type of question? 

Syed Hamid: This conversation comes up anytime you have that. I remember when I was graduating out of college in the mid to late ’90s is that everybody was talking about, “Hey, you have a computer coming therefore you’re going to be replaced.” “Oh, now you have the cloud coming so that we don’t need IT resources,” right? Now today people talk about, “Hey, OpenAI. Am I going to replace all the writers and things?”  

So, I think that’s what excites me in the technology. You are always thinking about what level of next efficiency. Imagine the same example that I gave. A QA tester who’s spending 70% time on manually setting it up knowing he has 70% time available to think broader. Think about what other use case, how do I use the data to increase the productivity of not only the QA but the engineering team itself and provide value to the companies.  

As I mentioned, the demand, every single company in the world is not a software company. Every single software need should be tested. We are just removing the manual mundane tasks so that humans can actually focus on more value-add scenarios, creating scenarios, running it and help educating how the software itself can become better. So, I truly believe it’s transformation more so than the replacement. 

Cassandra Schwartz: Okay. We heard it here. Our jobs are safe for the QA team. [Laughter] You mentioned OpenAI and I have to ask because it is the topic of all the forums I’m on, all the different media outlets around tech. Everybody is talking about what’s happening with AI. I think for some of us, it felt like it came out of nowhere. So much of AI was like it’s not there, it’s not performing, there’s so many bugs, every time you see it, somebody using a chatbot, it would turn evil. [Laughter] And so now we’re seeing it useful. We’re actually seeing it being leveraged in ways that are serving businesses. Do you have… I know you have opinions. Tell me your opinions on what’s the future when it comes to AI and for us with mobile app testing. 

Syed Hamid: That’s kind of really interesting. When I talk to kind of the potential lead and customers, I always tell, “Okay, look. As entrepreneurs, it’s our job to dream and it’s our job to dream big.” When we started this, we said, “Okay, look. What’s the progression of QA and where it’s heading?” Then I mention ’80s was all about manual testing, ’90s and 2000s was about coding, right? Write code, run it fast, right? Last five, seven years, we’ve been talking about no-code. Not only in just testing environment but generally in the development community, that no-code.  

I see the future as the intelligent testing. And what is intelligent testing? And actually I was in one of my talks, I had predicted by 2025, we will have, we’ll start talking about intelligent testing. And intelligent testing is software’s ability to identify or self-identify what other areas I need to go test. Just like analogy would be that in the world where Tesla is driving cars autonomously, why can’t software be test autonomous, even certain pieces of it, right?  

So, intelligent testing has three core elements. Number one – ability to understand what’s changed. Number two – ability to understand what’s impacted and what’s the change for the use case perspective? Number three – ability to identify the test cases and be able to generate and run. And the fourth one is ability to run, execute, and report on it. Yeah, it’s kind of the [Inaudible 00:19:03] but we have all the building blocks today to do that.  

For example, if I get a completely new tester today, that’s fresh out of college and is starting in the company, and we ask them to, “Hey, go learn about security testing.” Today, you can actually instead of going and reading 20, 40, 50 different articles, you can actually just go on OpenAI and say, “Look. Help. I want to test a mobile app on Android. Tell me how do I test for security.” And you’ll get all contextual data. Right? So, what OpenAI enables is that it takes all the existing data that you have today and contextualize it. Meaning I can now figure it out what’s happening.  

And I believe that it’s the right time for companies like us to start thinking about it and it’s the right time to kind of say, “What’s the next level of productivity?” Because remember the goal is really about increasing the efficiency, removing the 60, 70% of mundane, manual, repetitive tasks with advancement in the technology and now it’s the AI. Five years back, it was the no-code. And it’s the same evolution for me, right? Because we have the data, and we have all the right building blocks to go do that and I see that that’s happening more and more, and you’ll see already happening to a certain extent where people kind of do the UI and UX testing by comparing the majors [Phonetic 00:20:29] and giving you the [Inaudible 00:20:31]. So, some pieces are coming in and I think now it’s the companies like us, we should kind of start thinking of what are the use cases that brings this thing together to bring the efficiency to engineers and the QA teams.  

Cassandra Schwartz: It’s super exciting. It gets me all jazzed up. Okay. So, I just want to do a little quick rehab here because we covered a lot in a really short amount of time here. So, testing is really reaching this point where, I mean, it has reached the point where traditional testing methods just don’t make sense. They’re not sustainable for teams, and we don’t have the personnel, right? We don’t have enough people to do all of this. And as we said from the get-go, apps, like we rely on them for everything. If our apps are not functioning, it can shut down our whole day.  

I certainly have been at a company where it’s like, “Oh, my gosh. An app went down,” or half the internet went down and we’re all just looking at each other like, “I guess we’re taking a vacation day today.” [Laughter] So, it’s super important that those are functioning and serving all of our needs. And that complexity, we don’t tolerate a poor experience anymore. And so no-code addresses a lot of these complexities, makes it faster to get to market, to get that feedback loop. We want to have this agile experience and really, this constant back and forth doesn’t enable that. But to be able to do it quickly, I love how you said with Sofy, you can get it up and running in minutes versus hours.   

I have to admit that the first time I used the platform, I was like, “Oh. I can do this.” I’m a marketer and I was able to set up an automated test and nobody was holding my hand. It wasn’t like the product team was being like, “Come on, Cassandra. This is how you do it.” [Laughter] So, I love that no-code really enables that. And to your point, that it’s not replacing folks, it really is opening the door for the more creative strategic work. And thinking about the use cases, the user experience, how to provide a great product for whoever is needing to use that app.  

And then of course, we just touched on AI and how the future really of like this self-healing software, which sounds slightly scary, I’m not going to lie, a little bit of like, “Ooh, is this a Skynet thing?” But not really. We’re seeing that we can reach those capabilities and so that is incredibly exciting. So, that’s what we’ve come away with here.  

I want to open this up for questions and I was not doing a good job of paying attention so there is actually a couple already here. So, my apologies but we’re going to address them now. Okay. So, as a QA leader, how should I be preparing for a future of that shift? So, like the shift of moving my team from not being focused…or from doing the mundane thing, is how I’m reading this, to most of their work is now being automated. How do you prepare your team to make that transition?  

Syed Hamid: Yeah. I think that each team is different and each team has kind of a different problem set, right? I think it’s very important to identify what is the core issue. Right? Most of the teams that we talked about is that they spend a lot of time, and they have automated things, it’s maintaining, maintaining the [Inaudible 00:24:04]. And then also they lose a lot of information about their team moving, right? So, the attrition is an interesting problem because when the domain expert leaves, you lose all the data as well, right? All the knowledge with them, right?  

 So, I think there are three ways that an engineer leader, in my opinion, should think about or the QA leader should think about. It’s never about today. It’s all about day after tomorrow. Right? What your N+1 goal is. And N+ goal is not the engineering goal. And a lot of engineers, and I’ve made that mistake all the time, it’s actually a business goal. What’s your customer? It should be starting from the customer. And what are the pain points the customers are having and how that pain point is translating, or it can be avoided in your engineering system. Because releasing a high-quality application is just not the QA team’s responsibility. It’s a collective responsibility of the team. And that’s one of the things that when we build the product, we looked about that, “Look. What are the needs of the quality across the discipline?” Right? From looking at the UX, UI, and all those things it brings into the picture. So, I think understanding that is very critical.  

And then two is that you must have a joint goal with your developer counterpart. Because the reason is that sometimes a lot of QA problem can be actually addressed in the product itself, not after the fact. Right? So, having a consistent goal, having a consistent story, and having a consistent what the N+1 goal looks like and to do that.  

And then third thing which I think is a very, very important thing is to really look at, have a hard look at your day in the life of QA. You will be surprised that when we talk to engineering leaders, they have no idea how much manual effort is being done to release an app. And sometimes what happens is that when the business as usual and people are working… Because we have to release every week, right, to start. It’s really difficult, right? Release every week and every other week. So, you’re trying to balance that with the activity that people are performing.  

So, if you have three of these approaches, then you’ll be able to go and say, “Okay, what is the right solution?” And sometimes the no-code, like us, will be a solution and sometimes it may not be. But having the clarity of these things, of these three core pivots [Phonetic 00:26:27] are more important and to think about the future. 

Cassandra Schwartz: I love that you had that down. [Laughter] That’s such a perfect encapsulation of all the things. And I also appreciate how you ended it because that really preps us for this next question which I feel like is such a… I appreciate developers so much, I’ve worked with y’all for years, and you don’t let us slide as marketers on the happy things. And so this person pointed out you’re sharing all these positive things about no-code but what are some of the use cases or what are some of the areas where it’s limited and it’s not going to solve your problem? 

Syed Hamid: Yeah. So, I think anybody who’s telling you this is the stage in the industry that no-code is the solution for all and everything. I think it’s wrong. I would not say that either. Because it’s a progression and you solve problem at the time, right? I mean, there’s still the case where when those operating system or the cloud may not be the right thing. So, having said that, how I see the problem in the no-code, the areas that it cannot solve today, there’s a couple of things, right?  

One is that when you’re having a heterogeneous asynchronous environment. What does that mean? It means is that, let’s take an economical example of ordering, it’s lunchtime, ordering sandwich, right? I could use my app to order sandwich and I have to have a restaurant who’s going to make that sandwich. And then I have a DoorDash or something else that comes and picks that up, right? So, you have a aggregation of three different events happening across the board. And out of the box, the no-code solution is very difficult to encapsulate all those asynchronous systems that are dependent on those things. So, that’s one use case that we are not there yet for the no-code, right? Because multiple app and a heterogenous environment, right?  

The second thing that the no-code is not very good is like it’s streaming and testing like on the streaming. Like you’re testing, as an example, the next [Inaudible 00:28:38] app and on a very specific captions and things like that. So, those things are kind of difficult to validate that, primarily because we are using the technology like machine learning and AI to figure it out, things, and then the responses are in milliseconds, or the system is not advanced enough to kind of generate those kind of a test to execute that. But it’s not that you cannot still do manual and other things in the platform, but you just cannot write the automation on its own that can seamlessly work on it.  

And I think the third category that I would say that no-code is not very efficient today is if you are doing a lot of kind of emojis and very specialized images which is the machine learning modules are still not advanced enough to take out what’s happening on that. For example, in an image you want to find out, “Hey, it’s a shirt that I bought. Is the coloring of that shirt is exactly the same what I intended it to be?” Or like you say, the accessibility type things. So, those are the things. But we are getting there. I think the industry as a whole, as the maturity like, as an example, the OpenAI has a very advanced image recognition algorithms that can be used. So, I think as the maturity of the machine learning modules improves, the no-code will automatically start getting better.  

Cassandra Schwartz: Yeah. And bringing that OpenAI again, and this is just me as a marketer wanting to talk more about Sofy. What’s some of the things that you think we’re going to be seeing here in the near future at Sofy? 

Syed Hamid: Yeah. I think [Inaudible 00:30:21] will be coming in very soon. We’ll be announcing the OpenAI pieces. It’s going to [Inaudible 00:30:28] things. And as I said, right, we are looking at, hard look at OpenAI and saying, “How do we make the QA super, super productive?” So, we have two, three very salient use cases that we think we can really transform and both the generation and execution and reporting using the OpenAI. Because what OpenAI gives you is that it gives you ability to transform your stack data into contextual data. What it means is that, hey, today I can find out what’s the top apps or what’s the top devices used in a different country and automatically go and execute against that. So, I think those contextual data becomes very relevant and we are actively working on it, and we’ll have hopefully soon a dedicated webinar on that too.  

Cassandra Schwartz: Oh, yes. I love a good chance to sit down and chat with you. [Laughter] Well, I’m going to leave it there. For those of you we didn’t get to your question, we will respond to you directly. This webinar will be hosted on our site. So, if you are watching this after it’s been recorded, you can reach out to us. We are happy to answer questions. Syed loves to talk about this, so I love an excuse to sit down and get more answers for you all. And just want to thank those of you that were here today. We really appreciate those questions, really great questions, and we hope to see you on our next one. And thank you, Syed, I appreciate you sitting down with me today. 

Syed Hamid: Thank you very much and have a wonderful day.  

Syed Hamid: Thank you for having me, Cassandra. Great to be here. 

 Cassandra Schwartz: Yes. A few housekeeping items first. We are going to be taking questions. There is a chat box as well so feel free to submit those. We’ll save time at the end, though I’m going to try and keep my eye on it. If there’s something that’s super-relevant that I can slide into our conversation with Syed, I will add those in so we can get them addressed. If we don’t answer all of the questions during the chat today, we will also follow up. I want to make sure that we get everybody’s questions answered. And so if there’s some that are super-specific to your use case or things like that, we will make sure to take it offline and give you some good insights there.  

Oh. You know what? And I forgot to introduce myself. I am Cassandra Schwartz, the VP of marketing here at Sofy as well, you should probably know that. [Laughter] So, okay, we are all super-familiar with apps. They are on your mobile phone. We now live off an app. I remember more than a decade ago, it was like, “Oh, there’s an app for that,” and now, we don’t say that anymore because there is an app for everything. Our mobile phones are attached to us. We give them to smaller children I feel like every year. They are the first thing we see when we get up and the last thing. And so we’re using these apps throughout our day and we’re really relying on it to live our lives. And frankly, it’s pretty incredible. I am old enough to remember cellphones without apps on them and living our lives around that. It’s pretty cool what’s happened over the last few decades. 

Syed Hamid: That’s very correct, Cassandra. It’s amazing how we have transformed everything in life and not only from the Gen Z but also every aspect of software in our life from healthcare to e-commerce to delivering food. There isn’t any task in a day that people don’t use apps. And to a certain extent is that not only we use apps all the time, but our expectations are very, very significant. We want the latest and greatest installed on our application without even talking to us. I mean, that was incomprehensible just a decade or two back, right? When companies like Microsoft that were shipping products every two years. Today, if you are not shipping every week with the latest feature in my hand, I get disappointed. So, you’re totally right. It’s so amazing to see how everybody’s life has evolved and transformed. 

Cassandra Schwartz: Yeah. And that specific, the expectation that it’s always being updated, I don’t even think about the updates, right? I remember when you had to update manually and you had to go in and push buttons to make it happen. And now, I have an expectation that my apps will always work and that I’m always going to get the best from it. I imagine for engineers, that creates a lot of stress. 

Syed Hamid: I think [Laughter] I’m under a bit of stress. I can remember that we had back in my days at Microsoft, we used to release apps every two years, right? And people were happy, “Oh, I release pretty good. I’m releasing every year.” Today, the engineer needs to write and release products literally on a daily basis. And the expectation, I was reading an interesting article that, hey, good teams [Phonetic 00:04:14], almost 125,000 apps are released every other week and 70% of those are at least shipped every two weeks and on average, people release their apps in seven days.  

Imagine you’re writing a code and you need to release to not only one country but worldwide and expect the same quality every time that you release. It does create a lot of stress on the engineering teams. And efficacy, and that’s why engineering efficacy is becoming of paramount importance for any company. Not just a tech company but any company in the world. And every company’s effectively becoming a software company. 

Cassandra Schwartz: That’s so true. When I first got into tech, it was like, “Oh, you’re in tech.” And now all of the people I know, whether they work in retail or healthcare, they’re all in tech. We’re all in tech no matter really what you’re building because apps, software, is part of really making everything function these days. And creating that quality experience that also then comes down to testing, right?  

Syed Hamid: Yeah. It’s a thing that I’m passionate about and I see that pain point literally every time. And I was also reading quite a few reports and one of the marketing reports talks about almost 70% of the engineering managers or engineering leaders says that testing is one of the biggest impediments for their faster releases. Which effectively means engineering efficacy. And it’s happening probably [Phonetic 00:05:55] because we haven’t evolved the testing per se. Like we just talked about, hey, people were releasing in years. Now they are releasing their apps in days and hours, right? We are building products faster; we have cloud computing that enables us to release products faster.  

Yet at testing level, it still takes the same manual approach of, hey, give me a device. I need to test an iPhone. Okay, I need to buy a $1500 device to test on it manually. And if an issue comes then I need to actually report that on skilled people like software engineers. And that’s why it’s becoming more complex. So, your metrics and if you have to release in 10 countries, it’s significant because every county is using different apps or devices. You need to release that. Every country is a different network configuration and setup, so you have to validate for that. And then you also have to validate for all the UX and UI and different languages, and that takes time.  

And most of the people still continue to do either by manually testing it or hiring software engineers to write code and validate it. Both of them are not only very time-consuming but most, most important, it’s actually you can’t find people to do it. So, you have a real double whammy situation which is causing additional stress on developers because they can’t effectively test on all their target personas on a fast and an efficient way and be able to successfully release the application. And that is a direct implication on your bottom line.  

I read another report that anybody who’s installing a new application, 70% of the people who download an application, before downloading they actually look at the customer feedback. And if feedback is bad, they would not install the application. So, now there’s a lot of things needs to be come together in order to make it a successful app release. 

Cassandra Schwartz: Yeah. It’s incredibly complex and you just said that. Do you think this complexity was predictable? Like has this happened before? Have we seen it in other areas that we should take lessons from?  

Syed Hamid: I mean, it has happened effectively every time, right? I remember, I mean, I’m old enough to remember the ’90s development in [Inaudible 00:08:21], right, when everything was becoming like… Actually, Java was coming in, right? And people were talking about abstraction layer. So, it’s gradually if you see that the productivity of engineers is improving year over year because now, they can… From the DOS world to the Windows to IDE to Cloud, all this enables us to release products faster, better, and efficiently across the world, right?  

So, I mean, obviously we can’t predict the future but historically we see that the demand for engineers or software engineer continue to increase. As an anecdotal point, one of the conferences in the US represented that, hey, there is a demand for like 60, 70 million developers by 2030 and we are not producing enough. So, having the ability for engineers to produce more better and faster with high quality is actually the highest [Phonetic 00:09:21] part of it. And I think that’s where people see the progression happening, right, and that’s where you see the APIs and the advancement in software development which basically helps us release products faster. 

Cassandra Schwartz: So, looking at that, we’re not going to have enough people. What’s the solution here? What’s the shift that’s going to solve that? 

Syed Hamid: I think if you kind of look at kind of a progression, right, if you just go 10 years back, big data was huge, everybody was talking about big data, and nobody talks about big data today. And the reason for the big data was, hey, now we are generating so much data across applications, like companies like DataDogs and New Relics of the world. If you want to know what your customer experience is, you have all the data that you can see. Slice it and dice it every way that you want it to be.  

And I see it’s that now you know what’s happening in your app application. Previously you didn’t have the data. Now you can understand your testing data, you can understand your production data, you can understand your customer data. So, that data is actually of the essence [Phonetic 00:10:33] to help us take the productivity to the next level.  

What do I mean by that? It’s that today if you look just at the performance testing as an example, you have to write or use a specialized tool to understand, “Hey, what’s happening on my IOS performance versus Android performance?” Right? Now with advancement in the…at the [Inaudible 00:10:57] level from New Relic and other APM solutions, you can actually get that information almost real-time. So, I see the same progression that must happen, right, from the manual testing to coded testing to literally a no-code platform where we shield all the complexity from people so that they can field test faster, better, and efficiently. So, we have to really simplify the problem in a consumable fashion. 

 Cassandra Schwartz: So, I mean, we know no-code. If you’re not aware of Sofy, that is the foundation of what we have built. Can you talk to me more about this shift into no-code and the future you see there? 

Syed Hamid: Yeah. So, I think if you see it like we see it it’s that no-code is that what problem we are solving first. The problem we are solving is that if you look at a QA resource which is testing an app, what do they have to do? What’s the day in a life for a tester looks like. They get up in the morning obviously, and then they get their app deployed through CICD pipeline on any number of devices. Then they install it through TestFlight, or any other tools out there and then manually test it. Actually go through. We have a lot of customers who are just doing this in all the places and if you have a team in like international location, then sometimes you have to ship physical devices for them to manually test it. So, you test that.  

Then you find issues, and when you find issues, you have to report it to your developers and the information going back and forth, “Oh, you didn’t repro the problem properly,” or “I need more information about how to reproduce that problem. And the cycle goes every day. Every time a new build comes in and do it. So, we believe that the new no-code is literally transforming an existing, repetitive, mundane task, that almost 70% of the individual QA time is spent on that, transform that and abstract it into no-code.  

What does it mean? It means that you should be able to get your environment up and running under 10 seconds versus 30 minutes. What it means is that in order to create automation that takes you two hours to create and run it on different platforms, Android versus IOS, it should take you 15 to 20 minutes to do that. And third, in order to validate as I mentioned on hundreds of devices, that requires you to write custom code. You can do it with a click of button.  

So, our goal is not to create a no-good solution. Our goal is to transform a QA who’s spending 70% of manual repetitive tasks to transform into highly productive where your release, if it’s taking days and weeks, how do we transform your application, do the high-quality release in hours. And that’s where the no-code is an actual step forward, right? Look, we are just abstracting you from writing all the coding in Appium or C# or Java or Xcode. Instead of doing that, anybody can do it. Because that’s one problem of the development lifecycle.  

And the second problem is you just can’t hire people of the highly skilled software engineers and put them to write test. Because that’s not providing value to the end user. You need to create feature and we should build an environment where anyone can create test. Anyone can run on real physical devices in minutes and hours and not in days and weeks. And that, we believe, is what our distinguished thing is. And that’s why I left Microsoft, to actually focus on this problem because this is not a specific problem to a particular company or other. It’s a problem that industry is facing, and we have like over 5 million QA resources worldwide who’s spending almost 70% of their tasks doing just repetitive manual tasks and we want to be the forefront of that revolution. 

Cassandra Schwartz: So, I mean, I have to ask the little elephant in the room is you’re talking about replacing some of those repetitive tasks. As a QA person, I feel like I would be like, “Oh, are you trying to replace me?” What do you stay to that type of question? 

Syed Hamid: This conversation comes up anytime you have that. I remember when I was graduating out of college in the mid to late ’90s is that everybody was talking about, “Hey, you have a computer coming therefore you’re going to be replaced.” “Oh, now you have the cloud coming so that we don’t need IT resources,” right? Now today people talk about, “Hey, OpenAI. Am I going to replace all the writers and things?”  

So, I think that’s what excites me in the technology. You are always thinking about what level of next efficiency. Imagine the same example that I gave. A QA tester who’s spending 70% time on manually setting it up knowing he has 70% time available to think broader. Think about what other use case, how do I use the data to increase the productivity of not only the QA but the engineering team itself and provide value to the companies.  

As I mentioned, the demand, every single company in the world is not a software company. Every single software need should be tested. We are just removing the manual mundane tasks so that humans can actually focus on more value-add scenarios, creating scenarios, running it and help educating how the software itself can become better. So, I truly believe it’s transformation more so than the replacement. 

Cassandra Schwartz: Okay. We heard it here. Our jobs are safe for the QA team. [Laughter] You mentioned OpenAI and I have to ask because it is the topic of all the forums I’m on, all the different media outlets around tech. Everybody is talking about what’s happening with AI. I think for some of us, it felt like it came out of nowhere. So much of AI was like it’s not there, it’s not performing, there’s so many bugs, every time you see it, somebody using a chatbot, it would turn evil. [Laughter] And so now we’re seeing it useful. We’re actually seeing it being leveraged in ways that are serving businesses. Do you have… I know you have opinions. Tell me your opinions on what’s the future when it comes to AI and for us with mobile app testing. 

Syed Hamid: That’s kind of really interesting. When I talk to kind of the potential lead and customers, I always tell, “Okay, look. As entrepreneurs, it’s our job to dream and it’s our job to dream big.” When we started this, we said, “Okay, look. What’s the progression of QA and where it’s heading?” Then I mention ’80s was all about manual testing, ’90s and 2000s was about coding, right? Write code, run it fast, right? Last five, seven years, we’ve been talking about no-code. Not only in just testing environment but generally in the development community, that no-code.  

I see the future as the intelligent testing. And what is intelligent testing? And actually I was in one of my talks, I had predicted by 2025, we will have, we’ll start talking about intelligent testing. And intelligent testing is software’s ability to identify or self-identify what other areas I need to go test. Just like analogy would be that in the world where Tesla is driving cars autonomously, why can’t software be test autonomous, even certain pieces of it, right?  

So, intelligent testing has three core elements. Number one – ability to understand what’s changed. Number two – ability to understand what’s impacted and what’s the change for the use case perspective? Number three – ability to identify the test cases and be able to generate and run. And the fourth one is ability to run, execute, and report on it. Yeah, it’s kind of the [Inaudible 00:19:03] but we have all the building blocks today to do that.  

For example, if I get a completely new tester today, that’s fresh out of college and is starting in the company, and we ask them to, “Hey, go learn about security testing.” Today, you can actually instead of going and reading 20, 40, 50 different articles, you can actually just go on OpenAI and say, “Look. Help. I want to test a mobile app on Android. Tell me how do I test for security.” And you’ll get all contextual data. Right? So, what OpenAI enables is that it takes all the existing data that you have today and contextualize it. Meaning I can now figure it out what’s happening.  

And I believe that it’s the right time for companies like us to start thinking about it and it’s the right time to kind of say, “What’s the next level of productivity?” Because remember the goal is really about increasing the efficiency, removing the 60, 70% of mundane, manual, repetitive tasks with advancement in the technology and now it’s the AI. Five years back, it was the no-code. And it’s the same evolution for me, right? Because we have the data, and we have all the right building blocks to go do that and I see that that’s happening more and more, and you’ll see already happening to a certain extent where people kind of do the UI and UX testing by comparing the majors [Phonetic 00:20:29] and giving you the [Inaudible 00:20:31]. So, some pieces are coming in and I think now it’s the companies like us, we should kind of start thinking of what are the use cases that brings this thing together to bring the efficiency to engineers and the QA teams.  

Cassandra Schwartz: It’s super exciting. It gets me all jazzed up. Okay. So, I just want to do a little quick rehab here because we covered a lot in a really short amount of time here. So, testing is really reaching this point where, I mean, it has reached the point where traditional testing methods just don’t make sense. They’re not sustainable for teams, and we don’t have the personnel, right? We don’t have enough people to do all of this. And as we said from the get-go, apps, like we rely on them for everything. If our apps are not functioning, it can shut down our whole day.  

I certainly have been at a company where it’s like, “Oh, my gosh. An app went down,” or half the internet went down and we’re all just looking at each other like, “I guess we’re taking a vacation day today.” [Laughter] So, it’s super important that those are functioning and serving all of our needs. And that complexity, we don’t tolerate a poor experience anymore. And so no-code addresses a lot of these complexities, makes it faster to get to market, to get that feedback loop. We want to have this agile experience and really, this constant back and forth doesn’t enable that. But to be able to do it quickly, I love how you said with Sofy, you can get it up and running in minutes versus hours.   

I have to admit that the first time I used the platform, I was like, “Oh. I can do this.” I’m a marketer and I was able to set up an automated test and nobody was holding my hand. It wasn’t like the product team was being like, “Come on, Cassandra. This is how you do it.” [Laughter] So, I love that no-code really enables that. And to your point, that it’s not replacing folks, it really is opening the door for the more creative strategic work. And thinking about the use cases, the user experience, how to provide a great product for whoever is needing to use that app.  

And then of course, we just touched on AI and how the future really of like this self-healing software, which sounds slightly scary, I’m not going to lie, a little bit of like, “Ooh, is this a Skynet thing?” But not really. We’re seeing that we can reach those capabilities and so that is incredibly exciting. So, that’s what we’ve come away with here.  

I want to open this up for questions and I was not doing a good job of paying attention so there is actually a couple already here. So, my apologies but we’re going to address them now. Okay. So, as a QA leader, how should I be preparing for a future of that shift? So, like the shift of moving my team from not being focused…or from doing the mundane thing, is how I’m reading this, to most of their work is now being automated. How do you prepare your team to make that transition?  

Syed Hamid: Yeah. I think that each team is different and each team has kind of a different problem set, right? I think it’s very important to identify what is the core issue. Right? Most of the teams that we talked about is that they spend a lot of time, and they have automated things, it’s maintaining, maintaining the [Inaudible 00:24:04]. And then also they lose a lot of information about their team moving, right? So, the attrition is an interesting problem because when the domain expert leaves, you lose all the data as well, right? All the knowledge with them, right?  

 So, I think there are three ways that an engineer leader, in my opinion, should think about or the QA leader should think about. It’s never about today. It’s all about day after tomorrow. Right? What your N+1 goal is. And N+ goal is not the engineering goal. And a lot of engineers, and I’ve made that mistake all the time, it’s actually a business goal. What’s your customer? It should be starting from the customer. And what are the pain points the customers are having and how that pain point is translating, or it can be avoided in your engineering system. Because releasing a high-quality application is just not the QA team’s responsibility. It’s a collective responsibility of the team. And that’s one of the things that when we build the product, we looked about that, “Look. What are the needs of the quality across the discipline?” Right? From looking at the UX, UI, and all those things it brings into the picture. So, I think understanding that is very critical.  

And then two is that you must have a joint goal with your developer counterpart. Because the reason is that sometimes a lot of QA problem can be actually addressed in the product itself, not after the fact. Right? So, having a consistent goal, having a consistent story, and having a consistent what the N+1 goal looks like and to do that.  

And then third thing which I think is a very, very important thing is to really look at, have a hard look at your day in the life of QA. You will be surprised that when we talk to engineering leaders, they have no idea how much manual effort is being done to release an app. And sometimes what happens is that when the business as usual and people are working… Because we have to release every week, right, to start. It’s really difficult, right? Release every week and every other week. So, you’re trying to balance that with the activity that people are performing.  

So, if you have three of these approaches, then you’ll be able to go and say, “Okay, what is the right solution?” And sometimes the no-code, like us, will be a solution and sometimes it may not be. But having the clarity of these things, of these three core pivots [Phonetic 00:26:27] are more important and to think about the future. 

Cassandra Schwartz: I love that you had that down. [Laughter] That’s such a perfect encapsulation of all the things. And I also appreciate how you ended it because that really preps us for this next question which I feel like is such a… I appreciate developers so much, I’ve worked with y’all for years, and you don’t let us slide as marketers on the happy things. And so this person pointed out you’re sharing all these positive things about no-code but what are some of the use cases or what are some of the areas where it’s limited and it’s not going to solve your problem? 

Syed Hamid: Yeah. So, I think anybody who’s telling you this is the stage in the industry that no-code is the solution for all and everything. I think it’s wrong. I would not say that either. Because it’s a progression and you solve problem at the time, right? I mean, there’s still the case where when those operating system or the cloud may not be the right thing. So, having said that, how I see the problem in the no-code, the areas that it cannot solve today, there’s a couple of things, right?  

One is that when you’re having a heterogeneous asynchronous environment. What does that mean? It means is that, let’s take an economical example of ordering, it’s lunchtime, ordering sandwich, right? I could use my app to order sandwich and I have to have a restaurant who’s going to make that sandwich. And then I have a DoorDash or something else that comes and picks that up, right? So, you have a aggregation of three different events happening across the board. And out of the box, the no-code solution is very difficult to encapsulate all those asynchronous systems that are dependent on those things. So, that’s one use case that we are not there yet for the no-code, right? Because multiple app and a heterogenous environment, right?  

The second thing that the no-code is not very good is like it’s streaming and testing like on the streaming. Like you’re testing, as an example, the next [Inaudible 00:28:38] app and on a very specific captions and things like that. So, those things are kind of difficult to validate that, primarily because we are using the technology like machine learning and AI to figure it out, things, and then the responses are in milliseconds, or the system is not advanced enough to kind of generate those kind of a test to execute that. But it’s not that you cannot still do manual and other things in the platform, but you just cannot write the automation on its own that can seamlessly work on it.  

And I think the third category that I would say that no-code is not very efficient today is if you are doing a lot of kind of emojis and very specialized images which is the machine learning modules are still not advanced enough to take out what’s happening on that. For example, in an image you want to find out, “Hey, it’s a shirt that I bought. Is the coloring of that shirt is exactly the same what I intended it to be?” Or like you say, the accessibility type things. So, those are the things. But we are getting there. I think the industry as a whole, as the maturity like, as an example, the OpenAI has a very advanced image recognition algorithms that can be used. So, I think as the maturity of the machine learning modules improves, the no-code will automatically start getting better.  

Cassandra Schwartz: Yeah. And bringing that OpenAI again, and this is just me as a marketer wanting to talk more about Sofy. What’s some of the things that you think we’re going to be seeing here in the near future at Sofy? 

Syed Hamid: Yeah. I think [Inaudible 00:30:21] will be coming in very soon. We’ll be announcing the OpenAI pieces. It’s going to [Inaudible 00:30:28] things. And as I said, right, we are looking at, hard look at OpenAI and saying, “How do we make the QA super, super productive?” So, we have two, three very salient use cases that we think we can really transform and both the generation and execution and reporting using the OpenAI. Because what OpenAI gives you is that it gives you ability to transform your stack data into contextual data. What it means is that, hey, today I can find out what’s the top apps or what’s the top devices used in a different country and automatically go and execute against that. So, I think those contextual data becomes very relevant and we are actively working on it, and we’ll have hopefully soon a dedicated webinar on that too.  

Cassandra Schwartz: Oh, yes. I love a good chance to sit down and chat with you. [Laughter] Well, I’m going to leave it there. For those of you we didn’t get to your question, we will respond to you directly. This webinar will be hosted on our site. So, if you are watching this after it’s been recorded, you can reach out to us. We are happy to answer questions. Syed loves to talk about this, so I love an excuse to sit down and get more answers for you all. And just want to thank those of you that were here today. We really appreciate those questions, really great questions, and we hope to see you on our next one. And thank you, Syed, I appreciate you sitting down with me today. 

Syed Hamid: Thank you very much and have a wonderful day.