To help us provide you with free impartial advice, we may earn a commission if you buy through links on our site. Learn more
There’s been massive backlash against AI in video game development recently, but I got a glimpse of how AI could benefit the industry at Gamescom last week.
Tucked away in a booth at Koelnmesse, I saw a demo of Modl.ai, a new AI tool testers can use to automate components of game testing and QA. The tool, created by a nine-strong team of Danish software developers, caught the attention of Side, a global video game services provider whose offerings include audio production, localisation and quality assurance, a couple of years ago.
The two parties have now signed a six-month memorandum of understanding “to focus on product development and commercial pilots in the video games sector”, and I spoke with leading figures at both companies about what the partnership means to them, and how it might impact the gaming industry.
The best solution for AI integration into Side’s QA business
Side’s Chief Commercial Officer, Martin McBride, told me he sees the Modl.ai tool as a compelling answer to many of the questions the company has asked during conversations about the best solution for AI in its QA line of business.
“We’ve been doing QA for about 30 years, and we’ve never really found a toolset that can fulfil our needs, our clients’ needs, and bring the efficiencies at scale that our clients want,” he said.
“The benefits of any AI tool have to be that it does something faster, better or cheaper; otherwise, there’s no point in doing it. We think that this partnership with Modl.ai creates a solution for our clients that does that.”
Human input remains essential to QA testing
Modl.ai is not about to fully automate the video game QA industry, however. The company’s founder, Christoffer Holmgard, describes it as a “force multiplier” for QA testers. “The way we see it is that it can become a power tool for testers, so they can take their experience and expertise and then leverage it to a much deeper end,” he commented.
Chief Product Officer, Benedikte Mikkelsen, added: “We’ve tried to make this an interface for humans to work with the AI. You will still have human creativity in the way that you task the agent or the way that you interpret the results that come in, so this is very much just a 10x for someone who’s testing.”
Modl.ai: How does it work?
Explaining (in simplified terms) how the tool works, Holmgard said: “You take the build, whether it’s in development or the final product, put it into the system, and the AI operates it according to instructions in natural language. When that is done, the second part of the system analyses it and finds issues.”
Through direct human interactions with the system, such as uploading reference material or descriptions of what the game is about, the tool builds out a knowledge base about what the game should be. Once this base understanding is established, the QA AI engineer can instruct the tool what to do and what to test, crucially without needing to access the source code. This independence from code means the QA team won’t need to engage code engineers, making the process significantly more efficient.
Non-invasive, automated gameplay testing
The human operator defines what the AI agent does via “task” commands, building a script for the system. The tester can then play a video feed of the running game build from their perspective without ever having amended the game’s code. Modl.ai then presents the tester with evidence of potential issues it has discovered, leaving the decision whether to escalate or ignore at human discretion. Testers can view full recordings of every test, including timestamps for any potential bugs encountered. Simple tasks like analysing and tweaking a game’s UI could take as little as two hours of QA tester-led work, according to Mikkelsen.
This two-way relationship builds over time; the more the human interacts with the game, the more information becomes available to guide the AI agent’s decision-making process and the better and faster it becomes at testing.
Ultimately, the aim is to save testers time by removing the need to run hundreds of repetitive tasks, the kind of tasks that often see human errors creep in. The time saved in QA allows developers to change up their workflows, for example, creating more builds of a game, to deliver a superior end product for consumers. With its ability to spot spelling errors and inconsistencies in language usage, it may also come in handy for those working on Side’s LQA (Localisation Quality Assurance) teams.
Training a new breed of AI QA engineer
McBride confirmed that Side and Modl.ai “are working with several different developers and publishers on the program and that because human input is essential, QA engineers are being upskilled to use Modl.ai effectively. “We have to help our testers start thinking more outside the box,” he said.
“One of the things that makes a good prompt engineer is not only being able to take a very complicated problem set and distil it down into something an LLM can understand, but also think strategically about how you want to solve a problem in the first place.
“That skillset isn’t inherent in a lot of testers because normally it’s ‘Here’s a test plan: execute on test plan’”. There’s not a lot of deviation. And a lot of that is because they have to do a lot of repetitive tasks, so they don’t have the time to be strategic.”
McBridge reckons it will take a QA engineer between three and six months to get up to speed, but that this will prove a worthwhile investment for all parties. “We get to move our people up the value chain,” he continued.
“One of the problems that the QA industry has had is that a lot of the work has gone to areas where it can be done for a lower price point. By training our people to become QA AI engineers as opposed to simply just testers, you can pay them more money, and those jobs can return to higher cost-of-living areas.”
Embracing the use of AI in video game QA
McBride drew a clear line between AI involving generative elements like imagery and audio and how Modl.ai is harnessing it for quality assurance. “Everyone we’ve talked to has been very receptive to QA AI solutions,” he told me.
“I think a lot of the negative feedback relates to customer-facing assets: anything where a player interacts with an NPC or another player, and a human asset is computer- or AI-generated, there’s some negativity around that.
“But what we’re trying to do is move games more effectively at a better price point and more efficiently for the end users. One data point I saw found that 77% of developers said they could have used more AI polish on their games before they went out the door. They simply didn’t have the opportunity, funds, or ability to, so tools like this close the gap.
“The project managers are trying their best. They want to deliver games that are good, fast and cheap, but you can only really have two in a lot of cases. Modl.ai allows us to sort of bridge the gap and somewhat have all three.”
Just the beginning
Modl.ai currently works best on slow-paced games such as puzzle games. Explaining why, Holmgard said: “A lot of it comes down to speed in the sense that there are certain latencies involved when you’re using some of the larger models.”
The AI engine picks from various LLMs, and these aren’t always responsive enough to handle huge, open-world games with multiple actions happening instantly and simultaneously just yet.
But Holmgard feels the partnership with Side will be “instrumental” in moving toward testing larger, more complex titles. “We are pushing on making it able to operate games directly, all the way up to, eventually, real-time action games.”
But Holmgard feels the partnership with Side will be “instrumental” in moving toward testing larger, more complex titles. “We are pushing on making it able to operate games directly, all the way up to, eventually, real-time action games.”
How quickly that happens will depend on how fruitful the next six months prove. This is not possible to track from the outside, but if the two parties are still working together when Gamescom 2027 rolls around, it’ll be fair to say the tool has legs.
Light at the end of the gaming tunnel?
It’s been a crazy year for video games. Despite an incredible array of quality releases (with plenty more to come in Q4), we’ve seen several high-profile projects cancelled, nearly 30 studios shut down, and over 10,000 jobs lost.
Studio closures and widespread layoffs are nothing new; it’s been a savage few years since the pandemic. With expectations higher than ever, corporate strategies shifting to cater for changing player habits, and games costing more and taking longer to make, developers and publishers are really feeling the heat.
In those conditions, it’s natural for game developers (and perhaps more pertinently, the companies that own them) to look for ways to make their processes more efficient and less resource-intensive. Harnessing Modl.ai to improve QA looks like a smart way to do this, and that’s coming from an AI sceptic who harbours existential fears about our reliance on technology.
The secrecy surrounding game development will make it nigh-on impossible to assess the impact Side’s working relationship with Mod.ai has on the wider industry. But if it leads to career growth for industry professionals and fewer bugs in games, it’ll be a big win for everyone. Gamers will get more polished products, which should result in more positive public and critical reception, giving more efficiently run studios a better chance of success.
You can’t automate the human soul
I still have concerns over AI: there’s no escaping its environmental footprint and the soaring costs of hardware, as components are gobbled up to power data centres. And I’ll leave turning to chatbots for relationship advice to others.
But as I walked away from my chat with Martin, Christoffer and Benedikte, a quote from Red Dead Redemption 2 came to mind. In that game, Reverend Swanson says to Arthur Morgan: “You’re not a good man, Arthur, but you’re not all bad, either.”
I feel very similarly about AI when I reflect on the moral and ethical issues it raises, so I will end with a final quote from Martin McBride that gave me hope. “My CEO has an expression that you can’t automate the human soul,” he said.
“I think that’s a very apt description of how we treat AI at Side. It’s a knob we can turn up and down depending on the client’s use case, but one that requires a human in the loop no matter where you are.”
