How Has Testing Changed in the World of AI?

Most teams using AI coding tools have had this moment. The AI writes a hundred automated checks for a new feature, every one of them passes, and still nobody can say for sure that the release is safe. That gap between lots of checks and actual safety was the topic of Tech Night #19 in September.

Marián Rusnák came at it from product quality. He is Lead Software Engineer at Definic, a platform that helps large enterprises choose IT service providers, and spent 14 years in London and New York, including building ad tools at Yahoo, before moving back to Košice. Lukáš Naď came at it from security. He has been at Siemens Healthineers since 2020, first as a penetration tester (an ethical hacker paid to break into the company's own software) and since 2025 as the lead of that team. Here is what each of them told us.

Marián: testing so you can release without fear

Marián's talk started from a tension every startup knows. The team has to ship fast and change direction often, but every release also has to work. Automated tests are how software teams square that: small programs that check the product still does what it should each time someone changes the code. The trouble is that the textbook approach, where every small piece of code gets its own test, produces a huge pile of tests that is slow to write and breaks every time the product changes course.

AI makes that pile grow much faster. The first feature arrives with dozens of AI-written tests. By the fifth there are a thousand, and the team feels safe. Then three features get redesigned, most of those tests break and have to be rewritten, and eventually there are more tests than the AI can keep track of. In Marián's experience AI is good at writing tests but bad at judging which ones protect the business, and nobody wants to maintain what it produces.

So Marián flips the question. Instead of asking how much of the code is tested, he asks whether the team can release without being afraid of it. That leads him to build up testing in a set order:

1. Agree on what you are building and which parts of the product customers rely on most.

2. Write end-to-end tests early. These are automated scripts that click through the product the way a real user would, for example signing up or creating a request.

3. Later, add tests for the system behind the screens, for example checking that one customer can never see another customer's data.

4. Test individual pieces of code only where a mistake would be really costly.

5. Later still, run the tests automatically on every change, cover every new feature once you have customers, and add other kinds of tests as you grow.

Starting with end-to-end tests keeps the pile small, because a single test that walks through a whole user journey covers several features at once. AI still has a role, just a narrower one: it creates realistic sample data and clicks through the app to see what happens. The team's testing rules live in a shared instructions file that developers and AI coding assistants both read.

It has worked at Definic. The team now runs about 180 user-journey tests, which finish in around 12 minutes, plus more than 800 smaller technical tests. Each developer looks after the tests for their own features, and a new version goes live with one click and no manual checking. Reported bugs peaked at 54 in July 2025, when the product was growing fastest, and fell once the tests caught up. When Marián got to his "what's next" slide, the first item was security testing, "thanks Lukáš", which is exactly where the second talk picked up.

Lukáš: finding the hole before an attacker does

Lukáš opened with: "There is already a flaw in your code. The only question is who finds it first."

His first argument for looking early was cost. A security hole caught while the software is still being designed is cheap to fix. Once it's live, fixing it means delayed releases, unhappy customers and possibly fines. According to IBM's 2026 Cost of a Data Breach Report, the average data breach now costs $4.99 million, up 12% in a year, and healthcare is the most expensive sector at $6.64 million. For a medical technology company like his, patient safety, regulators and customer trust weigh as heavily as the money.

That is why his team looks for weaknesses at every stage, from design reviews that ask "how could someone abuse this?" to watching live products. Along the way they rely on four main methods, and Lukáš was clear that none of them is enough on its own:

• Code scanning (SAST): a tool reads the code and flags risky patterns before anything runs. It raises many false alarms and can't see how the live system behaves.

• Live testing (DAST): a tool attacks the running app from the outside, like an intruder would. It only works once the app is running, so problems surface late.

• Vulnerability scanning: checks the software you use against public lists of known weaknesses. It misses anything that hasn't been publicly reported yet.

• Penetration testing: skilled people try to break in, combining small flaws into real attacks. It's a snapshot in time, and there are only so many experts.

Those gaps are where AI comes in. People often ask Lukáš whether every new AI model release puts their products at risk. His answer is that the model is only one of four parts. The model is the brain that decides what to try. The tools are its hands, the hacking utilities it actually runs. The workflow is the routine that lets the same test be repeated. The guardrails set what it's allowed to touch and make sure a person reviews the results. As his slide put it: "A model without tools cannot test, and tools without governance must not."

Put together properly, AI makes each of the four methods better without replacing any of them. It can sort through hundreds of scanner alerts and point out the ones that matter, come up with new ways to misuse an app for live testing, and take over the repetitive groundwork of a penetration test so the human experts can focus on the clever attacks. Penetration testing is where his team has gotten the most out of it so far.

For teams that want to try this, Lukáš suggested starting with the pile of alerts your scanners already produce, because that is where false alarms waste the most time and a mistake costs little. From there, give the AI context about your own systems and past problems, and keep a person checking its conclusions, because the AI doesn't take responsibility for what it finds.

The common thread

Lukáš ended with a line that could just as well have closed Marián's talk: "We find more than ever. Deciding what happens next is the hard part." His answer is a simple routine of confirming each problem is real, giving it an owner, ranking it and checking the fix, plus the idea of a dedicated day when the team does nothing but fix known issues.

That line sums up the evening: AI has made tests and warnings cheap to produce, so the hard part has moved to choosing which parts of the product deserve a test, which warning is a real risk, and who fixes it. Marián's answer is to keep his tests few and focused on what customers do. Lukáš lets AI search wider and leaves the decisions to people.

What to take away

If AI writes more and more of your code, focus on the few things that would hurt most if they broke or leaked, check them on every release, and have a person review whatever the AI produces before anyone acts on it. Thanks to Marián and Lukáš for sharing what they've learned.

If you're building AI systems that need to hold up in production, book a call with Sudolabs and we'll go through your setup with you.