Regression tests for your AI chatbot or voicebot, in your own CI
A test-automation engineer writes a regression suite for your chatbot or voicebot. It lives in your repo, runs in your CI and covers Slovak, Czech and German. There is no new SaaS to buy, and no conversation data goes to a testing vendor.
What's at stake for your business
A wrong answer from your bot is your company's answer. This is what it costs when answers drift unnoticed.
Your bot speaks for your company
In Moffatt v. Air Canada (2024), a tribunal in British Columbia ordered the airline to pay a customer after its website chatbot described a fare rule wrongly. It found the airline responsible for all information on its website, whether from a static page or a chatbot.
Source: Moffatt v. Air Canada, 2024 BCCRT 149 (Civil Resolution Tribunal, British Columbia)
Regulators expect AI disclosure
Since 2 August 2026, Article 50 of the EU AI Act requires that people are told they are talking to an AI, unless it is obvious. Breaches of these transparency duties can be fined up to €15 million or 3% of worldwide annual turnover, whichever is higher; for SMEs and start-ups, whichever is lower.
Source: EU AI Act, Article 50 (AI Act Service Desk); EU AI Act, Article 99 (AI Act Service Desk); AI Act Service Desk: when enforcement starts
Wrong facts cost sales and support time
A wrong shipping threshold, price or opening time sends customers away or to your support team. A bot bought to reduce tickets creates new tickets, and refunds, when its answers drift from your terms.
Trust is built in your customers' language
Customers share screenshots of wrong or strange answers. In Slovak, Czech or German, broken grammar, or Slovak and Czech mixed in one answer, makes your brand look careless, even when the facts are right.
Why bots break without anyone noticing
Small changes break last week's answers
You adjust a prompt, switch the model or update the knowledge base. Answers that worked last week can quietly change, and often nobody notices until a customer does.
Slovak and Czech are hard to check
Most generic testing tools are weak in Slovak and Czech. Their automatic checks miss wrong grammar, odd phrasing and answers that slip from one language into the other.
People must know they are talking to AI
Since 2 August 2026, Article 50 of the EU AI Act requires that people are told they are talking to an AI system, at the latest at the first interaction, unless it is obvious.
What we test
Every suite is built from scenarios that fit your bot, your customers and your languages. These six areas become repeatable checks that run in your pipeline.
Goal success in conversations
Multi-turn conversations driven toward a goal, such as booking, cancelling or making a complaint. We check that the customer actually gets there.
- Scripted and simulated multi-turn chats
- Personas with different goals and styles
- Repeated runs to spot unstable answers
Answers match your own facts
We compare answers with your website or knowledge base: prices, opening hours, addresses and policies. A wrong fact is a failed test.
- Exact checks for prices and opening hours
- Policies compared with your own terms
- No invented answer when the facts are missing
Language quality in Slovak, Czech and German
Automatic checks for language and grammar in Slovak, Czech and German, and for Slovak–Czech mixing. A native speaker reviews the language results, so a model is not the only judge.
- Catches Slovak and Czech mixed in one answer
- Grammar and tone scored against a rubric
- Results reviewed by a native speaker
Tool calls and knowledge retrieval
When your bot books, cancels or looks something up, we check that it calls the right tool with the right data and answers from the right source.
- Expected function name and arguments
- No forbidden or unexpected calls
- Answers backed by the retrieved sources
AI disclosure evidence
Evidence that your bot says it is an AI at the first interaction, in every channel and language we test, as Article 50(1) of the EU AI Act requires.
- Chat: checked in the first bot message
- Voice: checked in the first utterance
- Transcripts and timings kept as evidence
Latency and cost per conversation
Slow replies and rising model costs are regressions too. We measure response time per turn and tokens and cost per conversation against your budget.
- Response time per turn, median and p95
- Tokens and cost per conversation
- The run fails when a budget is exceeded
How it works
Scope and written authorization
We agree which bot, channels, languages and scenarios are in scope. You confirm in writing which endpoint we may test, how many conversations we may run and on which dates.
Built against your staging
We build the suite against your staging endpoint, or a test tenant with synthetic content, using a scoped test key you can revoke. Never against production with real users.
Handed over into your repo
You receive the suite as a pull request into your own repository. Your CI runs it: GitHub Actions, GitLab or Jenkins. You add the job and its secrets yourself.
Re-run on every change
The suite runs again whenever a prompt, model or knowledge base changes. With monthly upkeep, we add new scenarios and send you a monthly report.
What a finding looks like
An illustrative example, invented for this page. It does not describe a real client, shop or bot.
Illustrative example
- What we asked
- A customer asks the shop's assistant: how many days do I have to return goods without giving a reason?
- What your terms say
- Goods can be returned within 30 days of delivery, without giving a reason.
- What the bot said
- You can return goods within 14 days of delivery.
- Why it matters
- Customers are told they have fewer rights than you actually give them. The test stays red until the bot's answer matches your terms again.
Where to start
Start with a free sample. If it is useful, move on to a full audit, and then to a suite that runs in your own pipeline.
Free sample
Free
A first look at how your bot answers, at no cost. You nominate the bot and authorize the test in writing.
- Results typically within 3 working days
- 20 scenarios, text only, one language
- Run against a bot you nominate and authorize in writing
- 2-page report
- 15-minute walkthrough of the findings
- One sample per company
Baseline audit
from €1,490
A full audit of your bot in Slovak, Czech and/or German, with the language checked by a native speaker.
- Report typically within 10 working days
- 100 scenarios, text
- Slovak, Czech and/or German
- Language checked by a native speaker
- Full report with findings and evidence
- One retest after your fixes
CI suite and monthly upkeep
On request
The baseline audit, plus the suite handed into your repo and pipeline and kept up to date every month.
- First delivery typically in 3–4 weeks, then monthly
- Everything in the baseline audit
- Suite handed over by pull request
- Runs in GitHub Actions, GitLab or Jenkins
- New scenarios and re-runs every month
- Monthly report
All prices exclude VAT. Larger scopes and voice testing are quoted after a short call.
For agencies
You build chatbots and voicebots for your clients. We run a launch check on each bot you deliver and write the report under your agency's name. You keep the client relationship. Price on request.
- A launch check for each bot you deliver
- White-label report under your agency's name
- Language, fact and AI-disclosure checks
How we handle your data
- Synthetic test data only: invented names, phone numbers and orders. No real customer data in our scenarios.
- No production transcripts or recordings, unless we have first signed a data processing agreement with you.
- You choose the judge model and its provider, and the suite uses your own API key, so the data stays in your own chain of processors.
- No production credentials and no admin rights in your CI. We work with a scoped test key that you can revoke.
- We delete run artifacts after 30 days, unless we agree otherwise.
What this is not
- It is not legal advice, and it is not an official approval under the AI Act. You get test evidence; your counsel decides what it means for you.
- Machine-readable marking of AI output under Article 50(2) is out of scope. Whether and how it applies to your bot is a question for your counsel.
- Test results are a snapshot of the scenarios we ran, on the dates we ran them. They do not prove that your bot always answers correctly.
Our reports are test evidence, not legal advice and not a certification. They show how your assistant answered the scenarios we ran, on the dates we ran them.
Legal sources (checked on October 19, 2026)
Questions
Do you need access to our CI or production systems?
No. We build the suite against your staging endpoint or a test tenant, using a scoped test key you can revoke. You receive the suite as a pull request, add it to your own CI and keep the secrets there. We never need production credentials or admin rights in your CI.
Which chatbot and voicebot stacks can you test?
Any bot we can reach through an HTTP or API endpoint, or through a chat widget on a web page. The suite is written in TypeScript on promptfoo, an open-source testing tool, so your developers can read it, run it and extend it.
Can you test voicebots?
Yes. We test the conversation logic through your bot's text or API channel, which is faster and cheaper to run. Tests with real audio calls are available on request and quoted after a call.
Where does our data go?
As little as possible leaves your systems. Scenarios use synthetic data, the suite runs in your CI, and the judge model runs on your own API key with a provider you choose. We only process real transcripts under a signed data processing agreement, and we delete run artifacts after 30 days.
Why not just use a self-serve testing platform?
A platform gives you a tool, but someone still has to design the scenarios, judge the Slovak, Czech and German answers and keep the tests up to date. We do that work, with a native speaker reviewing the language. If you already use a platform, we can write scenarios for it too.
Does this cover our obligations under the AI Act?
No. It gives you evidence that your bot tells people they are talking to an AI, in each channel and language we test. What your obligations are, and whether you meet them, is a question for your counsel.
Start with a free sample
Tell us about your chatbot or voicebot. We will suggest a scope for a free 20-scenario sample and the written authorization it needs.
Prefer to see an example first? Mention it in your message and we will send you a one-minute video of typical findings.