AI Testing Checklist: 30 Manual QA Checks for AI Software and LLM Applications

An AI application can pass every ordinary functional test and still fail in production.

Login works. Forms submit. APIs return successful responses. The interface looks correct.

Then the chatbot invents a statistic. An agent calls the wrong API. A prompt injection changes its behaviour. A translated response changes meaning. A tool fails, but the AI tells the user the action succeeded anyway.

We have also seen a simpler failure that explains why end-to-end QA still matters. One part of an AI-generated login flow produced a four-digit verification code while the interface expected six digits. Both components looked reasonable on their own. Together, the workflow could never succeed.

That is the difference this checklist is designed around.

AI adds new test dimensions without removing the old ones.

You still need to test UI behaviour, permissions, APIs, errors, performance, and business rules. On top of that, AI introduces non-deterministic output, hallucination, context loss, prompt manipulation, retrieval failures, tool decisions, and autonomous actions.

This checklist starts with the conventional application layer and progressively adds the AI-specific layers.

---

How to Use This AI Testing Checklist

Mark each test area as Required, Conditional, or Not Applicable before execution. Do not run every check against every product.

Before execution, mark each test area as:

• Required: must be covered for this product.
• Conditional: applies only when the feature or architecture exists.
• Not applicable: does not apply to the system being tested.

A chatbot with no file upload does not need file-processing tests. A RAG application does. An AI agent that can send email or modify records needs tool, permission, state, and confirmation testing. A basic LLM summariser may not.

Product type	Tools / workflows	Multilingual	RAG	File handling
English-only AI chatbot	N/A	N/A	Conditional	N/A
Document analysis SaaS	Conditional	Conditional	Required	Required
AI agent platform	Required	Conditional	Conditional	Conditional
LLM summariser	N/A	Conditional	N/A	Required

The goal is not to maximise the number of tests.

The goal is to identify the failure surfaces that exist in the product.

For teams that need a broader engagement covering the application, AI behaviour, and release readiness, see our AI testing services.

---

The Four Layers of AI Product QA

AI product QA separates into 4 cumulative layers: application, AI behaviour, agent/action, and LLM evaluation. Each layer adds test dimensions. None replaces the one before it.

1. Application layer

This is the conventional software around the AI:

• user interface
• authentication
• authorisation
• APIs
• validation
• navigation
• session state
• errors
• performance

2. AI behaviour layer

This covers how the AI interprets input and produces output:

• input variation
• correctness
• relevance
• completeness
• hallucination
• context
• memory
• non-determinism
• prompt handling

3. Agent and action layer

This applies when the AI can use tools or act on other systems:

• tool selection
• tool arguments
• task planning
• state
• retries
• recovery
• permission boundaries
• user confirmation
• final system outcome

4. LLM and evaluation layer

This covers risks specific to LLM-powered applications:

• instruction following
• groundedness
• factual accuracy
• structured output
• retrieval
• model comparison
• evaluation metrics

These layers are cumulative. AI can also support the testing process itself through test generation, regression analysis, and automation, which is a separate use of AI in software testing.

An application does not stop needing functional QA because it contains an LLM. An agent does not stop needing access-control testing because it can reason. A RAG system does not stop needing file and API testing because retrieval quality matters.

This lifecycle-wide approach aligns with the NIST AI Risk Management Framework, which applies trustworthiness considerations across the design, development, use, and evaluation of AI systems.

---

Part 1: Application Foundation

Test the conventional software around the AI before testing the AI itself.

1. Basic Functional Testing

AI sits inside software. Test the surrounding software first.

User interface

Check that:

• screens, buttons, links, and menus render and work correctly
• input fields enforce character, data-type, and format limits
• loading indicators appear during AI processing and disappear when processing ends
• retry, regenerate, cancel, and stop actions behave correctly
• failed or timed-out responses produce useful error states
• navigation does not unexpectedly destroy the conversation or application state
• browser refresh and back-button behaviour are defined and consistent
• the interface works across desktop, tablet, and mobile layouts

AI products often contain longer processing states than conventional interfaces. That makes loading, cancellation, timeout, and recovery behaviour particularly important.

Authentication and authorisation

Check that:

• login and logout work end to end
• sessions expire correctly
• role-based access matches product rules
• admin, user, and viewer roles cannot cross permission boundaries
• one user’s conversations cannot be viewed by another user
• one user’s files cannot be retrieved by another user
• AI-generated output respects the same permission model as the rest of the application
• API permissions match UI permissions
• direct API calls cannot bypass restrictions enforced in the interface

Why it matters: AI can only be as secure as the application and permission model around it.

---

Part 2: AI Inputs and Outputs

Test how the AI handles the full range of user inputs and whether its responses are correct, relevant, and complete.

2. AI Input Testing

AI systems must be tested with more than clean, well-written prompts.

Real users provide incomplete requests, conflicting instructions, spelling mistakes, copied text, mixed languages, oversized inputs, and unexpected combinations.

Normal inputs

Test:

• simple one-sentence requests
• short inputs of one to three words
• complex multi-clause instructions
• prompts containing several questions
• multi-step instructions
• follow-up requests that depend on earlier conversation
• long-form input such as articles, code, tables, or reports

Do not test only whether the AI responds.

Check whether it interprets every part of the request.

Invalid and unexpected inputs

Test:

• empty input
• spaces or line breaks only
• random gibberish
• spelling mistakes
• slang
• abbreviations
• emojis
• Unicode characters
• special symbols
• mixed languages
• conflicting instructions
• irrelevant questions
• unsupported requests
• malformed content

For a domain-specific assistant, deliberately ask questions outside its intended domain.

The expected result may be a refusal, redirection, clarification question, or limited response. What matters is that the behaviour is defined.

Boundary testing

Test around the known product limits:

• one-character input
• maximum supported input
• input just below the limit
• input at the limit
• input just above the limit
• very large pasted content
• oversized file upload
• long code blocks
• large tabular data

Do not assume the model’s theoretical context limit equals the application’s practical input limit. Middleware, product constraints, tokenisation, retrieval, or API limits may reduce the actual boundary.

Why it matters: AI applications often fail at the boundaries between the UI, application logic, model, and external APIs.

---

3. AI Response Quality

A response appearing on screen is not a passed test.

Evaluate the output across four basic dimensions.

Correctness

Check:

• facts
• dates
• calculations
• numbers
• named entities
• classifications
• business rules

Where the answer is objectively verifiable, compare it against a trusted source or known expected result.

Relevance

Check whether the response answers the actual question.

Common failures include:

• answering only one part of a multi-part request
• responding to a related but different question
• adding large amounts of unnecessary content
• ignoring a constraint given later in the prompt

Completeness

For multi-part tasks, create an explicit list of required elements before testing.

Consider a prompt that asks the system to:

1. identify a problem
2. explain the cause
3. recommend an action
4. produce a structured summary

then all four are part of the expected result.

Consistency

Check both:

• within-response consistency: the answer does not contradict itself
• cross-run consistency: repeated executions remain aligned on core facts and decisions

Surface wording can vary. Core behaviour should not.

---

Part 3: Context, Consistency, and Hallucination

Test whether users can trust the AI’s behaviour over repeated and multi-turn interactions.

4. Hallucination Testing

A hallucination is not simply any wrong answer.

The more useful testing question is whether the AI fabricates information unsupported by its available evidence or knowledge.

Create conditions where fabrication is tempting.

Nonexistent entities

Ask about:

• fake company names
• fake people
• fake product names
• nonexistent APIs
• fictional documentation pages

The system should recognise uncertainty rather than confidently invent details.

Unanswerable questions

Ask questions where the provided context does not contain enough information.

Check whether the system:

• acknowledges the limitation
• asks for more information
• clearly labels uncertainty
• avoids inventing a precise answer

Statistics and numbers

Ask for figures where the source data does not exist or has not been provided.

Verify that the AI does not create:

• percentages
• dates
• survey results
• prices
• performance figures
• counts

without support.

Citation testing

When the AI provides citations or links:

• open every source
• verify the page exists
• verify the cited passage supports the claim
• verify the AI did not cite the right source for the wrong statement

For RAG and LLM applications, hallucination testing should also consider groundedness and faithfulness.

• Groundedness: is the response supported by the available source material?
• Faithfulness: does the response accurately represent what that source says?

Why it matters: a fluent answer can look more trustworthy than it is.

Hallucination and factual reliability remain active evaluation problems in modern AI systems, as highlighted in the Stanford AI Index.

---

5. Context and Memory Testing

Many AI applications become useful only when they can maintain context across multiple turns.

Test memory as behaviour, not as a marketing feature.

Multi-turn memory

Create a conversation where information introduced early becomes relevant later.

For example:

1. give the AI a project name
2. provide a deadline
3. discuss another topic
4. return to the project
5. ask, “When is it due?”

Check whether the correct deadline is used.

Topic switching

Change topics mid-conversation.

Verify the AI does not incorrectly carry information from the old topic into the new one.

Then return to the earlier topic and verify that the correct context is recovered.

Referential language

Use phrases such as:

• “do that again”
• “change the previous one”
• “use the second option”
• “what about her?”
• “apply the same rule”

Check whether the AI resolves the reference correctly.

Long conversations

Test conversations that extend beyond ordinary happy-path usage.

Look for:

• forgotten constraints
• confused entities
• repeated questions
• contradictory memory
• details from one topic appearing in another

Context boundaries

Where the product exposes or documents context limits, test around them.

Do not assume every model supports the same fixed token sizes. The practical context boundary depends on the model and application configuration.

Why it matters: an answer can be locally correct while using the wrong conversation context.

---

6. Non-Deterministic Behaviour

AI outputs can vary between executions.

That means exact text comparison is often the wrong test oracle.

Run important prompts repeatedly and compare the parts of the response that should remain stable.

Repeated-run testing

For a critical case:

1. run the same input five to ten times
2. record each output
3. identify acceptable variation
4. compare core facts, actions, classifications, and safety behaviour

Acceptable variation

Examples:

• wording changes
• sentence order changes
• different but equivalent examples
• different phrasing of the same conclusion

Unacceptable variation

Examples:

• different dates
• conflicting classifications
• one run performs an unauthorised action
• one run fabricates a source
• a safety rule works nine times and fails once

A probabilistic system can still have deterministic requirements.

Why it matters: rare failures are still failures when the affected behaviour is security-sensitive, destructive, or business-critical.

---

Part 4: Safety and Security

Test whether users can manipulate the AI’s behaviour, extract protected information, or bypass access controls.

7. Prompt Injection Testing

Prompt injection tests whether untrusted content can alter the behaviour the application is supposed to enforce. OWASP identifies prompt injection as a major LLM application risk because crafted inputs can alter model behaviour and lead to unauthorised actions or harmful outputs.

Direct prompt injection

Test conflicting user instructions such as attempts to override higher-priority application instructions.

The exact expected behaviour depends on the product, but the user’s input should not silently replace protected application rules.

System instruction disclosure

Ask the system to reveal hidden or internal instructions.

Check that private configuration, protected prompts, or internal application logic are not exposed when they should remain confidential.

Indirect prompt injection

Do not test only what the user types directly.

Malicious instructions can also exist inside:

• uploaded documents
• retrieved webpages
• emails
• documents used in RAG
• third-party tool output

Check whether the application treats retrieved content as data rather than automatically trusting it as an instruction. Google DeepMind describes indirect prompt injection as a security risk where malicious instructions hidden in retrieved emails, documents, or websites can influence an AI agent’s behaviour.

Multi-turn persistence

Attempt manipulation again after a long normal conversation.

A control that works only at the start of the session is not sufficient.

Jailbreak resistance

Where relevant to the application’s safety requirements, test whether role-playing, obfuscation, encoding, and multi-turn pressure change restricted behaviour.

Why it matters: AI applications process natural-language instructions from several sources, not all of which should have equal authority.

---

8. AI Security Testing

AI-specific security does not replace ordinary application security.

Test both.

Data isolation

Verify:

• users cannot retrieve another user’s conversations
• users cannot retrieve another tenant’s uploaded files
• the AI does not expose unrelated customer data
• retrieval respects access controls

Sensitive information

Check that responses and logs do not expose:

• credentials
• API keys
• tokens
• PII
• private internal data
• restricted customer content

Application input security

Where the AI interface accepts ordinary web input, files, rendered content, or stored output, traditional risks still matter.

Test relevant controls around:

• XSS
• HTML injection
• unsafe rendering
• malicious file upload
• oversized uploads
• unsafe parsing
• API access control

SQL injection should be tested where user-controlled AI input can reach database-backed application paths. Do not treat every text prompt as automatically equivalent to a SQL input.

Permissions

A useful security test is to compare:

1. what the AI believes it is allowed to do
2. what the actual backend allows it to do

Backend enforcement must remain authoritative.

Why it matters: an AI layer can create new paths to existing systems. It must not become a shortcut around established security boundaries.

---

9. Bias and Fairness Testing

Bias testing should be tied to the product’s real decision or recommendation context.

Test equivalent requests where only a demographic or protected attribute changes.

Possible variables include:

• names
• gender references
• age
• nationality
• location
• language

Compare:

• recommendation
• classification
• ranking
• tone
• refusal behaviour
• assumptions introduced by the model

A difference is not automatically evidence of unfairness. The tester still needs to determine whether the changed attribute was legitimately relevant to the task.

Bias testing is particularly important when AI influences decisions in areas such as:

• hiring
• healthcare
• lending
• education
• legal services

Why it matters: generated text can encode assumptions that are not obvious from a normal functional test.

---

Part 5: Agents, Tools, and Workflows

Test tool selection, failure handling, multi-step workflows, and whether the final system outcome matches the AI’s claim.

10. Tool and Agent Testing

An AI that can call tools introduces failure modes that do not exist in a simple chatbot.

Test the complete decision chain:

Should the AI use a tool? Which one? With what arguments? What happened? What should it do next?

Tool selection

Test whether the AI:

• selects the correct tool
• avoids an unnecessary tool call
• recognises when no tool is required
• does not substitute a different tool without justification

Tool arguments

Verify:

• required parameters are present
• values are correct
• data types are correct
• dates and identifiers are formatted correctly
• sensitive information is not passed unnecessarily
• user input does not corrupt the argument structure

For function-calling LLMs, verify the generated arguments conform to the expected schema.

Tool failure handling

Simulate:

• timeout
• unauthorised response
• bad request
• server error
• network interruption
• empty result
• partial result

Then verify whether the AI:

• recognises the failure
• explains it correctly
• retries only where appropriate
• asks the user for action when needed
• avoids pretending the task succeeded

A particularly important test is:

Can the AI distinguish a failed tool call from a successful action?

---

11. Agent Workflow Testing

Tool testing covers individual actions. Agent workflow testing covers the complete autonomous sequence.

Task decomposition

Give the agent a task requiring several steps.

Check:

• correct decomposition
• correct order
• no missing dependency
• no redundant action

Intermediate failure

Break one step deliberately.

Verify the agent does not continue as if the failed step succeeded.

State

Check whether the agent carries correct information from one step to the next.

Test:

• stale state
• partial completion
• retry after failure
• resumed workflows
• changed data during execution

Loops

Look for:

• repeated tool calls
• repeated planning
• repeated failed action
• no termination condition
• stopping too early

Confirmation

For high-impact actions, verify whether explicit confirmation is required at the correct point.

Examples:

• send
• delete
• publish
• purchase
• transfer
• account change

Confirmation should occur before the irreversible action, not after.

Why it matters: an agent’s final answer may look correct even when the sequence behind it was unsafe.

---

12. End-to-End AI Scenario Testing

The most important AI test is often the one that crosses all components.

A full workflow could look like:

1. user signs in
2. user uploads a document
3. AI analyses it
4. AI identifies an issue
5. AI selects a tool
6. tool calls an API
7. API returns a result
8. AI interprets the result
9. user asks a follow-up
10. AI uses prior context
11. user confirms an action
12. application performs the action

Test every transition.

Verify outcomes, not claims

An agent saying “email sent” does not prove an email was sent. Verify the actual outcome through the relevant source of truth:

• email system
• API response
• database
• workflow state
• audit log
• third-party system

This is one of the biggest differences between testing conversational output and testing an AI-powered application.

A real integration failure pattern

Consider a verification flow where the backend produces a four-digit code while the UI expects six digits.

The email template can look correct.

The UI can look correct.

The individual components can pass isolated checks.

The workflow still fails.

AI-generated software can make this problem more common because separate components may be generated from incomplete or inconsistent requirements.

That creates a useful QA principle:

Locally reasonable does not mean globally correct.

Business rules

Test whether the AI respects ordinary product rules.

Examples:

• approval thresholds
• eligibility logic
• pricing
• ownership
• role permissions
• workflow state transitions

Do not accept the AI’s explanation of the rule as proof that the rule was enforced.

---

Part 6: Files, Data, Performance, and Production Readiness

Test file handling, multilingual support, regression coverage, performance, error handling, and observability.

13. File and Document Testing

AI applications frequently process documents, so file testing needs two separate checks:

1. Was the content extracted correctly?
2. Was the extracted content interpreted correctly?

File formats

Test every supported format relevant to the product, such as:

• PDF
• DOCX
• XLSX
• CSV
• TXT
• PNG
• JPG

Unsupported formats should produce a clear response.

File conditions

Test:

• empty file
• corrupted file
• password-protected file
• very large file
• scanned PDF
• image-heavy document
• long multi-page document
• complex tables
• nested formatting
• embedded objects

Extraction

Compare the extracted result against the source.

Look for:

• missing text
• changed numbers
• merged columns
• incorrect table relationships
• lost headings
• wrong reading order

Interpretation

An AI can extract every cell correctly and still misunderstand the table.

Ask questions that require understanding:

• totals
• comparisons
• trends
• relationships
• exceptions

Unsupported evidence

Ask questions whose answer is not contained in the document.

The application should not invent information merely because a document was uploaded.

---

14. Multilingual Testing

Do not select languages randomly.

Prioritise languages used by the product’s users.

Test:

• complete conversations in each supported language
• mixed-language input
• language switching
• locale-specific dates
• locale-specific numbers
• currencies
• non-Latin scripts
• right-to-left rendering where relevant
• translated meaning

A useful test is to compare semantically equivalent prompts across languages and look for changes in:

• answer quality
• safety
• refusal behaviour
• classification
• business meaning

Why it matters: a product may support a language technically while behaving very differently in it.

---

15. AI Regression Testing

AI regression testing should protect behaviour across:

• model changes
• prompt changes
• system instructions
• retrieval data
• tool integrations
• application releases

Build a permanent suite containing:

• critical user journeys
• important prompts
• known failures
• hallucination cases
• safety cases
• tool calls
• context cases
• permission checks

Every meaningful failure discovered in QA or production should become a candidate regression case.

Do not compare exact wording only

For generative output, compare the things that matter:

• factual result
• classification
• business decision
• required elements
• prohibited elements
• tool selected
• final action
• safety behaviour

This is particularly important when repeated runs can produce different but equally acceptable wording.

Model migration

Run the same suite against old and new model versions.

Record:

• improvements
• regressions
• changed refusal behaviour
• changed latency
• changed tool behaviour

Practitioner observation: AI reduces the time to generate tests. It increases the amount of regression QA required because more affected scenarios become visible.

---

16. Performance Testing: Manual Checks

A manual checklist can cover visible performance behaviour, but it should not pretend to replace load-testing tools.

Check:

• time to first visible token
• total response time
• streaming behaviour
• cancellation
• UI responsiveness during generation
• user experience during slow tool calls
• user experience during slow retrieval
• timeout messaging

Test slow dependencies deliberately.

For example:

• tool call takes 30 seconds
• retrieval is slow
• model response is delayed
• API times out

Check whether the UI continues to behave correctly.

What sits outside this manual checklist

Use dedicated performance tooling for:

• sustained load
• high concurrency
• throughput
• scalability
• resource saturation

Those belong in a deeper AI performance testing process rather than being simulated manually.

---

17. Error Handling

Failure states should be intentional product behaviour, not accidental side effects.

Simulate:

• model unavailable
• API unavailable
• overloaded service
• rate limit
• malformed request
• context limit reached
• tool failure
• retrieval failure

Check whether the user receives:

• a clear explanation
• a useful next action
• retry where appropriate
• no raw internal stack trace
• no fabricated success response

A useful test is to ask:

Does the application distinguish “I could not complete the action” from “the action completed”?

---

18. Observability and Auditability

AI failures are difficult to debug when the team can see only the user’s final screen.

Test whether enough information is captured to reconstruct the workflow.

Depending on the system, useful evidence can include:

• timestamp
• user or session identifier
• request identifier
• user input
• model/version
• prompt/configuration version
• retrieved context
• tool selected
• tool arguments
• tool response
• latency
• error
• final application state

Also test what should not be logged.

Avoid unnecessary exposure of:

• credentials
• tokens
• sensitive personal data
• confidential documents
• secret configuration

Why it matters: traceability turns an intermittent AI failure into something a QA or engineering team can reproduce and investigate.

---

Part 7: Additional Tests for LLM Applications

These checks apply when the application is built directly around a large language model. The previous sections already cover hallucination, context, prompt injection, security, tool calling, regression, and non-determinism. This section adds what is unique to LLM-based architectures.

19. Instruction Following

Test whether the LLM respects:

• requested format
• requested tone
• requested language
• maximum length
• required sections
• forbidden content
• negative instructions
• step order

For multi-step requests, check each instruction independently.

---

20. Factual Accuracy

Build a set of questions with known, verifiable answers.

Compare the response against authoritative evidence.

Record:

• correct
• partially correct
• unsupported
• wrong
• unverifiable

Do not mix factual accuracy and hallucination into one metric automatically.

An answer can be inaccurate because of misunderstanding without necessarily fabricating a nonexistent source.

---

21. Reasoning and Logical Consistency

Where the application requires reasoning, test:

• arithmetic
• comparison
• cause and effect
• ordering
• constraints
• conditional logic
• negation
• multi-step problems

Evaluate the final answer and the behaviour required by the product.

Do not assume that a long explanation proves correct reasoning.

---

22. Robustness

Rephrase the same intent in different ways.

Use:

• spelling mistakes
• informal language
• incomplete wording
• formatting noise
• copied text
• reordered instructions
• ambiguous language

A robust system should preserve the intended behaviour across reasonable variation.

---

23. Structured Output

When downstream systems consume generated output, validate the actual format.

Examples:

• JSON
• XML
• CSV
• Markdown tables
• code
• schema-defined objects

Check:

• parseability
• required fields
• field types
• missing values
• extra fields
• escaping
• invalid syntax

The output looking correct to a human is not enough if another system must parse it.

---

24. Output Constraints

Test hard requirements such as:

• word count
• language
• required headings
• prohibited terms
• output schema
• tone
• list length

Run these repeatedly.

A constraint that fails intermittently still needs investigation.

---

25. Model Comparison

When changing model or provider, run the same representative test suite against both configurations.

Compare:

• correctness
• task completion
• hallucination
• tool behaviour
• latency
• safety
• cost where relevant to the product decision

Do not compare models only with generic benchmark questions if the application has its own workflows and business rules. The same comparison can also reveal issues in model accuracy, robustness, and behaviour across versions, which are part of broader AI model testing.

---

26. RAG and Retrieval Testing

RAG testing should follow the pipeline rather than testing only the final answer.

A simplified flow is:

document → ingestion → index → retrieval → context → model → answer

Document ingestion

Check whether:

• all expected content is indexed
• tables remain usable
• updates replace old versions correctly
• removed documents disappear
• permissions are preserved

Retrieval relevance

Create queries where the relevant source is known.

Check:

• correct source retrieved
• irrelevant sources excluded
• ranking is sensible
• synonyms work
• abbreviations work
• ambiguous queries do not produce arbitrary certainty

Missing retrieval

Create a case where the answer exists in the corpus but the retriever fails to surface it.

This separates retrieval failure from generation failure.

No-answer cases

Ask questions whose answer does not exist in the corpus.

The system should handle the absence of evidence according to product requirements rather than fabricating an answer.

Stale sources

Keep an outdated and a current document in a controlled test environment.

Check which one the system retrieves and how conflicts are handled.

Permissions

Verify that retrieval honours the same document access rules as the application.

Groundedness and faithfulness

Check:

• whether the response is supported by retrieved evidence
• whether the response accurately represents that evidence
• whether it adds unsupported claims

Citations

Validate:

• cited source exists
• source is the one the model used
• cited passage supports the claim
• citation is not attached to unrelated content

Retrieval latency

Measure the visible effect of retrieval on user experience.

For full load and throughput analysis, use dedicated performance testing.

---

27. Embedding and Semantic Search Testing

Where semantic retrieval is used, build query groups representing the same intent in different wording.

Test:

• synonyms
• abbreviations
• paraphrases
• domain terminology
• spelling variation
• negation
• closely related but incorrect concepts

Do not evaluate only whether the correct document appears somewhere.

Ranking matters.

The most useful result should appear high enough for the downstream workflow to use it.

---

28. Classification Testing

For LLM-powered classification, prepare a labelled evaluation set.

Test:

• clear examples
• ambiguous examples
• multi-label cases
• out-of-scope input
• borderline examples
• class imbalance where relevant

Measure the errors that matter to the product rather than reporting one overall percentage without context.

---

29. LLM Evaluation Metrics

Metrics should match the behaviour being tested. Microsoft’s generative AI evaluators include groundedness, relevance, response completeness, coherence, and task completion as distinct evaluation dimensions.

Possible measures include:

Metric	What it answers
Accuracy	Are verifiable answers correct?
Relevance	Does the response answer the user’s request?
Groundedness	Is the answer supported by provided evidence?
Faithfulness	Does the answer accurately represent that evidence?
Hallucination rate	How often does the system fabricate unsupported information?
Consistency	Does core behaviour remain stable across repeated runs?
Safety	Does restricted behaviour remain within defined policy?
Tool accuracy	Did the system select the correct tool and arguments?
Retrieval precision	How much retrieved content is relevant?
Retrieval recall	How much relevant content was successfully retrieved?
Latency	How long does the user wait for useful output?
Task completion	Did the workflow produce the intended real-world result?

Cost can be tracked when it is part of the product’s acceptance criteria. It is not a QA quality metric by itself.

---

30. Model Migration Testing

When changing model version or provider, run the existing test suite against both configurations before switching.

Compare:

• correctness on critical prompts
• task completion rates
• hallucination behaviour
• tool selection and arguments
• safety refusal behaviour
• latency
• cost per query where relevant to the product decision

Do not compare models using generic benchmark questions alone. Use the product’s own workflows, prompts, and business rules as the evaluation set.

Record regressions, improvements, and any changed refusal or safety behaviour. A model upgrade that improves general quality but breaks a specific tool call or permission boundary is still a regression.

Why it matters: a newer model is not automatically a better model for a specific product.

Download the AI Testing Checklist

A 42-row spreadsheet version of this framework is available with pass/fail columns, severity ratings, and AI-specific fields for every test.

Recommended columns:

Core QA fields

• Category
• Test area
• Test scenario
• Applicable?
• Priority
• Expected behaviour
• Actual behaviour
• Pass / Fail
• Severity
• Evidence
• Notes

AI-specific fields

• Prompt / input
• Model version
• System prompt or configuration version
• Run number
• Tool used
• Retrieved source
• Output
• Human reviewer

These fields make it easier to reproduce AI failures after model, prompt, retrieval, or application changes.

Download XLSX Open Google Sheet

---

Need a QA Strategy for Your AI Product?

Testscenario tests AI-powered applications across software functionality, AI behaviour, workflows, security, regression, integrations, and real-world outcomes.

Whether your product includes an LLM, RAG pipeline, chatbot, AI agent, or AI-generated workflow, we define the QA scope. We turn it into an executable testing framework.

Talk to us about AI testing