QA in the AI Era

AI is rapidly becoming part of the software testing lifecycle. Requirements, test scenarios, test cases, test data and automation scripts can now be generated or evaluated using AI. This is creating a significant increase in testing productivity—but it also creates a new quality challenge: how do we know that the outputs produced by AI are correct, reliable and compliant?

For organizations operating in regulated industries like BFSI, the question becomes even more important. It is no longer enough to test whether an application works as expected. Testing organizations increasingly need to determine whether an AI-powered application behaves within regulatory and organizational boundaries—and whether the AI being used to perform the testing can itself be trusted.

This is where AI compliance testing and LLM-as-a-Judge become important.

Testing AI is different

Traditional QA largely focuses on deterministic outcomes. Given the same input, we expect the same result. AI systems, particularly LLM-based applications, do not always behave this way.

An AI system may produce a response that is factually correct but inappropriate from a regulatory perspective. It may expose information that should not have been disclosed, provide inconsistent explanations, exhibit bias, or behave differently when the same scenario is phrased differently.

Therefore, AI testing needs to go beyond functional correctness.

Testing organizations will increasingly need to validate areas such as accuracy, consistency, bias, explainability, robustness, privacy, security, human oversight and regulatory compliance.

And this needs to be done continuously, because models, prompts, knowledge bases and surrounding applications can all change.

Compliance testing needs to become a testing discipline

Regulations such as the EU AI Act, FEAT, Veritas, financial-sector regulations and emerging AI guidance create another dimension of testing.

The QA organization should be able to take a regulatory requirement and translate it into:

Regulation → Control → Test Scenario → Test Data → Expected Behaviour → Evidence

For example, a regulatory requirement around customer data protection should not remain a statement in a compliance document. It should become executable test scenarios that can determine whether an AI application actually follows that requirement.

This also means that QA teams will need to understand more than testing techniques. They will need some understanding of AI governance, applicable regulations, model behaviour, data protection and AI risk management.

And then comes the LLM-as-a-Judge problem

There is another interesting development.

If AI is generating thousands of artefacts—requirements, test cases, test data, code, and responses —using another LLM to evaluate those artefacts becomes increasingly attractive.

An LLM can act as a judge and evaluate whether an AI-generated output meets a defined set of criteria.

For example:

Does this AI-generated response comply with the bank’s customer-data disclosure policy?

The judge can assess the response against the policy and provide a score, rationale and evidence.

But there is an obvious question:

Who tests the judge?

An LLM judge is itself an AI system. It can hallucinate, misunderstand the regulatory requirement, apply inconsistent reasoning or simply produce a convincing but incorrect assessment.

Therefore, we cannot blindly treat the LLM’s judgement as the ground truth.

The testing organization needs to test the tester

This creates a new layer of QA.

The organization needs to validate the judge itself—its prompts, context, evaluation criteria, reasoning consistency, sensitivity to variations in input, false positives, false negatives and ability to correctly interpret the underlying regulation.

A useful model is therefore:

AI Application → AI Judge → Judge Validation

And for high-impact or regulated use cases, this should be supported by human review, deterministic rules and appropriate evidence rather than relying exclusively on another LLM.

The future QA organization

This doesn’t mean traditional QA disappears. In fact, it becomes more important—but the scope expands.

The future testing organization will need three complementary capabilities:

1. Traditional software quality – functional, API, security, performance and integration testing.
2. AI quality testing – evaluating models, prompts, agents, RAG, guardrails and AI-generated outputs.
3. AI compliance testing – validating AI behaviour against regulations, policies and governance controls.

The organizations that start building these capabilities now will be better positioned for the next phase of AI adoption.

The real transformation is not simply “using AI to test software.”

It is building the capability to test software that uses AI, test artefacts created by AI, test the AI used for testing, and prove that the entire process remains compliant and trustworthy.

That is where AI-native quality engineering is heading.