How to Assess AI Governance and Compliance in 2026 | AI LLM Hacking Course Day 39 of 90

In Day 38, I took you inside the fine-tuning process and showed you where the security risks can hide. Today, in Day 39, we’re stepping back and looking at the governance layer around that technology — the policies, processes, risk controls, and human oversight that are supposed to keep everything accountable.

And there’s an important distinction here: having a policy doesn’t mean the organisation is actually following it. I’m going to show you how I test that gap between what the documentation says and what people actually do.

Then, in Day 40, we’ll move from prevention to response. I’ll walk you through how to detect, contain, and recover from AI security incidents — including the problems AI-enabled attacks create that traditional incident response processes aren’t always prepared for.

The AI Regulatory Landscape in 2026

If you’re coming from a traditional security background, the AI regulatory landscape can initially look like a collection of overlapping frameworks. NIST AI RMF, the EU AI Act, ISO/IEC 42001, ISO 27001, privacy regulations, sector-specific requirements — where does a security tester even start?

I find it much easier to understand these frameworks if you stop treating them as competing standards. They answer different questions. The EU AI Act is a legal and regulatory regime. NIST AI RMF is a risk-management framework that helps organisations identify, measure, manage, and govern AI risks. ISO/IEC 42001 is a management-system standard for establishing and continually improving an Artificial Intelligence Management System (AIMS). They overlap, but they are not interchangeable.

1. EU AI Act — the legal baseline

The EU AI Act is the one I treat differently from the other two because it isn’t simply a voluntary governance framework or an audit standard. It creates legal obligations based on the risk and role associated with an AI system.

As of August 2026, the Act has reached a major enforcement milestone. The AI Act entered into force in August 2024, and its general application began on 2 August 2026. Prohibited AI practices and AI literacy requirements have applied since February 2025, while governance requirements and obligations for general-purpose AI models have applied since August 2025. The Commission and national authorities now have enforcement powers for the provisions that are applicable.

There is an important detail here that I want you to remember: “the AI Act is applicable” does not mean every high-risk requirement became enforceable on 2 August 2026. Following the 2026 AI Omnibus changes, the high-risk rules for certain Annex III systems are scheduled to apply from 2 December 2027, while high-risk AI embedded in regulated products under Annex I moves to 2 August 2028.

That distinction matters during an assessment. I don’t want to write a finding that simply says, “The organisation is not compliant with the EU AI Act.” I first establish which AI system is being assessed, what role the organisation has, which provisions apply, and which application date applies to that provision.

For high-risk systems, the requirements eventually become very familiar to anyone who has worked in security assurance: risk management, data quality, logging and traceability, technical documentation, transparency, human oversight, accuracy, robustness and cybersecurity. Providers also have lifecycle responsibilities around compliance, incident handling and corrective action.

That gives me a useful testing checklist. I can take each requirement and ask a very practical question: “Show me the evidence that this control operates.”

2. The 2026 transparency requirements

One area that is particularly relevant to security testers right now is transparency. Article 50 transparency obligations apply from 2 August 2026. Depending on the system and use case, providers and deployers may have obligations around informing people that they are interacting with AI, identifying AI-generated or manipulated content, and informing people when they are exposed to certain systems such as emotion-recognition or biometric categorisation systems.

This creates a very practical testing opportunity. Don’t just read the organisation’s transparency policy. Test the actual user experience.

If the organisation says users are informed that they’re interacting with an AI system, open the application and verify it. If AI-generated content is supposed to carry machine-readable marking, inspect the output. If deepfakes are supposed to be disclosed, generate or retrieve an authorised test sample and verify the disclosure mechanism.

That’s the difference between compliance documentation review and compliance testing. One tells you what the organisation says it does. The other tells you what the system actually does.

3. NIST AI RMF — the risk-management layer

NIST AI RMF works differently. It is a voluntary framework designed to help organisations manage AI risks across the lifecycle. Its four core functions are Govern, Map, Measure, and Manage. The framework is deliberately flexible, so an organisation can adapt it to its own technology, sector and risk profile.

For me as a security tester, the value of NIST AI RMF is that it gives structure to questions that otherwise become very subjective.

• Govern: Who owns the AI risk? Who can approve deployment? What policies and accountability mechanisms exist?
• Map: What is the intended use? Who can be affected? What harms and dependencies have been identified?
• Measure: How is the organisation testing accuracy, security, privacy, bias, robustness and other trustworthiness characteristics?
• Manage: What happens when testing identifies unacceptable risk? Who makes the decision to mitigate, accept, transfer or reject that risk?

And notice something important: these aren’t purely technical questions. A penetration test might demonstrate that an API can be abused. NIST AI RMF pushes me to ask the governance question that comes next: Was that risk known? Was it measured? Who accepted it? Was there a mitigation deadline? Did anyone verify that the mitigation actually worked?

NIST is also continuing to evolve. AI RMF 1.0 is currently being revised, while NIST has already published the Generative AI Profile and is developing additional profiles, including work around trustworthy AI in critical infrastructure. So I would always record the exact framework version and profile used in an assessment rather than simply writing “NIST compliant.”

4. ISO/IEC 42001 — turning governance into a management system

ISO/IEC 42001 approaches the problem from a management-system perspective. It specifies requirements for establishing, implementing, maintaining and continually improving an Artificial Intelligence Management System. In other words, it is concerned with how an organisation builds a repeatable system for governing AI rather than simply checking whether an individual AI model is secure.

This is where the governance assessment starts becoming very familiar if you’ve worked with ISO 27001. I want to see the organisation’s policies, objectives, risk assessment process, roles and responsibilities, operational controls, performance monitoring, internal audits, corrective actions and continual-improvement process.

But again, I don’t stop at the documents.

If the organisation tells me it performs AI risk assessments, I ask to see a completed assessment. If it says management reviews AI risks, I ask for the review record. If it says suppliers are assessed, I ask to see the supplier assessment and then sample the evidence. If it says incidents are tracked, I ask to see the incident register and trace one incident from detection through corrective action.

ISO/IEC 42001 is particularly useful here because it is built around continual improvement. The organisation isn’t supposed to create an AI governance binder, put it on a shelf, and call the job finished. The management system is expected to be maintained and improved as the organisation’s AI risks and operations change.

5. Where the frameworks overlap

This is where the three frameworks become much more useful together.

Suppose an organisation operates a high-impact AI decision system. The EU AI Act may establish legal requirements around risk management, documentation, human oversight, cybersecurity and monitoring. NIST AI RMF gives the organisation a structured way to identify and manage those risks. ISO/IEC 42001 provides the management-system structure for assigning responsibilities, maintaining processes, evaluating performance and improving the governance system.

So I don’t recommend running three completely separate assessments. Instead, I build a control crosswalk.

The important point is that mapping is not proof of compliance. A spreadsheet showing that “Control A maps to NIST and ISO 42001” doesn’t demonstrate that Control A works. It only demonstrates that someone created a mapping.

6. How this connects to the first 38 days

This is where Day 39 connects directly to everything you’ve already learned.

The technical testing from Days 1–38 gives you evidence about the security of the AI system. Prompt injection testing, data leakage testing, model security, RAG security, agent security, supply-chain testing, fine-tuning security and the other controls don’t disappear when we move into governance. They become evidence for the governance assessment.

For example, if your technical testing discovers that a model can be manipulated through prompt injection, the governance question becomes: Was prompt injection included in the organisation’s AI threat model? Was the risk formally assessed? Was a mitigation assigned? Who accepted the residual risk? Has the mitigation been retested?

If you discover training-data leakage, you can move from the technical finding to the governance layer: Who approved the training dataset? Was data provenance documented? Was privacy risk assessed? Was the dataset subject to a security review? What happened when the issue was discovered?

This is why I describe governance testing as the layer that connects technical security to organisational accountability.

7. The real target: the governance gap

The biggest findings I’ve seen in governance assessments usually aren’t caused by the complete absence of a policy. Organisations know they need policies. They know auditors will ask for documentation. So the documents often exist.

The more interesting question is whether the documents describe reality.

A policy might say that every AI system receives a risk assessment before production. The evidence might show that three systems went live without one.

A procedure might say that human reviewers can override AI decisions. The interview might reveal that reviewers don’t understand when they are expected to intervene.

A supplier policy might require AI vendors to provide security documentation. The procurement team might have accepted a vendor’s self-attestation without validating anything.

A monitoring procedure might require periodic bias testing. The dashboard might exist, but nobody can explain when the last test ran or what happened after an anomalous result.

Those are governance findings.

And this is the mindset I want you to carry into the rest of Day 39: don’t ask only whether a control exists. Ask whether you can prove that the control operates.

That one change in mindset turns a document review into an actual security and compliance assessment.

EU AI Act Risk Classification and Requirements

One of the first questions I ask during an AI governance assessment is deceptively simple: “What risk category does this AI system fall into under the EU AI Act?”

I say “deceptively simple” because organisations often give me an answer based on the technology rather than the use case. They might tell me, “It’s only a chatbot,” or “It’s just a recommendation engine.” That doesn’t give me enough information. I need to understand what the system actually does, who uses it, what decisions it influences, who can be affected, and how much authority the AI has in the process.

The EU AI Act uses a risk-based approach with four broad categories: unacceptable risk, high risk, transparency risk, and minimal or no risk. The classification determines which obligations apply and, just as importantly, which obligations do not apply.

1. Unacceptable-risk AI — systems that are prohibited

At the top of the hierarchy are AI practices that the EU considers unacceptable because of the threat they pose to people’s safety, livelihoods or fundamental rights. These aren’t systems where I ask, “What controls are missing?” The first question is much more fundamental: “Is this use legally permitted at all?”

The prohibited practices include things such as harmful manipulation or exploitation of vulnerabilities, certain forms of social scoring, individual criminal-offence risk prediction based solely on profiling, untargeted scraping to build facial-recognition databases, emotion recognition in workplaces and educational institutions, certain biometric categorisation practices, and real-time remote biometric identification for law enforcement in publicly accessible spaces, subject to the Act’s specific exceptions and conditions.

There is also an important 2026 update: the prohibition covering AI systems that generate non-consensual sexually explicit or intimate content or CSAM material is scheduled to apply from 2 December 2026.

From a security-testing perspective, I treat prohibited-use testing differently from ordinary compliance testing. If an assessment identifies a potentially prohibited practice, I’m not going to recommend “improving the control” as though the system simply needs better logging. The finding may require stopping the prohibited use, changing the use case, or obtaining specialist legal advice on the precise applicability of the provision.

2. High-risk AI — where governance becomes much more demanding

The next category is high-risk AI. This is where most of the governance testing work becomes interesting.

And there’s an important point that I want you to get right: high-risk classification is not simply a judgment that an AI system is “important” or “dangerous.” Article 6 identifies specific routes to high-risk classification.

The first route concerns AI systems that are safety components of products, or AI systems that are themselves products, covered by the Union harmonisation legislation listed in Annex I where the relevant product requires a third-party conformity assessment. The second route covers AI systems falling within specified use cases listed in Annex III.

Those Annex III areas include sensitive domains such as biometrics, critical infrastructure, education, employment, access to essential private and public services, law enforcement, migration and border control, and the administration of justice and democratic processes. The exact classification depends on the statutory conditions and the particular use case, not simply on the fact that AI is being used in one of these sectors.

This is why I don’t accept an answer such as “We’re in finance, so our AI is high-risk.” I ask the organisation to show me the classification decision.

What use case was assessed? Which Article 6 route was considered? Which Annex III provision applies, if any? What facts were used to reach the conclusion? Who approved the classification? And what happens if the system’s purpose changes?

That last question matters more than it first appears. An AI system can evolve. A tool initially used to provide internal recommendations might later become part of an automated decision workflow. When that happens, the original risk assessment may no longer describe the actual system.

3. What I actually test in a high-risk AI assessment

Once I establish that a system falls within the high-risk requirements, I move from classification to controls. The Act sets expectations around areas including risk management, data quality, logging and traceability, technical documentation, information for deployers, human oversight, accuracy, robustness and cybersecurity.

This gives me a practical assessment structure:

• Risk management: Show me the AI risk assessment. What hazards and fundamental-rights risks were identified? What mitigation decisions were made?
• Data governance: Where did the training, validation and testing data come from? What quality checks were performed? How were discriminatory or inappropriate data characteristics identified?
• Logging: Can you reconstruct what the AI system did for a particular decision? Are relevant events recorded with enough detail to investigate an incident?
• Technical documentation: Can an auditor or competent authority understand what the system is intended to do, how it works at the required level of documentation, and what limitations exist?
• Human oversight: Is there a real human intervention mechanism, or are people simply approving whatever the AI produces?
• Accuracy and robustness: What testing demonstrates that the system performs within its intended parameters?
• Cybersecurity: What happens when someone deliberately attacks the model, its data, its interfaces or its supporting infrastructure?

Notice how many of these questions connect directly to the technical testing we’ve already covered in this course. Governance doesn’t replace security testing. It gives the technical findings an accountability and compliance context.

4. Human oversight — the control that often looks better on paper

Human oversight deserves special attention because this is where I frequently find a difference between “human in the loop” and “human actually exercising control.”

Imagine an AI system recommends whether a customer’s application should be approved. The organisation tells me, “Don’t worry, every decision is reviewed by a human.” That sounds reassuring until I ask the reviewer what they actually do.

Do they independently assess the evidence? Can they challenge the model? Can they override the recommendation? Do they understand the system’s limitations? Are they given enough information to identify a bad recommendation? Are overrides logged? Is the reviewer measured on how quickly they approve decisions, creating pressure to accept the AI’s recommendation?

If the reviewer simply clicks Approve on every recommendation, the existence of a human does not automatically make the oversight control effective.

So my test is simple: “Show me the last time the human overrode the AI.”

If the answer is “never,” I investigate further. Maybe the AI really has performed perfectly. But maybe the oversight mechanism isn’t functioning as intended. The evidence matters more than the wording of the procedure.

5. Transparency-risk AI — don’t confuse this with “low importance”

The third category is often described as limited-risk AI, but I prefer to explain it as AI systems subject to specific transparency obligations. The distinction is useful because “limited risk” can make people think there are no meaningful compliance requirements.

From 2 August 2026, Article 50 transparency obligations apply to certain AI systems. These include requirements concerning direct interaction with AI, AI-generated or manipulated content, deepfakes, certain AI-generated text concerning matters of public interest, and certain emotion-recognition or biometric-categorisation systems.

For example, if a user is interacting directly with a system covered by the transparency requirements, the system may need to make clear that the user is interacting with AI. Providers of relevant systems also have obligations concerning machine-readable marking of AI-generated or manipulated content, while deployers have specific disclosure obligations for certain deepfakes and other covered content.

This creates some excellent security-testing opportunities.

I don’t simply read the transparency policy. I interact with the application. Does the disclosure actually appear? Is it visible? Can a user easily understand it? Does the marking survive the content pipeline? Can the machine-readable marker be detected? What happens when content is edited, transformed or passed through another system?

The European Commission’s 2026 transparency guidance specifically addresses how these obligations can be demonstrated, and the Code of Practice on Transparency of AI-Generated Content provides a voluntary mechanism that can help organisations demonstrate compliance. However, following the code is not itself conclusive proof of compliance.

6. Minimal or no-risk AI

At the bottom of the hierarchy are AI systems considered minimal or no risk. The AI Act does not impose additional mandatory requirements simply because these systems use AI. The Commission gives examples such as AI-enabled video games and spam filters.

But I wouldn’t interpret that as “no security assessment required.” That’s an important distinction.

An AI system can be minimal-risk under the AI Act while still creating security, privacy, contractual, operational or sector-specific risks. The AI Act classification answers one regulatory question. It doesn’t magically eliminate every other obligation the organisation may have.

7. How I test the classification decision

This is the part I want you to take into your own assessments. Don’t treat the risk classification as a label someone entered into a governance spreadsheet. Treat it as an assessment that needs evidence.

I normally work through the following sequence:

1. Identify the AI system. What model or AI capability is actually being assessed?
2. Identify the provider and deployer roles. Who developed it, who places it on the market, and who is using it?
3. Document the intended purpose. What is the system officially supposed to do?
4. Document the actual use. What are people really using it for?
5. Check prohibited practices. Does the use case potentially fall within Article 5?
6. Check the high-risk routes. Does Article 6(1) or Article 6(2) apply?
7. Check transparency obligations. If it isn’t high-risk, are there Article 50 obligations?
8. Record the rationale. Can another assessor reproduce the classification decision from the evidence?
9. Establish change triggers. What change in purpose, capability, user population or deployment context would require reassessment?

That last step is especially important. I don’t want an organisation to classify an AI system once and then forget about the classification for the next five years. AI systems change. Models change. integrations change. The people using them change. Sometimes the business process around the model changes completely.

8. A practical example: AI customer-service agent

Let’s take the customer-service example because it demonstrates why context matters.

Suppose an AI chatbot answers routine customer questions and hands complicated cases to a human. I wouldn’t classify that system as high-risk simply because it is an AI system handling customers. I would first establish the exact use case and then test it against the Act’s applicable classification rules.

Now change the design. The AI can autonomously approve refunds, reject complaints, make decisions affecting access to an important service, or take actions that materially affect a person’s rights or interests.

The risk picture has changed substantially. That doesn’t automatically mean “high-risk” under Article 6 — I still have to test the actual statutory criteria — but it tells me that the classification deserves much closer scrutiny.

And that’s the mindset I want you to develop: don’t classify the technology; classify the regulated use case.

9. What evidence should I collect?

When I finish the classification stage, I want to be able to show exactly how the conclusion was reached. At minimum, I look for:

• AI system description and intended purpose
• Provider/deployer role analysis
• Business process and decision-flow documentation
• Users and affected individuals
• Article 5 prohibited-practice screening
• Article 6 high-risk assessment
• Applicable Annex I or Annex III analysis
• Transparency obligation assessment
• Risk assessment and identified harms
• Human oversight design
• Evidence supporting the final classification decision
• Triggers for reassessment when the system or its intended purpose changes

If the organisation cannot show me why it classified the system the way it did, that becomes a governance finding in itself. I may not yet have proved that the classification is legally wrong, but I have identified a weakness in the organisation’s classification process.

10. One final 2026 point: check the date before writing the finding

There is one mistake I want you to avoid when reporting EU AI Act findings in 2026: don’t assume that every requirement applies simply because the AI Act is now generally applicable.

The Act became generally applicable on 2 August 2026, and enforcement powers and several obligations are now active. However, the application of the high-risk rules has been extended: high-risk systems in the relevant Annex III areas are scheduled to come under those rules from 2 December 2027, while high-risk AI embedded in regulated products under Annex I has an extended date of 2 August 2028.

That means a good 2026 assessment distinguishes between current legal obligations, future obligations that the organisation should be preparing for, and voluntary governance practices. Don’t collapse all three into a generic “EU AI Act compliance” finding.

That’s the real skill I’m trying to teach you here. EU AI Act testing isn’t about memorising four risk labels. It’s about taking an actual AI deployment, understanding what it does in the real world, identifying the regulatory pathway that applies, and then testing whether the organisation has evidence to support its classification and controls.

NIST AI RMF — Testing the Four Functions

If you’ve never worked with the NIST AI Risk Management Framework before, the easiest mistake to make is to treat it like another compliance checklist. I don’t approach it that way.

NIST AI RMF 1.0 gives me a structure for asking whether an organisation actually understands and manages its AI risks. The Core is built around four functions: GOVERN, MAP, MEASURE, and MANAGE. NIST deliberately designed these as a flexible risk-management structure rather than a rigid sequence of steps, and the GOVERN function is intended to operate across the other three.

So when I assess an organisation against NIST AI RMF, I’m not asking, “Did you complete all four boxes?” I’m asking a more useful question:

“Can you show me evidence that these four activities are actually happening throughout the AI system lifecycle?”

1. GOVERN — Is somebody actually accountable for AI risk?

I start with GOVERN because it establishes the organisational foundation for everything else. NIST describes governance as a cross-cutting function: the policies, processes, accountability structures, risk culture and organisational practices should influence how AI risks are mapped, measured and managed.

This is where I stop looking at the model for a moment and start looking at the organisation around it.

I want to know:

• Who owns AI risk?
• Who can approve an AI system for production?
• Who can stop or suspend an AI system?
• Who is responsible for security testing?
• Who is responsible for privacy and data governance?
• Who is responsible for monitoring model performance after deployment?
• How are AI risks escalated to senior management?
• How are third-party AI providers governed?

Then I ask for evidence.

Show me the AI governance policy. Show me the roles and responsibilities. Show me the approval record for a production AI system. Show me the risk committee minutes. Show me an example where an AI risk was escalated and somebody actually made a decision about it.

That last request is particularly useful.

A policy telling me that “AI risks are escalated to management” is weak evidence. A meeting record showing that a specific AI risk was escalated, discussed, assigned an owner, given a treatment decision and subsequently tracked is much stronger evidence.

My GOVERN test is therefore simple: can I trace AI accountability from the policy all the way to an actual decision?

2. GOVERN failure example

Imagine an organisation has deployed an internal generative AI assistant. The governance documentation says the CISO owns AI security risk.

I interview the CISO.

The CISO tells me the AI platform is actually managed by the data science team, and security only gets involved when there is an incident.

I interview the data science team.

They tell me the business owner approves new AI use cases.

I interview the business owner.

They tell me they assumed security had already approved the platform.

Nobody is deliberately doing anything wrong. The problem is that accountability exists in the documentation but not in the operating model.

That is exactly the type of governance gap I want to uncover.

3. MAP — Does the organisation actually understand the AI system and its context?

Once governance establishes who is responsible, I move into MAP.

MAP is about understanding the context in which the AI operates and identifying the risks associated with that context. NIST describes the function around understanding the intended purpose, context, impacts, relevant actors, limitations and risks associated with the AI system.

This is where I ask the organisation to explain the system without hiding behind technical terminology.

What does the AI actually do?

Who uses it?

Who can be affected by its decisions?

What data does it consume?

What systems does it connect to?

What decisions does it influence?

What happens when it gets something wrong?

And perhaps most importantly:

“What did you identify as the worst realistic harm this system could cause?”

If the team cannot answer that question, I already know the MAP function needs attention.

4. How I test MAP

I normally compare three things:

1. The documented purpose — what the organisation says the AI is supposed to do.
2. The technical implementation — what the system actually does.
3. The operational use — what employees or customers actually use it for.

These three descriptions should broadly agree.

If the documentation says an AI assistant only answers internal HR questions, but I discover that employees are also using it to generate performance evaluations, I’m no longer looking at the same risk profile.

That’s why MAP is so important. Risk depends heavily on context.

5. MAP failure example

Imagine an organisation originally deployed an LLM as an internal writing assistant.

Six months later, employees have integrated it into customer support. It now summarises customer complaints, recommends refunds and drafts responses that agents send with minimal review.

The original AI inventory still describes the system as a “productivity assistant.”

That’s a MAP problem.

The technology didn’t necessarily change. The context changed.

And when the context changes, the risk assessment needs to change with it.

6. MEASURE — Can the organisation prove that it tests AI risk?

Next comes MEASURE.

This is where the framework becomes particularly useful for security professionals because MEASURE connects governance to actual testing and evidence. NIST describes the function as assessing, analysing and tracking identified AI risks.

This is where I want to see numbers, test results, metrics, evaluation reports, monitoring data and evidence of independent assessment.

Depending on the AI system, I might ask for evidence covering:

• Security testing
• Prompt-injection testing
• Data leakage testing
• Model robustness
• Accuracy and reliability
• Bias and fairness evaluation
• Privacy testing
• Adversarial testing
• Hallucination or factuality testing
• Performance monitoring
• Drift monitoring
• Incident and anomaly detection

The exact tests depend on the system. NIST AI RMF is intentionally use-case agnostic, so I don’t expect a chatbot and an autonomous industrial system to have identical evaluation requirements.

7. Don’t accept “we tested it” as evidence

This is another place where I push organisations beyond documentation.

If someone tells me, “We performed security testing,” I ask:

“Show me the test.”

Then:

• What was tested?
• When was it tested?
• What methodology was used?
• What were the results?
• What failed?
• Who reviewed the findings?
• What remediation was performed?
• Was the remediation retested?

If the organisation has a 40-page AI security policy but cannot produce a single completed security evaluation, the policy isn’t the evidence I need.

The evidence is the evaluation.

8. Test whether the metrics actually measure risk

A sophisticated organisation may have dozens of AI metrics. That doesn’t necessarily mean it is measuring the right things.

For example, an AI support assistant might report:

• 99.2% uptime
• 1.4-second average response time
• 94% user satisfaction

Those are useful operational metrics, but they don’t tell me whether the model is leaking confidential information or generating unsafe recommendations.

So I ask:

“Which metrics would tell you that the AI has become unsafe?”

That question usually produces a much more interesting conversation.

9. MANAGE — What happens after you discover a risk?

This brings me to MANAGE.

Finding an AI risk is not the end of the process. Somebody has to decide what to do about it.

NIST describes MANAGE as prioritising and responding to identified risks, including treatment planning, incident response, recovery and continual improvement.

So I ask:

“You found the risk. Now what?”

Does the organisation:

• Accept the risk?
• Mitigate it?
• Transfer it?
• Restrict the AI system?
• Change the model?
• Change the workflow?
• Pause deployment?
• Retire the system?

And who makes that decision?

10. Test the risk-treatment process

I like to take an actual finding and follow it through the organisation.

Suppose a red-team assessment discovered that an AI agent can be manipulated into exposing information it shouldn’t reveal.

I don’t stop at the technical finding.

I trace the issue into the risk register.

Was it recorded?

Was its impact assessed?

Was somebody assigned responsibility?

Was a remediation deadline established?

Did management accept the residual risk?

Was the fix implemented?

Was the system retested?

Can the organisation demonstrate that the risk actually decreased?

That entire chain is far more valuable to me than a statement saying, “We follow NIST AI RMF.”

11. Test the four functions together

Here’s where the four functions become much more powerful.

Notice that these aren’t four independent audits.

They form a feedback loop.

Govern establishes accountability. Map establishes context and identifies risks. Measure provides evidence about those risks. Manage determines what to do about them. The results then feed back into governance and the next assessment cycle.

NIST explicitly describes AI risk management as continuous and intended to operate throughout the AI lifecycle rather than as a one-time exercise.

12. A complete NIST AI RMF testing example

Let’s put all four functions together with a simple scenario.

An organisation deploys an AI system that assists employees in screening job applications.

GOVERN: I establish who owns the AI, who approved its deployment, who owns the risk, and who is responsible for monitoring fairness and security.

MAP: I document what the system actually does, what applicant data it uses, what decisions it influences, who is affected, and what harms could result from an incorrect or biased recommendation.

MEASURE: I examine the organisation’s testing for accuracy, bias, robustness, security and other relevant trustworthiness characteristics. I also check whether monitoring continues after deployment.

MANAGE: I take any identified risk and trace what the organisation does with it. Does somebody decide whether to mitigate, accept, restrict or stop the system? Is the decision documented? Is remediation verified?

Now imagine the organisation has excellent documentation but discovers that the model systematically produces worse recommendations for one applicant group.

If the organisation records the finding, investigates the cause, assigns ownership, implements a mitigation, retests the model and documents the result, the MANAGE function is working.

If the organisation records the problem but leaves the model in production indefinitely because nobody wants to own the decision, that’s where the governance system breaks down.

13. How I connect NIST AI RMF to security testing

This is particularly important for this course because we’re not studying governance in isolation.

Suppose one of the technical assessments from earlier in the course finds a prompt-injection vulnerability.

I can immediately translate that technical result into the four NIST functions:

• GOVERN: Who owns the risk from prompt injection?
• MAP: Where can prompt injection affect the system, and what could an attacker reach?
• MEASURE: Has the organisation tested the vulnerability and quantified its impact?
• MANAGE: Was the vulnerability remediated, accepted, or otherwise treated — and was the result retested?

The same approach works for data leakage, insecure RAG, excessive agent permissions, model supply-chain risks, insecure APIs, training-data problems, model manipulation and other AI security findings.

This is why I don’t treat governance testing as something separate from technical security. The technical finding tells me what can go wrong. The governance assessment tells me whether the organisation knows about it, owns it, measures it, and does something about it.

14. The evidence checklist I use

For a practical NIST AI RMF assessment, I would typically request evidence such as:

• AI governance policy
• AI system inventory
• AI ownership and accountability matrix
• AI risk assessments
• AI impact assessments
• Data-flow and system architecture documentation
• Threat models
• Security and adversarial testing reports
• Model evaluation results
• Monitoring dashboards and metrics
• Incident records
• AI risk register
• Risk-treatment plans
• Management approvals and risk-acceptance records
• Remediation evidence
• Retest results
• Third-party AI supplier assessments

I don’t expect every organisation to have every document. NIST AI RMF is voluntary and flexible, and the appropriate evidence depends on the organisation, AI system, use case and risk profile. The NIST Playbook itself is explicitly described as a source of suggested actions rather than a mandatory checklist.

15. Don’t report “NIST compliant” without qualification

One final point is important when you write your assessment report.

NIST AI RMF 1.0 is intended for voluntary use. It is not itself a certification scheme.

So I would avoid a conclusion such as:

Instead, I would say something more precise:

That tells the reader exactly what you assessed and avoids turning a voluntary framework into a claim of formal certification.

16. The question I want you to remember

When I finish a NIST AI RMF assessment, I want to be able to answer four questions:

GOVERN — Who is responsible?

MAP — What can actually go wrong, and in what context?

MEASURE — How do you know?

MANAGE — What did you do about it?

If an organisation can answer all four questions with credible operational evidence, I’m looking at a functioning AI risk-management process.

If it can answer only the first two with policies and diagrams, but has no meaningful testing or risk-treatment evidence, then I know exactly where the governance gap is.

And that’s the real value of the NIST AI RMF for a security tester. It gives you a way to connect technical findings to organisational accountability without pretending that a policy document is proof that a control works.

Human Oversight Effectiveness Testing

Human oversight is one of my favourite controls to test because it exposes the difference between governance that exists on paper and governance that actually works. An organisation can have a beautifully written human oversight procedure, clearly assigned reviewers, escalation rules, approval workflows, and training records. None of that tells me whether a human is actually exercising meaningful control over the AI.

This matters because human oversight appears across the major AI governance frameworks. The EU AI Act includes human oversight requirements for applicable high-risk AI systems, while NIST AI RMF and ISO/IEC 42001 provide governance structures in which human accountability, roles, monitoring, and intervention become important parts of responsible AI management. The exact legal obligation depends on the system and applicable requirements, so I don’t begin by assuming that every AI system has the same oversight requirement.

Instead, I ask a much more practical question:

“Show me how a human can detect, challenge, and override a bad AI decision.”

1. “Human in the loop” doesn’t automatically mean human oversight

This is one of the first concepts I want you to understand.

An organisation may tell you that its AI is human-in-the-loop. That sounds reassuring, but the phrase itself doesn’t tell you how meaningful the human involvement is.

Consider a simple workflow. An AI evaluates a loan application and produces a recommendation. A reviewer receives the recommendation and has two buttons: Approve and Reject.

Technically, a human is involved.

But now ask what the reviewer actually sees. Do they receive the evidence used by the model? Do they understand why the model reached its recommendation? Can they independently challenge the result? Are they trained to recognise model limitations? Do they have enough time to perform the review? Can they override the AI without managerial approval? Are overrides recorded and investigated?

If the reviewer sees only “AI recommendation: APPROVE” and clicks the same button, the organisation has a human somewhere in the workflow, but that doesn’t necessarily demonstrate effective human oversight.

This is why I don’t accept “there is a human approval step” as evidence by itself. I test what that human actually does.

2. Test whether the reviewer has real authority

The first thing I establish is whether the reviewer has genuine authority to intervene.

I ask:

• Can the reviewer override the AI recommendation?
• Can the reviewer reject an AI-generated action?
• Can the reviewer pause the workflow?
• Can the reviewer escalate an unusual decision?
• Can the reviewer request additional evidence?
• Can the reviewer disable or quarantine an AI workflow when a serious problem is detected?

Then I verify those answers against the actual system.

If the procedure says that reviewers can override the model but the application doesn’t give them that capability, I’ve found a control-design problem. If the capability exists but reviewers are discouraged from using it, I may have found an operational problem. If reviewers can technically override the model but don’t know when they should, that’s a training and governance weakness.

The distinction matters because I want my finding to identify the actual failure rather than simply saying, “Human oversight is insufficient.”

3. Test whether reviewers actually review

This is where the assessment gets interesting.

I take a sample of real, authorised AI decisions and trace each one through the review process. I’m looking for evidence that the reviewer performed an independent assessment rather than simply accepting the model’s recommendation.

For each sampled decision, I might examine:

• What information was presented to the reviewer?
• How long did the review take?
• Did the reviewer access the supporting evidence?
• Did the reviewer add comments or reasoning?
• Did the reviewer ever request additional information?
• Was the AI recommendation changed?
• Was an escalation triggered?
• Is there an audit trail showing what the reviewer actually did?

I don’t use review duration as proof that someone did or didn’t review a decision. A fast review could be perfectly legitimate for a simple case, while a long review doesn’t automatically prove meaningful oversight. I use these signals together with the workflow evidence and interviews to determine whether the control is functioning as designed.

4. Introduce controlled errors

One of the strongest ways to test an oversight control is to give it something that should be caught.

In an authorised test environment, I can create or use test cases containing deliberate errors. The objective isn’t to trick an unsuspecting production reviewer or manipulate real customer decisions. The objective is to create a controlled scenario where the correct outcome is already known and then determine whether the oversight mechanism identifies the problem.

For example, suppose a test AI is designed to identify whether an application satisfies a particular documented criterion. I can create a synthetic test case where the model’s recommendation intentionally conflicts with the known evidence.

Then I watch the control.

Does the reviewer notice the inconsistency? Do they challenge the AI? Do they request additional information? Do they override the recommendation? Is the intervention recorded?

If the reviewer catches the error, I have evidence that the control can work.

If the reviewer accepts the incorrect recommendation without examination, I have evidence that the control may not provide the level of oversight the organisation claims.

This type of testing should always be conducted within the agreed assessment scope and with appropriate safeguards, particularly when the AI system affects real people, financial decisions, employment, healthcare, access to services, or other consequential outcomes.

5. Measure the override rate — but don’t stop there

The override rate is another useful signal.

Suppose an organisation tells me that reviewers independently evaluate every AI decision. I ask for the review statistics.

Imagine there have been 25,000 reviews and not a single documented override.

That immediately gets my attention.

But here’s an important assessment principle: zero overrides does not automatically prove that oversight is ineffective. It is a signal that requires investigation.

Maybe the AI genuinely produces highly reliable recommendations and reviewers independently confirm them. Maybe the review population is extremely simple. Or maybe reviewers have become conditioned to accept whatever the AI recommends.

So I ask for more evidence.

• How many decisions were reviewed?
• How many were escalated?
• How many were challenged?
• How many were returned for additional information?
• How many were overridden?
• How are reviewer disagreements recorded?
• Has the organisation ever tested the reviewers using known-error cases?

If the answer to all of these questions is effectively “none,” while the organisation claims that reviewers provide meaningful independent oversight, I have a much stronger finding than simply reporting a zero override rate.

6. Test for automation bias

There is another risk I specifically look for: automation bias.

This happens when people give excessive weight to an automated recommendation simply because it came from a system that appears objective, technical, or highly accurate.

The danger becomes particularly obvious when the AI presents a confident answer. A reviewer may start thinking, “The model has already analysed this, so there probably isn’t much for me to do.” Over time, the human reviewer becomes a confirmation mechanism rather than an independent control.

I test for this through interviews and controlled scenarios.

I’ll ask reviewers questions such as:

• “What would make you disagree with the AI?”
• “When are you expected to override it?”
• “What are the model’s known limitations?”
• “Show me an example where you disagreed with the model.”
• “What happens if you believe the AI recommendation is wrong?”

The answers tell me much more than the policy document does.

7. Check whether reviewers are capable of exercising oversight

Giving someone an override button doesn’t make them qualified to use it.

I therefore look at reviewer competence as part of the control. Do reviewers understand what the AI is designed to do? Do they understand its known limitations? Have they been trained on the circumstances that require escalation? Do they know how to interpret confidence information or supporting evidence where those are provided?

I also check whether the organisation has created incentives that undermine independent review.

For example, imagine reviewers are measured primarily on the number of cases they process per hour. The governance policy says, “Every AI decision must receive meaningful human review.” But the operational KPI rewards reviewers for processing cases as quickly as possible.

That creates a control-design problem. The organisation is asking the reviewer to perform careful oversight while simultaneously creating pressure to approve decisions quickly.

8. Test the override and escalation trail

When someone does challenge an AI decision, I want to be able to reconstruct what happened.

A good audit trail should allow me to answer questions such as:

• What did the AI originally recommend?
• What information did the reviewer see?
• Who reviewed the decision?
• When did the review happen?
• Did the reviewer accept, reject, modify, or escalate the recommendation?
• Why was the decision changed?
• What final decision was made?
• Was the case subsequently used to improve the AI or the governance process?

This becomes especially important during an incident investigation. If an organisation can’t reconstruct how an AI decision moved through the human oversight process, it becomes much harder to establish whether the control actually operated.

9. Don’t test one reviewer and call it representative

Another mistake I see in governance assessments is testing one friendly reviewer who knows the process and then extrapolating the result to the entire organisation.

Instead, I sample.

Depending on the engagement, I might sample different reviewers, teams, decision types, risk levels, business units, and time periods. I’m looking for consistency. If one team genuinely challenges AI decisions while another team approves almost everything, that difference itself is valuable evidence.

The sampling methodology should be documented so another assessor can understand how the conclusion was reached.

10. What strong and weak oversight evidence looks like

11. How I would write the finding

When I identify ineffective human oversight, I avoid writing a vague finding such as “Human oversight needs improvement.” That doesn’t tell the organisation what failed.

Instead, I document the control, the expected behaviour, the evidence I tested, the observed behaviour, and the resulting risk.

For example:

12. The question I want you to remember

When you’re assessing human oversight, don’t ask only, “Is there a human?”

Ask:

“Can this human understand the AI decision, recognise when it may be wrong, challenge it, override it, and prove that they did so?”

That is a much better security-testing question.

A policy can tell me that oversight exists. A workflow can show me where the reviewer sits. An interview can tell me what the reviewer believes they are supposed to do. But only operational evidence and controlled testing can give me confidence that the control actually works.

And that’s the larger lesson of AI governance testing: don’t test whether the control is documented. Test whether the control changes what happens when the AI gets something wrong.

Building the Compliance Gap Register

Once I’ve tested the governance controls, I need somewhere to put the results. This is where the compliance gap register becomes useful.

I don’t use the gap register as a dumping ground for everything that looks imperfect. I use it to answer a much more useful question: “What was the organisation expected to do, what evidence did I actually find, and what is the difference?”

That distinction is important. A missing document, an ineffective control, a technical vulnerability, and a regulatory requirement that doesn’t apply are four completely different things. If I put all of them into one generic “compliance gap” column, the register quickly becomes difficult to use.

A good gap register gives management a clear picture of where the AI governance system is working, where it is weak, what needs to be fixed first, who owns the fix, and how I will know when the gap has actually been closed.

1. What the gap register is actually for

Think of the gap register as the bridge between assessment evidence and remediation.

During the assessment, I might discover that an organisation has:

• an AI governance policy but no documented AI inventory;
• a risk assessment process but no evidence that it was applied to several production systems;
• a human oversight procedure but no evidence of meaningful reviewer intervention;
• security testing but no process for feeding the results into the AI risk register;
• supplier due diligence requirements but incomplete evidence for several AI vendors;
• an incident process but no AI-specific escalation criteria;
• excellent technical controls but weak management accountability.

Each of these represents a different type of gap. The register turns those observations into structured findings that someone can actually manage.

NIST’s AI RMF is particularly useful here because its Core connects Govern, Map, Measure and Manage, with governance operating across the other functions. NIST also emphasizes continuous risk management throughout the AI lifecycle.

2. A gap is not automatically a violation

This is one of the most important distinctions I want you to make when writing the register.

Suppose I find that an organisation doesn’t have a particular document.

I don’t immediately write:

First I ask whether that document is actually required for the system, role, use case and applicable provision being assessed.

The same principle applies to NIST AI RMF. NIST describes AI RMF 1.0 as a voluntary, use-case-agnostic framework, and its Playbook explicitly says that its suggested actions are not a mandatory checklist.

So my register distinguishes between:

• Regulatory requirement: an applicable legal or regulatory obligation.
• Standard requirement: a requirement being assessed against a standard such as ISO/IEC 42001.
• Framework outcome: an expected or useful outcome from NIST AI RMF.
• Organisational control: an internal policy or control the organisation has committed to implementing.
• Good-practice recommendation: an improvement that isn’t necessarily mandatory.

That classification prevents me from overstating findings.

3. The fields I put in the register

A practical compliance gap register should contain enough information for someone who wasn’t involved in the assessment to understand the finding.

4. Write the gap so another person can understand it

A weak gap statement says:

That tells the owner almost nothing.

A stronger statement says:

Now I know what failed, what I tested, and what evidence supports the conclusion.

5. Separate expected state from observed state

One technique that makes my findings much clearer is separating what should happen from what actually happened.

Expected: AI systems should undergo documented risk assessment before production deployment.

Observed: The organisation provided risk assessments for 7 of 10 sampled production systems. Three systems had deployment approval records but no corresponding AI risk assessment.

Gap: The pre-deployment AI risk-assessment control is not consistently applied across the sampled population.

Impact: AI systems may enter production without documented identification and treatment of relevant risks.

That structure makes the finding much harder to argue with because the reasoning is visible.

6. Map each gap to the right framework

The next step is cross-mapping.

I don’t want three separate registers for NIST, EU AI Act and ISO/IEC 42001 if the organisation is trying to manage one AI governance programme. Instead, I map a single finding to the applicable requirements.

The mapping should be treated as a crosswalk, not proof of equivalence. A NIST category isn’t automatically the same legal obligation as an EU AI Act article, and an ISO control isn’t automatically evidence of EU AI Act compliance.

The purpose of the crosswalk is to show where one remediation activity can address multiple governance objectives.

7. Prioritise the gaps instead of treating everything equally

A register with 80 open findings is not useful if everything is marked “High.”

I normally prioritise using several dimensions:

• Legal or regulatory exposure
• Potential harm to individuals
• Security impact
• Likelihood
• Number of affected systems or users
• Ability to detect the failure
• Whether the control failure is systemic
• Whether the gap blocks another critical control

For example, a missing AI policy template may be a relatively low-priority documentation issue. A production AI system making consequential decisions without effective human oversight is a completely different risk proposition.

NIST’s MANAGE function specifically calls for AI risks to be prioritised and responded to based on assessments and analytical outputs from MAP and MEASURE, including consideration of impact, likelihood, resources and available treatment options.

8. Don’t confuse severity with compliance importance

Another distinction I find useful is separating security severity from compliance significance.

A governance gap might not create an immediately exploitable vulnerability but could still be extremely important from a regulatory or accountability perspective.

For example, imagine the organisation has no documented process for determining who is responsible for an AI system after deployment.

There may be no CVSS score.

There may be no obvious exploit.

But when something goes wrong, nobody may know who has authority to suspend the system, investigate the incident, approve remediation, or communicate the decision.

That is a governance weakness with potentially significant downstream consequences.

9. Record the root cause, not just the symptom

Suppose I find five AI systems without current risk assessments.

I could create five separate findings.

But during the investigation I discover that the organisation has no process requiring the AI inventory to trigger a periodic risk reassessment.

Now I’ve found the underlying problem.

The five missing assessments are symptoms. The broken lifecycle process is the systemic gap.

I might still record the individual affected systems as evidence, but the remediation should address the process that allowed all five gaps to occur.

This is one of the most valuable things a compliance gap register can do: show patterns rather than just individual failures.

10. Example compliance gap register

11. Every gap needs a measurable remediation action

A surprisingly common mistake is to close a finding with a statement such as:

That’s not remediation. That’s a promise.

I want the corrective action to be testable.

Instead of:

I would write:

Now I can come back later and verify whether the gap is actually closed.

12. Closing a gap requires retesting

This is where the gap register connects directly to security-testing discipline.

I don’t mark a finding “Closed” simply because somebody sends me an updated policy.

I ask:

“What evidence proves the control now works?”

Suppose the original finding was that human reviewers were not demonstrating meaningful oversight.

The organisation updates the procedure and provides a new training document.

That’s useful, but it doesn’t close the finding by itself.

I want to see the new process operating. I might sample decisions, inspect review records, interview reviewers, and use an authorised controlled test case to determine whether the revised process actually catches an AI error.

Only then can I confidently move the finding toward closure.

13. The gap lifecycle

I normally think of every finding as moving through a simple lifecycle:

Identify → Validate → Record → Prioritise → Assign → Remediate → Retest → Close

The important word is Validate.

Before I put something into the register, I make sure I’ve got enough evidence to support it. That prevents the register from becoming a collection of assumptions.

The other important word is Retest.

Without retesting, the register tells me what the organisation promised to fix. With retesting, it can tell me what the organisation actually fixed.

14. Turn the register into a management dashboard

Once the register contains enough structured information, I can extract useful management metrics from it.

• Total open AI governance gaps
• Critical and high-priority gaps
• Gaps past their target date
• Gaps by business owner
• Gaps by AI system
• Gaps by NIST function
• Gaps by regulatory requirement
• Gaps by ISO/IEC 42001 control area
• Average remediation age
• Percentage of findings successfully closed after retesting
• Recurring findings

That last metric is particularly valuable.

If the same category of governance gap keeps appearing after remediation, the organisation may be treating symptoms rather than fixing the underlying process.

15. One remediation can close several gaps

The cross-framework view can also make remediation much more efficient.

Imagine I identify a weakness in AI risk assessment.

That one weakness may affect NIST GOVERN and MAP outcomes, an applicable EU AI Act risk-management requirement, and the organisation’s ISO/IEC 42001 management-system processes.

I don’t want three separate remediation projects producing three different spreadsheets.

I want one properly designed AI risk-management process that produces the evidence needed across the relevant frameworks.

This is one of the major advantages of building the register around controls and evidence rather than around framework names.

16. The test I use before closing the register

At the end of the assessment, I go through each finding and ask five questions:

1. What requirement are we testing?
2. What evidence did we examine?
3. What exactly is missing or ineffective?
4. What risk does that create?
5. What evidence will prove the gap is closed?

If I can’t answer one of those questions, the finding probably isn’t ready.

And that is the mindset I want you to take into your own AI governance assessments. Don’t build a register that merely lists what the organisation doesn’t have. Build a register that shows what the organisation needs to change, why the change matters, who owns it, and what evidence will prove that the control finally works.

That’s what turns a compliance gap register from an audit spreadsheet into an actual AI risk-management tool.

Governance Assessment Deliverables

By the time I finish an AI governance assessment, I don’t want to hand everyone the same 80-page report and expect them to figure out what matters to them. Security engineers, compliance teams, and senior leadership need different levels of detail because they make different decisions.

For a typical engagement, I therefore separate the results into three layers: technical findings, the compliance gap register, and an executive or board-level summary.

These aren’t three unrelated reports. They are three views of the same evidence.

The technical report explains what is technically wrong and how to fix it. The compliance register explains which governance or regulatory expectations are not being met and how the gaps should be remediated. The executive summary explains what leadership needs to know and what decisions need to be made.

1. Technical Findings Report — for security and engineering

The first deliverable is the technical findings report. This is where I put the technical evidence from the assessment.

If I’ve discovered prompt injection, insecure agent permissions, data leakage, weak authentication, exposed APIs, inadequate logging, insecure model configuration, supply-chain weaknesses, or other technical issues, this is where I document them.

I want an engineer reading the finding to be able to understand the problem and reproduce it without having to interpret regulatory language.

A typical technical finding contains:

• Finding ID: A unique identifier such as AI-SEC-001.
• Title: A short description of the vulnerability.
• Severity: Critical, High, Medium, Low, or the agreed scoring system.
• Affected component: The model, API, agent, RAG pipeline, application, infrastructure or integration involved.
• Description: What I discovered.
• Attack or test method: How I validated the issue within the authorised scope.
• Evidence: Logs, screenshots, requests, responses, test cases or other supporting material.
• Impact: What an attacker or failure could achieve.
• Remediation: What engineering or security teams should change.
• Retest criteria: What I will test to determine whether the vulnerability has actually been fixed.

For example, I don’t want to write:

That’s too vague.

I want the engineering team to know exactly what happened, what boundary was crossed, what data or capability became accessible, why the existing control failed, and what remediation will prevent the same attack path.

2. Connect technical findings to governance

Here’s where Day 39 becomes different from a normal penetration test.

I don’t throw the technical report over the wall and walk away.

A technical finding can also reveal a governance failure.

Suppose I discover that an AI agent has excessive permissions and can access a sensitive internal system.

The technical report tells engineering how to fix the excessive permissions.

But I then ask governance questions:

• Was the agent’s access documented?
• Was the permission included in the AI threat model?
• Who approved the access?
• Was least privilege part of the AI security standard?
• Was the risk recorded in the AI risk register?
• Was the system assessed before production?

If the technical control failed because the organisation never established a process for reviewing AI agent permissions, the technical vulnerability is only part of the story.

That’s the governance finding.

3. Compliance Gap Register — for compliance, risk and legal

The second deliverable is the compliance gap register.

This is deliberately different from the technical findings report. The compliance team doesn’t necessarily need a 15-step reproduction of a prompt-injection attack. They need to understand the governance requirement, the evidence examined, the gap, the potential impact, ownership, and remediation status.

For each gap, I would normally capture:

I also map the finding to the relevant framework where appropriate. For example, one AI risk-management weakness might map to NIST AI RMF, an applicable EU AI Act requirement, and an ISO/IEC 42001 process.

But I keep one rule in mind: a framework mapping does not automatically prove a legal violation. The register should clearly distinguish mandatory legal requirements from voluntary frameworks, standards, contractual obligations, and recommended good practice.

4. Be careful with “fine exposure”

This is an area where I would be particularly careful in the report.

I don’t put a large regulatory fine number beside every gap and call that the organisation’s “exposure.” A potential penalty depends on the applicable provision, the organisation’s role, the facts of the violation, jurisdiction, enforcement circumstances, and other factors.

Instead, I distinguish between:

• Applicable requirement
• Current applicability date
• Observed compliance gap
• Potential enforcement consequence
• Business and operational impact
• Legal interpretation requiring counsel

That keeps the assessment evidence-based rather than turning the security report into a legal opinion.

5. Board-Level Summary — for leadership

The third deliverable is the one most likely to be read in a few minutes rather than a few hours.

Senior leadership usually doesn’t need to know exactly which API request demonstrated the vulnerability. They need to understand the organisation’s overall exposure, the most important weaknesses, what could happen if nothing changes, and what decisions need to be made.

My executive summary therefore answers five questions:

1. What did we assess?
2. What did we find?
3. How significant are the findings?
4. What is the highest-priority risk?
5. What should leadership do next?

I try to make the first paragraph understandable without any technical background.

That’s much more useful to a board than a paragraph full of NIST subcategories and ISO clause numbers.

6. Give leadership the aggregate risk picture

I like to give leadership a one-page view of the assessment.

The numbers themselves are only useful if the methodology behind them is clear. I always tell the reader what population was assessed, what sampling was used, and what the severity scale means.

7. Give leadership a 30-day action plan

One of the most useful things I can put in the executive summary is a short action plan.

I don’t give leadership 37 recommendations and call that prioritisation.

I identify the small number of actions that will reduce the most significant risk first.

The exact timeline depends on the risk and engagement, of course. The point is to convert the assessment from a static report into an actionable remediation programme.

8. Keep an evidence chain between all three deliverables

This is probably the most important quality-control step.

Every major statement in the executive summary should be traceable to findings. Every compliance gap should be traceable to evidence. Every technical finding should have reproducible evidence.

Think of the chain like this:

Test → Evidence → Finding → Governance Gap → Risk → Remediation → Retest → Executive Summary

For example:

Test: Controlled prompt-injection assessment.

Evidence: The authorised test demonstrated that an attacker could cause the application to expose information outside the intended workflow.

Technical finding: AI application has insufficient controls against indirect prompt injection.

Governance gap: The organisation’s AI threat-model process did not identify or assess prompt-injection risk for the application.

Risk: The technical weakness was not adequately represented in the organisation’s AI risk-management process.

Remediation: Implement appropriate technical mitigations and update the AI threat-model and risk-assessment process.

Retest: Repeat the authorised attack scenarios and verify the governance process records the risk and its treatment.

Executive message: A material AI security weakness was identified and was not adequately captured in the existing governance process.

Now everyone is looking at the same underlying evidence from a different perspective.

9. What makes a good deliverable?

Before I send the final assessment, I run a simple quality check.

• Accurate: Every finding is supported by evidence.
• Reproducible: Technical findings can be independently validated.
• Specific: Findings describe an actual control weakness rather than vague concerns.
• Scoped: Conclusions don’t exceed the systems, evidence and period assessed.
• Traceable: Findings can be traced back to tests and evidence.
• Actionable: Remediation tells the owner what needs to change.
• Measurable: Closure criteria are defined.
• Prioritised: Leadership can distinguish urgent risks from lower-priority improvements.
• Legally careful: Regulatory conclusions are limited to what the evidence and engagement scope support.

10. The final test: could someone act on it?

This is the question I use before I consider the assessment finished:

“If the organisation gives this report to the people responsible for fixing the problems tomorrow morning, do they know what to do?”

If engineering knows which technical controls need to change, the technical report has done its job.

If compliance knows which governance requirements need attention, who owns each gap, and what evidence is required for closure, the gap register has done its job.

If leadership understands the overall risk, the most important unresolved issue, and the immediate decisions required, the executive summary has done its job.

That’s how I think about governance assessment deliverables. I’m not producing three versions of the same report. I’m translating the same assessment evidence into three different decision-making languages: engineering, compliance, and leadership.