AI Compliance Monitoring
Tested and fine has a shelf life.
What is compliance monitoring?
AI compliance monitoring means running repeated checks against your own written rules and dating the results, so that drift between what you documented and what actually happens becomes visible rather than being discovered at audit.
Compliance decays in three ways. Somebody changes the system, somebody skips a step under pressure, or the provider updates the model and your outputs change without anything inside your organisation having moved.
The third one is the one nobody watches. You approved a system that behaved a particular way, and the version answering your customers this month is not the version you approved. No announcement, no change request, no record.
That is why a sign off has a shelf life. Monitoring is not repeating the assessment, it is checking a small number of things often enough that drift shows up while it is still a correction rather than an incident.
When we tell you to skip the automation
Duara sets up compliance monitoring for AI systems in Kenya, checking against your own rules rather than a generic checklist.
We will tell you when a quarterly manual check is enough. Automated monitoring is worth paying for when something changes frequently, and selling it for a system that barely moves is selling you a subscription.
Who this is for
- Policies exist but nobody checks compliance
- Evidence is assembled in a panic before audits
- You rely on AI providers who update their models
- AI use is spreading across departments
Industries we serve
- Banking and financial services
- Insurance
- Healthcare
- Telecommunications
- Public sector
- Any regulated organisation using AI
Problems this solves
These are the ways a system stops matching what you wrote about it.
- The provider changes the model. Outputs shift on identical inputs with no announcement, and the version in use is no longer the version anybody approved.
- Steps skipped under pressure. A busy month removes the review step first, and the process quietly becomes something nobody documented.
- Evidence created afterwards. A pack assembled the week before an audit is a claim, and a check dated the day it ran is evidence.
- Generic checks. Monitoring products arrive with default rules that measure somebody else policy rather than the one you wrote.
What we deliver
A small number of checks against your own rules, run often enough to catch drift, dated as they run, with alerts only where somebody will act.
Checks written against your own rules
Results dated as they run
Detection of provider side model changes
Alerts limited to things somebody will act on
Evidence packs produced on request
A stated list of what monitoring does not cover
Features
- Provider changes detected. Watching for output shifts on unchanged inputs, since the model you approved and the model answering today are not always the same.
- Checks from your rules. Written against your policy rather than a vendor default, because a default check measures somebody else organisation.
- Evidence dated as it happens. A result recorded the day it ran carries weight that a spreadsheet compiled before an audit does not.
- Few enough alerts to read. Alerting on everything trains people to ignore alerts, which is worse than not alerting at all.
- Gaps stated openly. What monitoring does not cover is written down, since an unstated gap becomes a false sense of coverage.
Benefits
- Drift caught while small. A change found in a week is a correction, and the same change found at audit is a finding.
- Audits stop being a scramble. Evidence that accumulated as you went is already assembled when somebody asks for it.
- You notice what the provider did. Output changes you did not cause become visible instead of being explained away as unusual cases.
- Alerts get taken seriously. A small number of meaningful alerts produces action, which a constant stream never does.
What monitoring actually checks
- Output stability. Whether the same inputs still produce comparable results, which is how a provider side change becomes visible.
- Process conformance. Whether steps you documented are still happening, particularly the ones that get skipped under pressure.
- Data handling checks. Whether systems are still holding only what your policy permits and for no longer.
- Access review. Who can reach these systems now, since permissions accumulate quietly over time.
- Dated evidence records. Results recorded when they run, which is what separates evidence from an assertion.
- Coverage statement. A written account of what is not being checked, so nobody mistakes the monitoring for completeness.
Monitoring measures conformance to the rules you wrote. It cannot tell you the rules are right, so a badly chosen rule produces a reassuring record of doing the wrong thing consistently.
How the process works, step by step
Setting up monitoring takes two to four weeks. The useful question afterwards is whether anybody reads the results.
-
Step 1
Free review call
We look at what rules exist and whether anything currently checks them.
-
Step 2
Checks chosen sparingly
A small number tied to rules that matter, since a long list produces alerts nobody reads.
-
Step 3
Baselines recorded
How systems behave now, which is what later comparison depends on.
-
Step 4
Monitoring runs and dates results
Checks executed on a schedule with results recorded as they happen rather than reconstructed later.
-
Step 5
Gaps written down
What is not covered, stated plainly so nobody assumes the monitoring is complete.
Quality assurance and how we work
The commitments attached to telling you your systems have drifted.
- We say when manual checks are enough. Automated monitoring earns its cost where things change often, and selling it for a static system is selling a subscription.
- We only alert where somebody will act. An alert nobody acts on teaches people to ignore alerts, which leaves you worse off than before.
- We state what is not covered. An unstated gap becomes a false sense of coverage, which is more dangerous than a known blind spot.
- We do not claim monitoring validates your rules. It measures conformance. A badly chosen rule produces a reassuring record of doing the wrong thing consistently.
- We date evidence as it is produced. Records created after the fact carry far less weight, and everybody reviewing them knows it.
Frequently asked questions about compliance monitoring
How does compliance drift without us changing anything?
The provider updates the model. Outputs shift on identical inputs, with no announcement and no version anybody in your organisation approved.
Can we stop providers changing models?
Rarely on standard terms. What you can do is notice, which is why output stability is checked rather than assumed.
How would we detect that?
By running known inputs regularly and comparing results against a recorded baseline. Without a baseline there is nothing to compare to.
Is monitoring the same as reassessing?
No. A reassessment repeats the whole exercise. Monitoring checks a few things often enough that drift appears while it is still small.
Why not check everything?
Because alerting on everything trains people to ignore alerts, and monitoring that nobody reads is an expense rather than a control.
What if we already have monitoring tools?
They usually arrive with default checks measuring somebody else policy. The checks have to come from your rules to mean anything.
Does this prove our rules are correct?
No, and that is the honest limit. Monitoring measures conformance, so a badly chosen rule produces a record of doing the wrong thing consistently.
Why does evidence dating matter?
A result recorded the day it ran is evidence. A pack assembled the week before an audit is a claim, and reviewers treat them differently.
What gets checked most often?
Output stability, whether documented steps still happen, what data systems hold, and who has access now.
Why check access?
Because permissions accumulate quietly. People change roles and keep what they had, and nobody notices until somebody looks.
Would you tell us we do not need this?
Yes, where systems barely change. A quarterly manual check is enough for a static setup, and we would rather say so.
How quickly does drift appear?
Sooner than most organisations expect, particularly where a busy period removes a review step that then never comes back.
What happens when a check fails?
Somebody named gets told and acts. A failing check with no owner produces a record of a problem rather than a fix.
Can this produce audit evidence?
Yes, and that is the practical benefit. Evidence that accumulated as you went is already assembled when somebody asks.
When would the first checks run?
Two to four weeks from starting. The harder question is whether anybody reads the results, which is a decision rather than a build.
Does it work with systems we bought?
Yes, and those are the ones that most need it, since you control neither the model nor when it changes.
Can our own team run it?
Yes. It is deliberately small enough to be run internally, and we would rather that than a dependency on us.
What is not covered?
Whatever we tell you is not covered, in writing. An unstated gap becomes a false sense of coverage.
Who can see the monitoring results?
Whoever you decide. The results describe your own weaknesses, so access is usually kept narrow.
What does monitoring cost?
The number of checks and how often they run determine it, with setup and ongoing cost quoted separately.
Glossary of terms
Monitoring terms in plain language.
- Drift
- The gap that opens between what you documented and what actually happens.
- Provider side change
- A model updated by its vendor, which alters your outputs without anything in your organisation changing.
- Baseline
- A record of how a system behaves now, without which later comparison is impossible.
- Dated evidence
- A result recorded when the check ran, as distinct from a pack assembled before an audit.
- Alert fatigue
- What happens when there are too many alerts to read, after which none of them work.
- Coverage gap
- Something monitoring does not check, which is only safe when it is written down.
Related services in AI Compliance
Talk to us about compliance monitoring
Run last year test inputs through your AI system today and compare the answers. Any difference is drift nobody recorded.
Start your project