Compliance across a large code estate is usually assessed by sampling, because assessing all of it costs too much. You pick a dozen representative repositories, audit those properly, and extrapolate. Everyone involved knows the extrapolation is doing a lot of work.
In a recent engagement I needed a compliance position across more than 300 repositories. Sampling was not going to be enough, because the question being asked was not "is the estate broadly in order" but "which repositories fall short, and on what". That is a per-repository question and it does not average.
So I ran it with Claude Code.
How it actually ran
The mechanism was unglamorous. A scripted loop over the Azure CLI to clone each repository in turn, then a fixed set of checks against each one: documentation present and current, specific code and configuration requirements met, evidence of the controls the compliance framework expected. Where a check passed, record it. Where it failed, record the gap. Where the documentation was missing but the underlying control was clearly in place, draft the documentation that should have existed.
That last part is where most of the value sat. A large fraction of compliance gaps in a mature estate are not missing controls. They are controls nobody wrote down. Drafting that evidence is tedious, highly patterned work, and it is exactly what a model is good at.
My estimate is that a careful human review would have run to something like two days per repository. The automated pass took about half an hour per repository including the clone. I want to be clear that this is my estimate of the manual alternative rather than a measured comparison, because nobody was ever going to fund the manual version to find out. The honest claim is narrower: the work happened at all, and it would not have otherwise.
What it found that I was not looking for
The compliance checks were the brief. The findings that mattered most were not on the list.
Credentials committed into repositories. Not in the current head of every case, which is the version people check, but present in the code estate.
And the one that stopped the exercise for a day: a substantial volume of personal data sitting in test fixtures. It had every appearance of synthetic test data. It was real client information.
That second finding is worth sitting with, because of how it got there. Nobody made a decision to put production personal data into a test fixture. Someone needed realistic data to test against, took a slice of what was available, and committed it. The person who wrote that test may well have left years ago. Every review since has looked at that file and seen test data, because that is what it is labelled as and that is what it looks like.
A sampling approach finds this only by luck. A full pass finds it because it reads every fixture in the estate and has no prior assumption about which ones are interesting.
Where the automation stopped
The pass could not close everything. Plenty of compliance questions are not answerable from a repository at all: why a control was designed a particular way, who owns a decision, whether a documented process is the one actually followed.
For those I ran interview-style sessions with the development teams, the compliance function, and the product owners. Recorded, transcribed, then the transcripts used as evidence against the outstanding items. That is not a workaround for a limitation in the tooling. Those answers only exist in people's heads, and no amount of static analysis was going to extract them.
The shape that worked was automation for everything derivable from the code, conversation for everything that was not, and a clear line between the two. Getting that line wrong in either direction is expensive: automate what needs a conversation and you get confident nonsense, interview what the code already answers and you waste everyone's time.
The guardrails
Two things made this safe enough to run.
The checks were fixed and written down before the first clone, so the model was answering a defined question rather than forming an opinion about what compliance meant. A model asked to "review this repository for compliance concerns" will produce something plausible for every repository, including the compliant ones.
And every finding carried its evidence: file, line, the specific check it failed. A finding I could not verify in ten seconds was treated as a lead rather than a result. That mattered more than it sounds, because the volume was high enough that an unverifiable finding rate of even a few percent would have buried the real ones.
What this changes
The reason estate-wide compliance review gets sampled is cost, and that cost has moved. Not to zero, and not without the human stages either side, but far enough that the full pass is now a reasonable thing to propose rather than something you talk a client out of.
The second-order effect is the one I keep thinking about. When a full pass is affordable, it can be repeated. A compliance position from a sampled audit is a statement about a point in time that starts decaying immediately. A position you can regenerate is closer to a property of the estate than a snapshot of it.
I have run this once, at one organisation, against one compliance framework. I would want to run it two or three more times before claiming the approach generalises. But the finding in the test fixtures would have justified the exercise on its own, and it had nothing to do with the framework we were assessing against.
If you are sitting on a code estate large enough that nobody has read all of it, I am happy to compare notes on how to approach it.
