Skip to main content
← All posts

I'm in a regulated industry - how can I remain compliant and still have autonomous agent delivery?

Modulo Team

governanceauditcompliance

Regulated teams hear "autonomous agent delivery" and assume it means skipping the controls their auditor already tests for. It doesn't. SOC 2, HIPAA, SR 11-7, FedRAMP, and the NAIC's AI model bulletin all ask the same question in different words: can you prove who approved this and what it touched? None of them ask whether the approver read every single line.

What catches the bad change?

What stops the broken change from reaching production? In any mature engineering organization, the answer is a verification layer that runs before anything ships - a system of deterministic checks that catch regressions, not a person reading every line.

SOC 2's change-management criterion, CC8.1, requires that a change be authorized, documented, tested, and approved before it reaches production. It does not specify how much of the change the approver must read. What an auditor samples is an offline-verifiable evidence chain: was the change authorized, is there a record of testing, is there a record of approval, does the record match what shipped.

The domain-specific layers ask for the same shape of thing:

  • Financial services. SR 11-7 wants a record of which model produced a change and who signed off on it; DORA and PCI DSS ask the equivalent for operational resilience and cardholder data. See agent governance for financial services.
  • Healthcare. HIPAA obligations land on covered entities and their business associates. What an auditor looks for is access control, an audit record, data location, and a retention policy. See agent governance for healthcare.
  • Government and defense. FedRAMP and the IL levels assess the deployment: where the data sits, where the network boundary is, who holds the keys. See agent governance for government and defense.
  • Insurance. The NAIC's Model Bulletin on the use of AI systems, adopted by 24 states and the District of Columbia as of April 2026, requires a written AI Systems Program with senior-management and board accountability, risk controls, and model documentation.

Every framework asks for an evidenced, authorized, documented process.

How teams actually make it safe

Duckbill Group faced 60 open pull requests for five people and asked "what if we just didn't review the PRs." They raised guardrails first: nearly every ruff, prettier, and eslint rule enabled; a unit-test coverage floor of 85%; post-deploy observability; stricter type checking. Then they defined where human review still applies: changes touching the public API, authentication, the design system, non-additive database schema changes, or agent skills. Everything else merges unread. Weekly PR throughput nearly doubled (80 to 154).

OpenAI's Harness engineering team has pushed almost all review effort towards agent-to-agent review: Codex reviews its own changes locally, requests additional agent reviews, and iterates until all agent reviewers are satisfied. Humans may review pull requests but aren't required to. Verification runs on linters, CI jobs, and structured knowledge-base checks.

Cursor publishes AIUC-1, ISO/IEC 27001, and ISO/IEC 42001 certifications, along with a SOC 2 Type II attestation. At least one of its own engineers runs agents that auto-merge straight to Cursor's product, reviewed after landing on main rather than before - a workflow built on an eval-tested verification skill where the change passes deterministic and automated checks before it moves.

Nvidia's Jensen Huang has said publicly that every one of his software engineers would rather build agents than write code, and that the mundane work is what goes to the agent. In his account, Nvidia's engineers now spend their time building agentic systems, writing evaluation frameworks, and designing the guardrails that keep those systems operating safely.

Accountability stays human

Shopify CEO Tobi Lütke has said one in eight merged pull requests across Shopify is coauthored by River, its internal AI agent, which reads code, runs tests, and opens PRs directly in the channel. He has also warned publicly about what he calls "slop grenades" - unreviewed AI output an employee forwards without reading it closely, dumping the verification work on whoever catches it next.

Lütke's central claim is that machines cannot take responsibility - only a human can be fired, sued, or jailed. What moves is the reading of every line, not the accountability. The failure he describes is unstructured autonomy with no boundary and no trail. The fix is a deliberate, documented, evidenced boundary on where review applies and where it doesn't. Modulo's HITL gates enforce exactly that: a human-only gate cannot be claimed by an agent, a reviewer claims it with a short-lived token, and the decision lands in the audit chain. See Human-in-the-Loop.

Where the evidence is already running

Finance

Goldman Sachs is piloting Cognition's Devin across its 12,000-person engineering organization, scaling from hundreds toward thousands of instances with human review in the loop.

Commonwealth Bank of Australia's chief AI officer reports that over 80% of CBA's 7,800 engineers use AI coding agents, delivering three times more code changes last year and resolving over 15,000 quality and security issues. "We are a regulated industry," he says. "When it comes to making the final decision, we absolutely would want to retain a human in that."

Sony Bank and Fujitsu have made generative AI part of the standard development process for Sony Bank's core banking system, which is in live operation. They report a 40% reduction in man-hours, a 30% shorter development period, and a 99% source code generation rate. Each AI agent operates under human judgment, approval, and quality assurance.

Monzo has built an internal agent ecosystem that now authors roughly 10% of all merged PRs and runs more than 1,800 tasks every day. "We're a bank, and observability and control is paramount," the engineering team writes. Agents propose fixes in plain English and open pull requests, and an engineer reviews and merges every one.

Paisabazaar's CTO reports that almost 86% of the company's new code is now AI-generated, and production builds have risen from roughly 30 a month to more than 150. AI touches core systems within ring-fenced boundaries the team defines. Generated code cannot move directly into production: a developer reviews and commits it first, and it passes through existing code review, automated testing, QA, security checks, and user acceptance testing.

Government and defense

Cognition states that Windsurf holds FedRAMP High authorization and DoD Impact Level 5 compliance through Palantir's FedStart program. GitHub states that Copilot's underlying model infrastructure is FedRAMP Moderate authorized for US government customers as of April 2026, with full Copilot authorization in progress.

Where the evidence is thinner

Healthcare and insurance lag finance and government. Adoption there is still mostly at the copilot-assistant stage, even though the regulatory door is open.

What auditors actually ask

An auditor's interpretation of CC8.1 or segregation of duties can differ from the control text. Some accept automated verification; others want a human sign-off. The practical requirement is an offline-verifiable evidence chain strong enough to survive that conversation: who authorized the change, what tests ran, what gates fired, what the approver saw. A system that produces that evidence gives an auditor something concrete to evaluate.

Where that record comes from decides how much it is worth. A chain generated by the platform that ran the work is a platform's own event log, checked only by asking that platform. Modulo's record is hash-chained, exportable, and stored in your own self-hosted database, so the check does not depend on the vendor.

How Modulo helps

Every state-changing action appends an immutable, hash-chained audit event, verifiable on demand. Each run holds a versioned snapshot: prompt, output, schema result, model backend, and every gate decision. Because Modulo is self-hosted, all of it writes to your own Postgres database. See audit trail for AI-generated code and audit as a first-class output.

Typed contracts validate against JSON Schema before anything moves. Eval gates score output against deterministic criteria. A gate that never fires can be raised to auto-pass after your team watches it catch nothing - the audit trail SR 11-7, the NAIC bulletin, and SOC 2 CC8.1 are built to receive. See Agent Governance for Regulated Engineering Teams.

Audit exports, SSO/SAML, RBAC, and spend limits are on the paid Team tier; the Community tier keeps the same hash-chained audit trail. The record is generated the moment the work happens, not reconstructed after the fact.