Back to Guides
Enterprise Software 14 min read

Rethinking AI governance for companies without a compliance function

Rethinking AI governance for companies without a compliance function

Every guide to AI governance for small companies tells you to build a smaller version of what a large company has. The problem with that advice is that the large version does not work. In a 2026 survey of board members and senior leaders at organizations above a billion dollars in revenue, 98% reported formal AI governance policies and 47% said their organization had not applied that process to urgent deployments.

So the question is not how to imitate a compliance function with a spreadsheet. It is what governance looks like with no enforcement body to appeal to, no legal review queue, and nobody whose full-time job is to say no. That is a different design problem, and it has a better answer.

What the enterprise playbook assumes

Read the first page of results and the shape is uniform. Build an AI inventory. Tier your use cases by risk. Write an acceptable-use policy. Name an owner. Vet your vendors. Review quarterly. Six steps, repeated almost word for word by managed-IT firms, governance vendors, and consultancies.

None of it is wrong, and you should do all of it. But look at what the advice assumes. One widely syndicated guide tells a small business to stand up a cross-functional AI ethics and compliance committee with leadership, technology, legal, and risk representatives. A governance vendor whose page is titled for small business asks you to name four accountable owners per AI system. Both are written for a reader with a legal department. You are reading them because you have none.

The deeper assumption is the one worth attacking. Each guide treats a compliance function as the missing ingredient, so the prescription is a miniature of it: a committee instead of a department, a spreadsheet instead of a platform, a quarterly afternoon instead of a standing team. That only makes sense if the full-size version works.

The 47% that should change your plan

It does not, and the numbers are specific enough to plan around.

EY released an AI risk and governance survey in September 2026 covering 202 board members, C-suite executives, and VP-level leaders with oversight of AI systems, all at organizations above a billion dollars in revenue. Nearly all, 98%, had formal AI governance policies. Just under half, 47%, said their organization had previously not applied that process to urgent deployments. Separately, 36% had already had an AI incident with a materially negative business impact.

Grant Thornton’s 2026 AI Impact Survey, fielded across 950 C-suite and senior leaders in February and March 2026, found the matching result from the other direction: 78% lacked strong confidence they could pass an independent AI governance audit within 90 days, and 48% of boards had not set AI governance expectations. Its diagnosis of the mechanism is the part a smaller company should read twice. Centralized review bodies are becoming bottlenecks as use cases multiply.

That failure is structural rather than lazy. A review body creates a queue, a queue creates delay, and delay under deadline pressure creates a route around the queue, used by the people with the most urgent work and the most standing. A committee does not add control; it adds a thing to bypass. At 50 to 500 people the bypass is faster, because the person who would say no sits twelve feet away and has four other jobs.

So the constraint reframes. Having no compliance function means you cannot build controls that depend on an enforcement body. It also means you have nowhere for governance to hide. In the same data, 7% of companies still piloting AI were very confident of passing that audit against 74% of those running AI fully integrated, which puts the confidence in operating practice rather than documentation.

Three jobs, not one document

Here is why “write a one-page policy” produces a document nobody follows. A compliance function does three separate jobs, and the policy is the output of only one.

Job What it answers Who holds it at 50–500 people Failure mode when vacant
Interpretation Which rules reach us, in what role One named person, plus a few counsel hours a year The team splits into “everything applies” and “nothing applies”
Control design What we will and will not do The same person, drafting with the teams doing the work A policy written for a reader who does not exist
Assurance How we show the controls hold Whoever answers client security questionnaires Questions arrive mid-deal, answered by improvisation

Most programs at this size assign the second job and leave the first and third vacant. With nobody on interpretation, half the team believes the EU AI Act bans what they are doing and half believes no law touches them. With nobody on assurance, the first enterprise questionnaire with an AI section becomes a fire drill answered by someone guessing. Assigning all three to one person is fine; leaving one unassigned is what breaks.

Which rules actually reach you

Settle interpretation before writing anything, because most of what you will read overstates your exposure.

The EU AI Act scopes by role, not headcount. Regulation (EU) 2024/1689 puts the heavy duties on providers of high-risk systems: Article 16(a) assigns the risk-management obligations of Article 9, and Article 3(2) defines a provider as whoever develops a system, or has one developed, and places it on the market under their own name. Duties on deployers sit in Article 26 and are far narrower. Buy seats in commercial assistants and use them on your own work, and you are a deployer. Two routes promote you to provider by accident: fine-tuning a model and putting it in front of customers, or white-labeling someone else’s system. Regulation (EU) 2026/1744, the Digital Omnibus on AI, in force since 27 July 2026, also deferred the Annex III obligations to 2 December 2027.

The provision that reaches a pure user is Article 4, applicable since 2 February 2025, which asks for AI literacy among staff operating these systems; the Digital Omnibus softened it to require measures rather than a demonstrated level. For a US company with no EU establishment and no EU customers, none of this binds you. Read it anyway, as the clearest published account of what a reasonable program contains.

US law already covers most of the harm. Employment discrimination law applies to resume screening whether a model does it or a person does, Section 5 of the FTC Act covers deceptive claims about what your AI does, and state privacy regimes now reach automated decision-making. No new statute is needed to be sued under an old one.

The voluntary frameworks are reference material. NIST’s AI Risk Management Framework, published as AI 100-1 in January 2023, is explicitly voluntary, and its subcategory Govern 2.2 names personnel training as a control in its own right. ISO/IEC 42001 is certifiable, which matters only when a contract demands the certificate.

Write your conclusion in two sentences and date it. That is the interpretation job, done.

Five controls that survive without an enforcer

A control survives at this size if it holds when nobody is watching and nobody can be overruled. Five qualify.

1. An inventory you keep by asking, not by scanning. List every AI product and feature in use, who owns it, what data goes in, and whether it touches a customer or an employment decision. Discovery is the hard part: the Microsoft and LinkedIn 2024 Work Trend Index found 78% of AI users bringing their own tools to work, across 31,000 respondents. No procurement report surfaces those. You find them by asking in a way that carries no penalty, which means sanctioning tools rather than banning them.

2. A data rule in three tiers, not five. Open, internal, restricted. Five tiers is a project; three is a sentence people recall when they need it. The rule has to name specifics. “No client deliverables in draft” is usable. “Handle confidential information appropriately” is not.

3. One named owner, never a committee. One person for the page, one person per material risk. The failure here is vacancy rather than conflict: operations assumes IT owns it, IT assumes legal does, and there is no legal. A single name, usually the COO or chief of staff, costs an hour or two a week and removes the queue people route around.

4. A sanctioned-tool list with a fast path onto it. The list is a control only if getting on it takes days rather than a quarter. A slow path is a shadow-tool factory, the bottleneck logic Grant Thornton found at large companies reproduced in miniature.

5. One short artifact that answers what you will be asked. Not for an auditor. For four readers: a client’s security questionnaire, an insurance renewal form, a prospect’s procurement team, and your own staff. A one-page list of what could go wrong, who owns each item, how you would find out, and what the first hour looks like covers all four. The six-column, nine-row version of that page is the full specification.

Note what is absent: a review board, bias testing of models you did not build, an approval workflow, a platform. Those need the function you lack.

The control that is usually a signature

Nearly every source lands on human oversight as the highest-value control at this size. It costs nothing to write down, needs no software, and covers the failure that happens: fluent output that is wrong, reaching a client.

Then they stop, and the gap they leave is the most expensive thing on the page. A review step is a control only if the reviewer can tell a right answer from a confident wrong one. Put a junior analyst in front of a model’s summary of a contract they have not read and you have added a signature, not a check. Ask someone to confirm figures without telling them which part of the output fabricates, and they will confirm it. The control shows green and transmits nothing.

This is where the program becomes an AI enablement problem rather than a drafting problem. Every control above reduces to a judgment made in the moment: what not to paste, when to escalate, whether this output is checkable at all, which claim needs a source. None of those arrive with the license. They are learned by doing the work on your own documents with someone who can say that one would have gone out wrong, and here is how you would have caught it.

That is the gap between having a policy and having the practice it describes, and the reason NIST put personnel training inside the Govern function. A policy nobody has been trained against is a record of intent. The capability is the control.

What to refuse

Refusals are where a program this size gets its force. Decide them on purpose.

  • An AI governance committee. Review bodies become bottlenecks, and bottlenecks get bypassed. A working group that meets when something changes does the useful part without the queue.
  • A forty-page policy. Length is a proxy for nobody having had to read it. The rules that make a short policy hold matter more than coverage.
  • ISO/IEC 42001 certification before a customer asks. Align to the standard now and buy the certificate when a deal is blocked on it.
  • Bias testing of models you did not build. You cannot test what you cannot access. Push it to the vendor, as one of the questions procurement should be asking.
  • Monitoring as a substitute for trust. Logging that reads like surveillance buys one quarter of compliance and ends honest reporting of near-misses, the only early-warning system you have. There are ways to enforce a policy without it.

What a working program costs

In hours: 20 to 40 for the initial pass, then one to two hours a month plus a quarterly half-day to re-run the inventory. In money: a few hours of outside counsel for interpretation, and whatever it costs to move your team off consumer logins onto accounts with administrative controls and data-handling commitments.

The variable line is capability. Applied sessions on LLM fluency run in a market range of roughly $2,000 to $15,000 depending on scope and headcount, and that is the line converting written controls into judgments people can make. Review monthly in twenty minutes on an agenda that already exists, and immediately after any incident or vendor change to data handling.

Frequently asked questions

Do we need an AI governance program with no compliance function?

Yes, and it should look different rather than smaller. Build the controls that hold without an enforcement body: an inventory, a three-tier data rule, one named owner, a fast path to sanction tools, and one short artifact for the people who ask.

Who should own AI governance at a company of 50 to 500 people?

One named person, usually the COO, VP of operations, or chief of staff, plus a named person per material risk. Not a committee. The failure at this size is vacancy: operations assumes IT owns it and IT assumes legal does.

Does the EU AI Act apply to a US company of our size?

Only through its own scoping rules, which key on role and market rather than headcount. Article 16(a) puts the high-risk duties on providers; a company buying AI seats for internal work is a deployer under Article 26. With no EU establishment and no EU customers it does not bind you, though fine-tuning or white-labeling can make you one.

Is ISO/IEC 42001 certification worth it yet?

Only when a contract demands the certificate. Market reporting puts first-year all-in cost for a small organization in the mid five figures. Align to the structure now and buy the audit when a deal is blocked on it.

What is the minimum that satisfies a client security questionnaire?

A dated one-page policy, a current inventory of AI tools and the data they touch, a named owner, evidence that staff were trained against it, and a one-page risk list with detection routes.

Should we form an AI governance committee?

No. Grant Thornton found review bodies becoming bottlenecks as use cases multiply, and EY found 47% of large organizations routing around their own process under deadline. A group that convenes when something changes gets the judgment without a queue.

How is AI governance different from our security policies?

Security policies govern access and systems. AI exposure is behavioral, so the question is what a well-meaning employee does with legitimate access to a sanctioned tool. The controls are judgment-shaped rather than permission-shaped, which is why security awareness training transfers only in part.

What makes a reviewer a control rather than a signature?

Knowing where the output fails. A reviewer who can name which claim in a paragraph is the fabrication risk, and has caught one before, is a control. Someone confirming a document they have not read is a record of approval.

Key takeaways

  • The enterprise model is not a target to scale down. EY found 98% of large organizations have formal AI policies and 47% have not applied them to urgent deployments.
  • Review bodies fail by becoming queues, and queues get bypassed. Having no compliance function removes that failure mode along with the resources.
  • A compliance function does three jobs: interpretation, control design, and assurance. Most programs assign only the second, which is why the policy goes unread.
  • Exposure keys on role, not headcount. Article 16(a) binds providers, a company buying AI seats for internal work is a deployer, and Annex III now lands on 2 December 2027.
  • Keep the five controls that hold without an enforcer. Refuse the committee, the forty-page policy, early certification, and bias testing of models you cannot access.
  • Human review is the cheapest control and the one most often faked. Every control reduces to a judgment made in the moment, which makes capability the substrate.

If you want a second opinion on which of your controls are real, book a free AI-readiness call. We will read your policy, name the control resting on a judgment nobody has been taught to make, and say what it would take to close it.

Last Updated: Nov 16, 2026

DJ

Dirk Jan van Veen, PhD

SFAI Labs helps companies build AI-powered products that work. We focus on practical solutions, not hype.

Make your team fluent in AI — then automate what proves out

  • Hands-on training applied to your own documents and workflows
  • Often reimbursable through state small-business training programs
  • Built for 5–50 person firms with no IT department

Related articles