What does AI governance look like for a company that isn't a tech company?

AI governance for a company that isn't a tech company is not a policy document. It is a short set of decisions, made before the build, about what AI is allowed to touch, who stays accountable for its output, and how you will know it is working. Get those three right and you can adopt AI aggressively. Skip them and you are one confident wrong answer away from a problem you cannot see.

Do you even need governance at a 40-person company?

Yes, and in one specific way you need it more than a large company does. You do not have a compliance department, a legal team, or a security function to catch a bad decision after the fact. So the governance has to be light, and it has to live inside how the work gets built, because there is no second line of defense behind it.

The goal is not paperwork. It is a few rules that keep a useful tool from quietly doing damage while everyone is busy running the business.

What are the decisions that actually matter?

Three, and they are smaller than the phrase "AI governance" makes them sound.

What data the AI is allowed to touch. Not all of your data belongs in an external model, and that is a call you make before the build, not after something has already been sent somewhere you cannot pull it back from. For most companies the off-limits list is short: customer records, anything regulated, anything you are contractually not allowed to share. Draw that line first.

Who is accountable for the output. AI produces a draft, a classification, a recommendation. A named person owns whether that output is right, especially where being wrong costs something. "The model said so" is not accountability. Either someone signs off, or the system is built so a human sees the cases that matter before they go anywhere.

How you will know it is working. If you cannot measure whether the AI is helping, you cannot govern it. You are just hoping. Pick the measure before you turn it on.

Where do non-tech companies get this wrong?

They treat it as a document instead of a design choice. A policy that says "we use AI responsibly" changes nothing about what happens on Tuesday. Governance that works is built into the workflow. The model only sees the data it is allowed to see, the risky cases route to a person, and the output is logged so you can check it later.

The other common mistake is reaching for a model when the real problem is that two systems do not talk to each other. That is an integration problem, not an AI problem, and no amount of governance fixes it. Putting a model on top of a broken process just makes the breakage harder to spot.

What does this look like in practice?

Two examples from my own work.

When I embedded AI across our software delivery, from requirements and prototyping through development, testing, and code review, it went in under written rules. It only touched work that a person then reviewed. That combination is the governance: repetitive enough to be worth automating, reviewed enough to be safe. Feature discovery went from weeks to days, and nothing shipped that a human had not checked.

More recently I built an operating system for a healthcare client's shipping process and put an AI agent over it to monitor the system and answer questions about it. The rules there are concrete. The agent reads the operational data but does not make irreversible decisions on its own. Routine cases move automatically, unusual ones surface to a person, and the whole thing is measured so we can tell it is actually working. The CEO does not have to trust the AI blindly. They can see what it is doing.

How do you start without turning it into a project?

Pick one workflow. Write down the three decisions for that one workflow: what data it touches, who owns the output, how you will measure it. Build it that way. Now you have a working example and a governance pattern you can reuse on the next one, instead of a forty-page policy nobody opens.

That is the whole approach. Problem first, then the workflow, then the AI, then the controls, then the measurement, then scale it to the next thing. Governance is not a gate you clear before you are allowed to start. It is three questions you answer as you build, and they are the difference between AI you can run the business on and AI you are quietly hoping is fine.

If you are about to put AI into something that matters and you are not sure what the guardrails should be, that is exactly the conversation worth having before the build, not after.