While most people were away on their Christmas holidays, I spent the break building my own AI-powered system.
After acquiring the necessary server hardware, I began trialling locally hosted AI models and building workflows and governance rules around them.
For me, the biggest learning wasn’t which AI model is superior; it’s that the choice of engine matters less than the system you build around it.
When most people think about AI, they think about ChatGPT and Claude.
In reality, there’s now a galaxy of AI models out there. Many of them are very impressive, and you can run them locally so long as you have the hardware to do so.
For many everyday reasoning, writing and content tasks, the gap between the best frontier models and smaller or open models has narrowed dramatically.
That doesn’t mean model choice is irrelevant. It means the choice of engine isn’t the most important part of your strategy.
Most generative AI platforms have one thing in common: they’re built around a large language model, commonly referred to as an LLM.
Now everyone is talking about “AI agents”.
An AI agent isn’t a better type of LLM. Building an agent involves integrating an AI model into a system that can perform tasks in the real world.
An LLM is a bit like a brain without a body.
An LLM can accept an input — usually text — and generate an output, whether that’s sentences, code, or structured data. But it can’t act in the world unless it’s connected to tools, permissions and rules that let its outputs trigger action.
The model is the brain.
The tools are the limbs.
Together, they give a system the ability to act.
System design is the difference between a basic chatbot and an agentic system that has serious potential in an enterprise setting.
These days, the biggest factor influencing the effectiveness of an agent isn’t the choice of AI engine. It’s the design of the system you create around the engine.
A powerful engine inside a rudimentary system will often produce worse results than a weaker model operating inside a system that’s properly grounded in the business’s operating model and risk environment.
Most people experience AI through a single interface, which creates the illusion that “the AI” is a single, all-purpose intelligence.
In reality, many useful AI products combine multiple layers: models, tools, retrieval sources, instructions, permissions and workflows.
An advanced agentic system can break a request into smaller tasks and route them to specialist sub-agents.
A complex writing task might spawn a sub-agent specialising in academic research.
A heavy reasoning model might then crunch the research outputs.
A chain of sub-agents might do the drafting, editing and fact-checking.
The exact architecture will vary, but the point is the same: the user sees one interface while the system orchestrates the work going on behind the scenes.
Each sub-agent might use a different model, toolset, instructions, retrieval source or permission level.
Governance is what turns a collection of models, tools and workflows into something an enterprise can actually rely on.
That sounds abstract until you try to build something useful inside a real organisation, where information is fragmented, ownership is unclear and risk has consequences.
Let’s say you want to build a system that automates the production of sales pathway content for an insurance company.
To perform this task, the system will need access to structured information about each product:
- benefits
- restrictions and exclusions
- eligibility rules
- premiums and discounts
- approved marketing claims
- disclosure obligations
- terms and conditions
- brand and tone guidelines
All of this information will be owned by different parts of the business and stored in different systems, in different formats.
If the product page says one thing, the terms and conditions say another, and the marketing brief frames the offer differently again, a better AI model won’t magically resolve the conflict.
The system needs rules for deciding which source is authoritative, which claim is approved, and what information the customer needs at the point of decision.
Implementing such a system requires answers to difficult operational questions like:
- How will the AI access the right information?
- Which source is authoritative when documents disagree?
- Is the information organised in a way the system can find and retrieve what’s relevant for the task at hand?
- Is the information structured in a way the AI can use reliably?
- How do you make sure outputs are accurate and consistent across products?
- How do you make sure the system retrieves the right information at the right point in the workflow?
- Who reviews and approves the output before it goes live?
- What does the system do when confidence is low?
- How do you make sure the content fragments add up to a human-centred experience that complies and converts?
The answers to these questions have two things in common:
- they have almost nothing to do with the choice of AI model
- they all relate to system design and governance
AI governance is not something that you layer on top of an AI-powered system. It’s an integral component that must be designed into the machinery.
But there is one part of the machinery that is even more overlooked: the human.
Someone still has to frame the problem, judge the output, understand the risk and decide what the system is actually for.
That raises a harder question:
What skills do people need to manage and operate AI systems safely and effectively?
More about that later.