AI implementation: what actually works, and what quietly kills projects

Most organisations we talk to aren’t short on AI ideas. They’re short on a working answer to one question: why did the demo look great in October and still isn’t running in March? The technology usually isn’t the problem. The gap between a convincing prototype and a dependable production system is architecture, evaluation, and integration work — and it’s the part that rarely gets scoped, let alone budgeted, up front.

This is a longer, more technical look at that gap: where the real opportunities are, what tends to go wrong, and the concepts worth understanding before committing budget — models, context engineering, agents, MCP, inference, and training.

Where the opportunity actually is

The organisations getting real value aren’t the ones with the flashiest chatbot. They’re the ones that picked a narrow, well-bounded workflow — triaging support tickets, drafting a first-pass contract review, reconciling records across two systems that were never meant to talk to each other — and made it reliable. AI is leverage on work that’s high-volume, well-defined, and currently done by a person copying information between systems. It’s a poor fit for anything where “mostly right” is worse than “clearly someone’s job.”

The honest opportunity is narrower than the pitch decks suggest, and that’s exactly why it’s valuable — a narrow win that actually ships beats a broad one that doesn’t.

The technical foundations worth understanding

You don’t need to be able to build a model to make good decisions here. You do need to understand what these terms actually mean, because they map directly to where budget and risk sit.

Models. Almost no organisation implementing AI today needs to train one from scratch. The real decision is between hosted frontier models (fastest to start, ongoing per-token cost, data leaves your infrastructure unless the provider offers otherwise), and self-hosted open-weight models (more control and predictable cost at scale, more infrastructure to own). Most projects should start hosted and revisit that decision once usage patterns are actually known — optimising infrastructure cost before you know your usage shape is solving a problem you don’t have yet.

Context engineering. This has quietly become the actual discipline, more than prompting. A model’s output is only as good as what’s in its context window at the moment it answers: retrieved documents, tool outputs, conversation history, system instructions, examples. Getting this right — what to retrieve, how to rank it, how much to include, how to keep it current — determines output quality far more than which model you’re calling. Most “the AI gave a wrong answer” complaints are context problems, not model problems.

Agents. An agent, in the current sense, isn’t one model call — it’s a model deciding what to do next across multiple steps, calling tools, reading the results, and deciding again, in a loop, until it judges the task done. That loop is where most of the real engineering risk lives: reliability compounds across steps, so a model that’s right 95% of the time on any single step is wrong more often than that over a ten-step chain. The fix isn’t a smarter model, it’s scoping — narrow the task, cap the loop, add checkpoints where a human or a rule reviews progress before the agent continues, and log every step so failures are diagnosable rather than mysterious. The agents worth shipping are usually the boring ones: doing one job reliably inside an existing workflow, not an open-ended assistant that can theoretically do anything.

MCP (Model Context Protocol). An open protocol, introduced by Anthropic and since adopted more broadly, for connecting models to the tools and data sources they need — a standard way for an AI application to discover and call a database query, an internal API, a file system, without every integration being bespoke. Practically, it matters because it changes agent integration from “custom glue code per tool” to “implement the protocol once, connect to anything that speaks it.” Worth understanding as an architectural option, not worth adopting just because it’s current — the right call depends on how many tools an agent actually needs and whether they already expose sane APIs.

Inference. This is where AI’s operating cost actually lives, and it’s usage-shaped, not fixed — cost and latency scale with how much context you send and how often, which is exactly why context engineering and inference cost are the same conversation. A system that re-sends an entire document on every turn will be slower and more expensive than one that retrieves only what’s relevant. This is architecture work, not a pricing-page decision.

Training / fine-tuning. The most commonly over-reached-for tool. Most business problems that look like “we need to train a model” are actually context or retrieval problems, solvable without touching model weights at all. Fine-tuning earns its cost when you need a consistent style or format at volume, or behaviour that in-context instruction genuinely can’t achieve — not as a default starting point.

Where projects quietly fail

• Treating the demo as done. A demo proves the idea is plausible. It says nothing about behaviour on messy real-world input, under real load, or when the underlying data changes next month.
• No evaluation framework. Without a way to measure whether a change made things better or worse, every iteration is a guess. This is the single most common gap we see, and the cheapest one to fix early.
• Skipping data access and quality work. An agent is only as trustworthy as what it’s grounded in. Ungoverned access to internal data is a security problem; poor-quality source data is a credibility problem — and both surface after launch, not before.
• Building a chatbot instead of solving a workflow. A chat box bolted onto an existing product is usually a symptom of skipping phase one: figuring out where AI actually fits before deciding what to build.
• No human-in-the-loop where the cost of being wrong is high. Full autonomy is a design choice, not a default — it should be earned by evidence, not assumed from day one.
• Governance as an afterthought. In the EU, AI Act risk classification and compliance are much cheaper to handle at design time than retrofitted after a system is already in production.

What actually works

None of this is exotic — it’s closer to standard engineering discipline applied to a newer kind of system. Start with a real workflow, not a technology. Define success criteria before writing code, not after. Treat context and retrieval as the primary design surface, not the model choice. Pilot against real data. Put humans in the loop wherever a wrong answer is expensive. Plan for governance from the start rather than bolting it on. And evaluate continuously, because both your usage and the underlying models will keep changing after launch.

This is the same shape as the delivery model we use with clients — readiness, architecture, pilot, production, governance — because it’s the sequence that keeps a project honest at each step, not a sequence we invented for its own sake.

Get in touch if you’re weighing where AI actually fits in your organisation.