From using AI ad hoc to building with LLMs on purpose.

We go through the balance sheet of the last two years: which announcements never materialised, which technologies actually reached production, and where the market is focused today. We cover AI-assisted coding, RAG on your own data, tool calling, agents and MCP, reasoning models, and open-weight models you can run in your own infrastructure. That last one decides whether your data has to leave the organisation at all.

We cover the economics of running models separately: why a falling price per token does not always translate into a lower bill, and how to count cost per outcome rather than per token.

OutcomeYou can tell what is real today from what is still a promise, and justify the difference to your team and your board.

We set up three intuitions that are enough for everything that follows:

• why a model predicts the most probable continuation rather than the truth, and where hallucinations come from
• why it has no memory beyond the current context
• what its non-determinism actually is

Most of the time goes to context: what it is, how it is maintained, and why managing it decides the quality of everything downstream. We deliberately stay out of transformer architecture and the mechanics of attention, because they are not central to everyday work.

OutcomeYou understand why the model behaves the way it does, without looking under the hood.

We show where the reflexes of classic software delivery fail: determinism, repeatability, testability. Among other things, we cover:

• why a larger context window does not always mean better accuracy
• how an excess of irrelevant content degrades the answer
• why a model is an unstable external dependency, not a fixed piece of infrastructure

OutcomeYou know what you must not assume when designing a system with an LLM, and why someone has to design it deliberately and watch it.

Live, we take a real case: classifying text written by people. First a keyword-based router handles it, then the same case goes to a model.

On that basis we talk about the difference between listing every possible phrasing up front and recognising meaning, and about where pattern matching ends and reasoning begins. Expert systems once hit the same wall. We also cover the reverse case: when the model should only work out what is being asked, and ordinary, predictable code does the rest.

We put the evaluation criteria in order and every participant can apply them to their own case: a developer to a piece of code, a product manager to a feature, an IT manager to a process.

Example criteria:

• the nature of the input: does it have a fixed structure, or is it written freely by a person?
• the cost of a mistake: what does a wrong answer cost, and who will catch it?
• the availability of context: is what the model needs for the decision present in your systems?
• tolerance for variability: do you accept different answers to the same question?

OutcomeYou leave with criteria to apply to the next ideas, and with the decision for the case discussed in the room.

We start with why a model does what you wrote rather than what you meant, and why a narrowly posed task gets an answer that settles a single case. We also cover why this does not go away with each new generation of models.

We then show how to describe the class of problem instead of hunting for better wording.

We break such a task into parts that can be judged separately:

• scope: what still counts as the same problem, and what to treat as a separate case
• context: what the model needs to know to handle them
• response format: in what shape the result goes to the next step
• edge cases: what to do when data is missing or the question is out of scope

OutcomeYou can tell when you got a solution to a class of problem and when you got a patch for one example.

Context is the whole body of information an LLM receives along with the task. We cover what needs to be in it for the task to be doable, and what the consequences are of a badly planned context structure.

In practice it comes down to a few decisions:

• selecting information: what is essential to do the task, and what we add just in case
• a long session: when to compress, when to cut the thread, and when to start fresh
• project rules: what to write down once so you are not pasting it with every task
• the bill: what it costs to keep all of this in context

OutcomeYou know what to take out of the context, not just what to add to it.

We go through the techniques that actually change the result, and show what makes them work.

• examples: when two well-chosen examples do more than a paragraph of description
• response format: forcing a structure that can be checked automatically
• splitting the task: what you gain by breaking it into steps, and what you lose
• self-checking: when asking for verification helps, and when it only adds to the bill

Separately we cover the advice that has gone stale, and the advice that only works on some models. We cover why “think step by step” stopped being the lever on reasoning models, while it still counts on smaller ones and inside tool loops.

OutcomeYou know which technique makes sense on the model you are calling, instead of applying all of them at once.

We cover how to gather a set of real cases, decide for each what a good answer means, and run successive versions of the instruction through them.

• choosing the cases: cases from real use, including the ones that ended in an error
• the scoring criterion: an unambiguous definition of a good answer, independent of who is judging
• size of the set: how many cases it takes before a single outlier stops deciding the verdict
• comparing versions: how to tell a real improvement from a coincidence

Separately we cover the trap that is easy to fall into: a set you start tuning the instruction towards stops measuring anything at all.

OutcomeAfter changing the instruction you know whether the result is better, or merely different.

We supply a prepared task along with a dozen or so cases and their expected answers. You work in pairs, and at the end we compare the results.

• scope: which cases the instruction should handle and which it should refuse
• context: what to add to the request, and what to leave out
• first version: run against the prepared set
• the revision: a second version compared on the same set

At the end we put the pairs’ results side by side. On a single task you can see how far the solutions diverge and which wordings came out better.

OutcomeYou have practised the full cycle from instruction to comparison, and you see how differently worded instructions fare on the same set.

We show how these tools differ in practice: what works in the editor, what in the terminal, and what gets access to the whole repository and runs commands.

• scope of access: one file, a project, or the whole machine
• mode of work: a suggestion as you type versus a task handed over and done in the background
• the agent’s output: a diff, a record of its decisions, or just code with no context

We do not score them against each other, because the ranking shifts every quarter. We give you the criteria you will apply yourselves to whatever comes next.

OutcomeYou know which tool fits which work, and what to demand of the next one that arrives.

We take each trap the same way: how to spot it in a diff, where it comes from, and what to do about it.

• hardcoding: the agent kills one specific error instead of solving the class of problem, and special cases keep piling up
• context rot: in a long session quality drops and earlier decisions blur together
• sycophancy: the model confirms your wrong premise instead of challenging it
• silent assumptions: it fills in requirements nobody gave it

On hardcoding we also cover why it does not go away with each new generation of models. Two years ago it was crude and visible at a glance, today it can be subtler, and the mechanism is the same.

OutcomeYou recognise these behaviours in code before they reach the main branch.

Code from an assistant reads differently from a colleague's code: there is nobody to ask why it was implemented this way, and the whole thing looks polished whether or not it is correct.

We cover what to look at first and which control questions to ask, so you catch what looks fine but will not hold outside the example it was born on.

OutcomeYou read such code faster and more accurately, instead of reviewing everything or trusting that passing tests mean it is fine.

We go through what an assistant shifts in a team’s work, not only in one developer’s.

• estimates: the first eighty percent speeds up, the last twenty usually does not
• review: turns from a bottleneck into the main point of quality control
• onboarding: an assistant explains the system better than a busy colleague, except that writing code by hand forced understanding and generating it does not
• code ownership: who answers for a piece nobody wrote by hand

OutcomeYou know which team measures and rituals stop working once assistants arrive, and what replaces them.

We take a task from a real repository and run it with an assistant from the description to a finished diff.

OutcomeYou see the whole cycle live, including the places where it usually breaks down.

Three things that conversation tends to call by the same name. We draw the lines: what decides the next step, who has access to systems, and what happens when something goes wrong.

• chat: it answers, and acting stays on your side
• automation: it runs steps someone laid out in advance
• agent: it picks the next step itself, within the bounds it was given

OutcomeYou can tell what a vendor is actually offering when they say “agent”.

We take an agent apart and walk one lap: the model gets context, picks a tool, gets a result, and decides whether to stop.

• the decision loop: what stops it, and what happens when nothing does
• tools: how the model knows they exist and how it chooses between them
• memory: what survives between steps and what disappears
• system state: what has to live outside the model so you can resume interrupted work, including after a failure

OutcomeYou can point to where the agent makes a decision and where it can be stopped.

We cover the mechanism by which a model calls code: how a tool is described, what the model gets back, and why the description often matters more than the implementation.

We take MCP as the standard that tidies this exchange between vendors. We show what it genuinely gives you and what it does not solve.

OutcomeYou know how to expose your own system to an agent, and what to ask a vendor who claims MCP support.

Splitting a task across several specialised agents sounds orderly, and often just multiplies the places where something can go wrong.

We go through the criterion that settles it: not the number of agents, but what passes between them. Parts that run in parallel and do not wait for one another split well, because each agent works on its own context. Steps that follow one another make sense when they hand over a complete result, a document or a set of data, rather than a summary of a conversation the next agent cannot reconstruct. Where that clean handover cannot be guaranteed, one agent with well-described tools does better.

Against that background we review specific architectures: the ones with deployments behind them, the ones that get talked about more than they get built, and the ones that did not hold up.

• a fixed path with model steps: most working deployments come from here, because the order of steps and the error handling stay in code while the model covers single pieces
• a lead agent with helpers: works where parts of the task are independent of each other and write nothing, such as searching and comparing information
• one agent does the work, another checks it: works when the checker has something to verify against, such as tests, rather than the opinion of the same model
• agents with assigned roles: one model under several system prompts with the same access to tools, whereas a real split requires separating the tools, the permissions or the model itself
• a long chain of handoffs with no shared state: each step gets only a summary of the previous one, so when a single step works ninety five percent of the time, after ten the whole thing works sixty

Separately we count the bill: every agent is its own paid model call, and steps run one after another stretch the response time.

OutcomeYou can justify why one agent is enough in your case, or why it is not.

We settle where the agent acts alone and where it waits for a person. The criterion is not technical: it follows from what can be undone and what cannot.

• irreversible actions: sending, paying, deleting, changing permissions
• scope of rights: what the agent can see and do on someone’s behalf
• content from outside: what happens when a fetched document carries an instruction for the model
• the emergency stop: how to halt an agent mid-run, what to do with the steps that already went through, and what has to be in the log to reconstruct the run

OutcomeYou have the list of places that need an approval before an agent touches production systems.

We build an agent with a few simple tools live and watch how it picks one of them and carries the task on.

OutcomeYou see the mechanism from the inside rather than from a vendor’s documentation.

RAG tends to be the first answer to any question about company data, though some cases resolve more simply and more cheaply.

We cover at what scale and what rate of change RAG starts to pay off, and what its real alternatives are today.

• the whole set in context: when there are few documents and they rarely change, passing them in full and relying on caching costs less than maintaining an index
• agentic search: the agent searches the source system the way a person would, so nothing is duplicated and permissions stay with the system
• plain keyword search: enough when the questions repeat and the documents are well described
• RAG: justified where the set is large, changes often, and the questions cannot be anticipated

OutcomeYou settle whether your case needs RAG at all before anyone starts building it.

We follow a document from source to answer and stop wherever a decision is made.

• splitting documents: what counts as one piece of content, and why that decides the rest
• retrieval: by meaning, by words, or both
• ranking the results: what decides which passages reach the model when retrieval returns a dozen of them
• freshness: what happens when the source document changes
• the answer: how the model should cite a source for each claim

OutcomeYou can read someone else’s architecture and know what to ask about.

The usual diagnosis is “the model is weak”, and the usual cause sits earlier: the document was not found, or the wrong passage was.

We show how to separate those two cases, because they are fixed in completely different ways.

OutcomeYou can say whether the problem is in retrieval or in the model, instead of swapping in a bigger model.

Searching company documents runs into the fact that not everyone may see everything. We cover what has to be settled before the first document reaches the index.

• permissions: whether the answer respects what the person asking may access
• sensitive data: what should never enter the index at all
• where it runs: what stays with you and what leaves
• permission changes: what happens to the index when someone loses access to a document that has already been indexed
• the query record: who asked what, and what came back

OutcomeYou have the list of decisions without which a RAG project will not clear security.

We take a prepared set of documents and show how retrieval behaves across different questions.

The collection is chosen so that specific failures can be shown on demand, rather than waited for. Your own cases we discuss against that background, with no need to bring data into the room.

OutcomeYou see where accuracy is most often lost along this chain.

We extend what we did for a single instruction to a whole solution.

• evaluation set: cases with an expected answer, maintained alongside the system
• automatic scoring: when a model may judge a model, and where that fails
• the gate: the threshold below which a change does not ship

OutcomeYou know what to demand from a vendor who claims their solution works well.

The bill rarely grows with the number of users. It grows with how much context travels on every call, and how many times the system calls the model to answer once.

• context size: what you add on every request
• number of calls: how many steps the agent takes before it answers
• retries: what happens on errors, and whether anyone counts them
• choice of model: where the strongest model is needed and where a cheaper, faster one is enough

OutcomeYou can estimate the cost of a solution before rollout and point to what raises it most.

A system that worked on launch day need not work a quarter later. Traffic changes, data changes, and the vendor swaps the model version.

• quality over time: whether answers still clear the evaluation set
• model change: what you do when the vendor ships a new version
• cost and latency: whether they grow faster than traffic
• the user’s signal: how they report that an answer was wrong

OutcomeYou have the indicators that tell you something is breaking before a customer does.

Everyone writes criteria for one solution from their own organisation: what must hold for it to ship, and what would mean pulling it back.

OutcomeYou leave with criteria that can go into a contract or a definition of done.

A few months after assistants arrive, a team usually has as many ways of working as it has people. Everyone has their own instructions, their own settings and their own beliefs about what works.

We cover what is worth settling together and what can safely stay individual. Separately we look for a home for those decisions, so they do not end up as another document nobody reads.

OutcomeYou know which decisions have to be the team’s, and you have a place where they live.

A feature with a model inside needs a different description from a deterministic one. You cannot write down what it must return, so you write down what is acceptable.

• scope: what the feature handles, what it refuses, and how it says so
• acceptable result: when an answer is good, and when it is merely plausible
• behaviour on failure: what the user sees when the model cannot cope
• how it is accepted: what you run and what result you treat as good enough

OutcomeYou write a specification against which the work can be accepted.

A demo takes days and the rollout takes a quarter. That gap is not the team dragging its feet: the last twenty percent is integration, edge cases and agreements that a demo never shows.

We show how to plan work where the number of iterations is unknown up front, and how to have that conversation with the business before a date is named.

OutcomeYou can justify a schedule instead of defending a date set after someone watched a demo.

We cover what in this shift belongs to the leader rather than the team: choosing tools and the limits on their use, the review standard, and developing people whose work moves from writing to judging.

We also cover the question asked most often and answered least: what to do with someone who does not want to use any of it.

OutcomeYou leave with a list of decisions to make, and you know which of them cannot wait.

We hand you the description of an existing feature that is to gain an AI component. You rewrite it in teams: scope, acceptable result, behaviour on failure and how it is accepted.

OutcomeYou have a pattern to apply to the features that follow.

We go through the things whose absence only surfaces after launch, when they cost the most.

• quality measurement: an evaluation set and a threshold below which you do not ship
• safeguards: what the system does with content it should not act on
• oversight: who watches the indicators, and how often
• rollback: how to get back to the previous version within an hour

OutcomeYou have a checklist to run before every launch.

We discuss what is worth building yourselves and what to buy, and how this line moves faster here than in other areas.

• the differentiator: whether this is what sets you apart, or infrastructure
• data: what would have to leave the organisation
• upkeep: who develops it a year from now
• the exit: what happens when the vendor changes terms

OutcomeYou can justify the decision to the board, and you know when to revisit it.

The last part of the day is for sorting things out: everyone writes down what they want to do on their return, and we say what usually works in the first month and what needs longer preparation.

OutcomeYou know where to start and what to postpone, so you do not begin with the hardest thing.