I build AI factory: one control plane that holds every model provider, your data, your agents and your spend under a single rule. The company stops shipping isolated proofs of concept and starts to manufacture AI products in series, at a predictable cost and pace.
16 providers · one gateway · switching a model is configuration, not a refactor
Six lines · a business brief goes in, a monitored running service comes out
A project ends with a demo. A factory has a line that every following use case runs through. The first ninety days build the line, everything after that rides on it, and both cost and delivery time fall with every repetition.
Shared gateway, shared evaluations, shared guardrails, shared data. The second use case does not rebuild the infrastructure, it consumes it. Inference cost drops through caching, routing simple work to a smaller model, and batching.
The architect owns the model catalogue, the budget caps, the policies and the kill switch. The business owns the brief and the acceptance. Engineering owns the code. Nobody ships a model to production outside the line.
Time from brief to first production release, cost per thousand requests, share of agent runs that finish successfully, escalation rate to a human, and accuracy against the golden set. Daily, not quarterly.
Interactive model · the diagram follows the exact path your configuration produces
Single ingress for every request in the company. Verifies the caller, applies the per team quota and opens a trace that follows the request through every later stage.
Without it there is no single place to enforce quotas, and nobody can tell you afterwards who spent what.
Classifies the task before any model is touched. Routine extraction and classification go to a small model, open ended reasoning goes to a large one.
Turn it off and every request pays large model prices, including the ones a small model answers just as well.
Matches the incoming question against answers already produced, by meaning rather than exact wording. A hit skips retrieval and inference entirely.
Turn it off and repeated questions are paid for again every single time. In support workloads this is the single largest cost line.
Pulls the passages the model needs from corporate systems, with access rights applied and lineage recorded from source document to final answer.
Without it the model answers from general knowledge, which is where hallucinations about your own products come from.
Reserved for genuine reasoning: ambiguous cases, long documents, multi step decisions. Highest quality, highest cost, highest latency.
The routing decision above determines how often this box is reached, and that ratio drives most of the bill.
Handles the repetitive majority: classification, extraction, formatting, short answers. Roughly seven times cheaper and three times faster.
A well tuned stack sends around sixty percent of traffic here without any measurable loss of quality.
Checks the primary provider before committing the call. Latency, error rate and quota are evaluated on every request, not on a dashboard once a day.
This is the point where a provider outage becomes either a two hundred millisecond delay or a failed request.
A second provider on standby with an equivalent model, kept warm and continuously evaluated against the same golden set.
Toggle the outage scenario with fallback disabled and watch the success rate collapse to whatever the cache can still answer.
Policy check on both input and output: prompt injection defence, data leakage, tone, forbidden claims. Every decision is written to the audit trail.
Turn it off and error rate roughly quadruples, and you lose the evidence you need when a regulator asks.
Decides whether a person must confirm before the action executes. The threshold is a business rule, expressed in money or in impact, not in model confidence.
This is where the business keeps control. Raise the threshold and throughput rises with it, along with exposure.
The write. Creating the ticket, sending the reply, posting the invoice. Bounded by the per run budget and reversible through the kill switch.
Everything before this point is reversible. This step is where an AI system stops being advisory.
The answer, plus the record: latency, cost, model used, passages cited, escalation flag. The same record feeds the finance report and the evaluation set.
Without this the system cannot be improved, only guessed about.
Pick a process and move the sliders · every number recalculates instantly
What leaves the process is the part nobody enjoys: retyping between systems, hunting through documents, the first pass of triage. People handle exceptions, decisions and the customer relationship. That is where their hour is worth the most.
Answering a customer in minutes instead of days moves both conversion and satisfaction. The same team capacity absorbs more work without hiring.
Every run has a golden set, an accuracy score and an audit trail. For the first time you know how well the process performs, because it is measured by machine rather than estimated by a manager.
Budget caps, a kill switch and approval on risky steps are part of the line. Nothing runs outside the rules you set.
A closed loop · target, build, measure, re-prioritise
Not a technical request, a number. Cut claim settlement from eleven days to three, at the same error rate, for no more than eight cents a case.
›Data, model, agent, guardrails and monitoring. Briefs enter the queue by value, not by who pushes hardest.
›Whatever does not pay gets switched off. Line capacity moves to the next process. The call is made on last week data, not on the mood of a meeting.
›Cost per run, handling time, accuracy against the golden set, escalation rate. The architect and the CFO look at the same numbers.
›Pick a case · same factory, different process
The business layer and the factory run in parallel · gates sit between the waves
Seven axes · scored on production deployments, not on certificates
The path from a whiteboard to Terraform modules a team can take over without me watching. C4 models, ADR records, clear ownership boundaries.
Tool orchestration over MCP, planners, fallback chains, human in the loop gates and a hard budget cap on every run.
One gateway over Bedrock, Vertex, Azure AI Foundry and self hosted vLLM. When a provider raises prices or goes down, traffic moves within a minute.
Ingest, chunking, hybrid search, lineage from source to answer, and evaluation that catches regression before the customer does.
EU AI Act, ISO 42001, risk classification, red teaming, prompt injection defence and an audit trail that survives an inspection.
Cache layers, quantisation, batching, the right model for the right task. Cost per thousand requests is a tracked metric, not a surprise on the invoice.
I lead engineering teams through deployment and leave only once the system runs without my name in the on call rotation.
Five phases · fixed deliverables with a date, not consulting hours
A map of models, cost, risk and the places where the stack quietly bleeds. At the end you know what to kill, what to keep and what to rebuild.
Diagrams, decision records, Terraform modules and a CI pipeline. Ready for a team to take over, not for a slide deck.
A real process running under monitoring, with a budget and a kill switch. Not a demo, but work somebody stops doing by hand.
Model switching at runtime, cost caps, per team reporting. A provider outage or price rise stops being your problem.
EU AI Act, ISO 42001, audit trail, evaluation and a red team cycle. Evidence that the system does what you claim it does.
Abridged record · the real sequence of steps, not a demo