Managed Operations

Stage seven, the learning loop, is the stage that quietly stops running. Evaluation suites go stale. Sources drift. A policy library gets superseded and the retrieval index does not hear about it. Nobody notices until an agent answers confidently from material that was replaced a year ago — and by then the question of who owns context health has the worst possible answer, which is everyone, meaning no one.

Managed Operations exists because that stage needs a named owner. It is a monthly engagement, on a quarterly rolling commitment, covering the agreed stages of your context supply chain: monitoring their health, maintaining the evaluation machinery, and running the loop that turns production failures into system changes. The fee range tracks coverage — a single domain with a stable content estate sits at the bottom; multi-domain coverage over fast-changing sources sits at the top.

It is deliberately the smallest rung by monthly cost and the longest by relationship, because context health is not a project. It is operated infrastructure, and the alternative to operating it is watching a well-built chain drift back to ad hoc over four quarters — a decay we have watched often enough to price the prevention.

• Teams whose go-live went well and who intend for that sentence to still be true in eighteen months
• Organizations whose content estate changes weekly — product catalogs, policy libraries, clinical documentation — where the chain must keep pace with the sources or become a liability with a search box
• Internal platform teams that own the agent runtime but want the learning loop externally owned and externally accountable
• Anyone who has discovered that keeping the eval suite updated ourselves is the most confidently made and least kept promise in production AI

• Source freshness monitoring — sampling the content estate for supersession and drift, so stale material is found by us before it is found by an agent
• Retrieval fitness re-testing — the ground-truth question set run against production retrieval on a schedule, so quality regression is a detected event with a date, not a slow suspicion
• Evaluation suite maintenance — new failure modes become new test cases; the suite grows with the estate instead of fossilizing at go-live
• Agent operations signal review — production behavior signals triaged, with each incident attributed to its actual layer: context, model, or application. The attribution matters, because the three have different owners and different fixes
• Learning-loop execution — validated operational findings routed back into sources and structure, with the change recorded. The loop's output is changes made, not observations filed
• Backlog and cadence — a maintained continuous-improvement backlog, groomed monthly, drawn on quarterly

ROKA — Return on Knowledge Assets — is the quarterly statement of account: what changed in the chain, what it cost, and what it returned, in terms a business reader can check. Retrieval fitness trend against the ground-truth set. Incidents by layer, with time-to-attribution. Freshness posture by source. Changes shipped through the learning loop, each traceable from the failure that prompted it to the change that resolved it.

It is written so the executive who funded the chain can see whether it is still earning its keep — and so that if the honest trend line is flat, the report says flat, and the review meeting is about why.

• A context supply chain whose health is measured on a cadence instead of assumed
• An evaluation harness that is alive — growing with the estate, run on schedule, trusted enough to gate changes
• Incident attribution that ends the is-it-the-model-or-is-it-us argument with evidence, each time
• A ROKA report your finance and leadership can read without translation
• One accountable partner for long-running context work — a named owner for the stage that otherwise has none

A chain built well at go-live degrades on no particular day. Sources turn over, the eval suite stops being run, and two years in, answer quality is visibly worse with no visible cause. Managed operations is priced against exactly this decay curve — the monitoring exists so the drift is caught in month two, not month eighteen.

A commerce platform's catalog changes daily; an insurer's policy library changes with every regulatory cycle. For these estates the chain is never done, and the question is only whether keeping it current is someone's job or everyone's assumption. This rung makes it a job.

A strong internal team runs the agents and wants to keep running them — but wants the learning loop owned by a party whose quarterly report will say flat when the trend is flat. External accountability for stage seven, internal ownership of everything else, is a clean and common split.

• It is not a helpdesk or a NOC for your AI stack. Coverage is the agreed stages of the context supply chain — model provider incidents, application bugs, and infrastructure outages have their own owners, and the attribution work exists partly to route things to them quickly.
• It is not a substitute for source ownership. Your people still own the content and make the editorial calls; we detect, attribute, route, and verify. A managed loop with no engaged source owners degrades into a very well-documented backlog, and we will say so if we see it happening.
• It is not open-ended. Quarterly rolling means each quarter is a renewal decision made against a ROKA report — which is exactly the discipline we would want if we were the buyer.