// Discipline 02

AI Systems

Agents, MLOps and observability

From agent architecture to the model in production: governed agents, measurable evaluation and observability to know what the system does and why.

01 · Who it is for

Who it is for

02 · How we work

How we work

03 · Specialties

Specialties

04 · Method

Method

  1. 01

    Diagnosis

    What data exists, which decision depends on it and what has failed so far.

  2. 02

    Design

    Hypothesis, method and success criteria in writing, before building anything.

  3. 03

    Build and test

    Reproducible code and analysis, tested against adverse cases.

  4. 04

    Handover

    Everything is documented and in your team’s hands.

05 · Sectors

Sectors

Finance

  • Research, risk and compliance agents
  • RAG over regulatory documentation

Defense

  • Evaluation and assurance of AI systems
  • Assisted analysis of large volumes of text

Civil

  • Internal assistants over company documentation
  • Process automation with human oversight

06 · Use cases

Use cases

Illustrative cases: they describe the kind of work we do, not specific clients.

  1. From prototype to production

    Problem
    An agent that gets the demo right and nobody knows why it fails afterwards.
    Approach
    Traces of every step, an in-house evaluation set and explicit limits on what it can do.
    Deliverable
    Deployed system, quality metrics and an operations guide.
  2. RAG that cites its sources

    Problem
    Plausible answers with no way to check them.
    Approach
    Indexing with metadata, evaluated retrieval and answers with verifiable citations.
    Deliverable
    Retrieval pipeline and quality dashboard.
  3. Governed agents for finance

    Problem
    Automating research or risk without losing control.
    Approach
    Agents with roles, permissions and human review at the critical steps.
    Deliverable
    Reference architecture and code (the basis of the QAO course).
  4. Model observability

    Problem
    Costs, latencies and errors nobody measures.
    Approach
    Telemetry for every call, alerts and continuous evaluation.
    Deliverable
    Dashboards and alerts integrated into your infrastructure.

07 · Data and topology

Data and topology

An agent system is a graph: tools, memories, models and people connected. We design and measure it as such, node by node and path by path.

08 · Research

Research

Everything under the tag

A data problem in this area?

Tell us the context: what data exists, which decision depends on it and what has failed so far.

Get in touch