Article 72 for AI agents: post-market monitoring in practice

What Article 72 requires

Providers of high-risk AI systems must establish a post-market monitoring system that actively and systematically collects, documents, and analyses data about the system's performance throughout its lifetime, so that continued conformity with the requirements (including Article 15 robustness) can be evaluated. It has to be proportionate, based on a documented plan, and it feeds corrective action when the system drifts or a new risk appears.

The key word is continuous. A pre-launch red-team run is a point-in-time snapshot; Article 72 asks for an ongoing signal. For an agent whose behavior shifts with prompt changes, model updates, and evolving attacker inputs, that distinction is the whole point — a jailbreak that fails today can succeed after next week's prompt tweak.

What post-market monitoring looks like for an agent

• Evaluators that keep running. The same injection, PII, and tool-misuse judges from your Article 15 testing, but scheduled over live traffic — not archived after launch.
• A sampling plan. You don't grade every span; you grade a representative sample continuously and report with error bars, which is both sound statistics and documentable process.
• Thresholds and alerts. A defined level at which a rising failure rate triggers review — the "systematically analyses" part.
• Traceable records. Findings tied back to the runs that produced them (this is where Article 12 logging and Article 72 meet).

Why evaluators, not just guardrails

A runtime guardrail blocks a bad request in the moment — useful, but it doesn't produce the longitudinal performance record Article 72 asks for. Evaluators scoring a sample of traffic over time do: they give you a rate, a trend, and a dated history you can show. The two are complementary; monitoring evidence comes from the evaluators.

How evsec fits

The evaluator packs evsec generates are built to run continuously in Progress Observability — the span-selection config targets historical and realtime traffic by default, with a sampling rate. That is a post-market monitoring setup: define once from your real traces, run indefinitely, keep the results. When your agent changes, regenerate the packs from fresh traces so the monitoring tracks the current attack surface, not last quarter's.