Developing an AI agent is not finished when a demonstration produces a sensible response. A model that answers a prepared question accurately may still cite the wrong document in production, leak sensitive retrieval content, retry a business action twice or silently fail during a tool call. These failures often happen outside the final answer text. The evaluation and observability portions of AI-103 therefore require a way to compare behavior, diagnose failures and decide whether a version is safe to release.<\/p>\n
Consider a fictional technical-support agent that diagnoses equipment faults using a document index and prepares repair tickets for a human dispatcher. It should identify relevant manual sections, avoid unauthorized records and stop when the evidence is incomplete. A regression might show up as invented citations, inappropriate escalation, slow retrieval or an approval bypass. The useful question is not whether a model \u201cpassed 90 percent of prompts\u201d but whether the test set exposed the failures that matter to this workflow.<\/p>\n
A model’s fluency is only one system attribute. Retrieval quality<\/a> asks whether the correct manual sections are returned under appropriate permissions. Groundedness asks whether an answer is supported by the sections actually supplied to the model. Task adherence asks whether the agent performed the requested legitimate workflow rather than wandering into unrelated steps. Tool accuracy asks whether the right tool was chosen with valid arguments. Safety asks whether harmful or unauthorized requests were handled appropriately. Latency and cost measure whether the solution remains usable when traffic increases.<\/p>\n These dimensions should have separate acceptance rules. For example, a severe violation of a purchasing approval boundary must stop release even if average answer relevance improves. A slight improvement in response tone may not justify a doubling of latency. Do not collapse critical failures into a single weighted grade that can be rescued by better scores in easier categories. Record severity and the customer outcome alongside each test failure.<\/p>\n OpenTelemetry and Azure Monitor<\/a> can correlate application spans, but an agent trace must also distinguish model invocations, retrieved source IDs, proposed tool calls, server authorization, approval decisions and externally confirmed effects. Correlate those events using bounded request and deployment identifiers without copying private documents into every trace. Otherwise an operator may see a successful HTTP response and mistakenly conclude the assistant consulted authorized evidence or completed a business transaction.<\/p>\n Start with scenarios and expected outcomes rather than collecting only successful answers from the development playground. Include straightforward user requests, ambiguous part numbers, missing evidence, changed policies, restricted documents, tool timeouts and a request that requires a human decision. Give every case a stable identifier, the relevant user role, approved knowledge version, expected tools or prohibited actions, and the condition under which the agent should abstain. Keep evaluation data synthetic or properly governed where privacy matters.<\/p>\n Microsoft’s Foundry dataset guidance<\/a> supports reusable test cases, including JSONL-based datasets and workflows that build cases from trace evidence. A permanent dataset is useful for comparing prompts, models or agent versions; smaller ad hoc tests still have their place while debugging. Version the dataset and hold back some cases from prompt tuning. Otherwise repeated optimization against the same examples can create apparent improvement without evidence of better general behavior.<\/p>\n Build labels that reflect the job. The fictional repair agent could have a case where manual R1 contains one torque setting and R2 contains another. An acceptable answer must cite the correct revision and state the measured setting without inventing certainty. Another case should deliberately omit the revision, expecting the agent to ask a clarifying question. A policy-refusal case should verify that the agent rejects an unauthorized part order but still offers a helpful next step. These expected outcomes are more informative than an unqualified \u201cgood answer\/bad answer\u201d label.<\/p>\n Foundry supports agent-targeted evaluation<\/a> with rubrics and built-in quality or safety evaluators. Different evaluators measure different constructs, and some depend on a judge model. An evaluation result is evidence about the selected cases and criteria, not a certification that the agent is safe in every environment. Read the rubric, check example failures manually and compare evaluator judgments against a small set of expert-reviewed cases before relying on its score.<\/p>\n A reflection or self-critique pass can expose a missing assumption, but it is not an independent source of truth. Give the same authorized question to an initial response and a bounded revision pass; then compare the two versions against an external answer contract and the source IDs actually retrieved. A revised answer that sounds more confident while dropping a required policy exception is a regression, not an improvement. Retain the first answer, revised answer, observable tool results, rubric version and validator decision for the release review. Do not treat unverifiable internal reasoning text as a substitute for citations, application logs or real approval evidence.<\/p>\n If a groundedness evaluator marks an answer as excellent, verify that its cited passages really came from the intended corpus and were available to that user. If a task-completion evaluator gives a high score, inspect whether the requested action was authorized. A simulated approval response in test data should not be confused with a real business approval. Build independent deterministic checks for tool names, invalid arguments, repeated transaction IDs and prohibited write requests; these can catch failures that a language-model evaluator glosses over.<\/p>\n Keep the test design transparent. The release report should list how many cases were run, how many were manually reviewed, which metrics were computed and which critical scenarios remained untested. Avoid reporting a \u201c95% accuracy\u201d number without explaining its denominator, label definitions and methodology. A model that succeeds in 19 of 20 ordinary questions may still be unusable if the missing case involves a serious data-access violation.<\/p>\n Microsoft’s Foundry tracing guidance<\/a> uses Azure Monitor Application Insights and OpenTelemetry conventions. For supported prompt and hosted agents, server-side tracing can provide a starting point after connecting an Application Insights resource; additional client instrumentation may be needed for custom application and business-API behavior. Trace coverage and preview status can differ across agent types, so check the current feature support in the chosen environment.<\/p>\n Give each application request a correlation identifier that connects the user’s authorized request to retrieval, model calls, tool operations and the final response. Keep event timestamps, deployment and prompt versions, agent identity, tool names, failure statuses and document IDs available to qualified operators. Avoid saving passwords, raw authentication tokens or entire private documents just because the tracing library can capture arguments. Restrict telemetry access, set retention deliberately and confirm which content fields are recorded by default.<\/p>\n Suppose the agent says a repair ticket was created, but no ticket appears in the service system. A useful trace should show whether the model requested create_ticket, whether the application validated it, whether the API accepted it and what status came back. A conversational transcript alone cannot separate a fabricated action from a failed API or delayed transaction. The recovery path should consult the system of record before retrying, and an operator should be able to explain the failure from evidence.<\/p>\n After release, monitor latency, errors, safety signals, tool failure rates, retrieval freshness and unexpected behavior. Production data can reveal patterns absent in the test set, but real conversations may contain private information. Sample traces according to an approved data policy, scrub unnecessary fields and avoid copying confidential records into an unrestricted evaluation dataset. When monitoring detects a recurring issue, create a sanitized, reproducible regression case rather than repeatedly querying a customer’s private interaction.<\/p>\n Do not mistake a metric anomaly for its cause. Higher end-to-end latency might come from a slower model, an indexing backlog, a dependent API or a new multi-agent step. Break down time by span and correlate against deployment versions. An increase in failed retrieval could be caused by permissions changes that correctly restrict access or by a broken connector. The decision to alert, rollback or investigate should reflect the actual user consequence, not just a threshold crossed.<\/p>\n For a monitoring architecture that must retain audit evidence, consider the logging and retention decisions described in Azure logging architecture<\/a>. Application traces, platform diagnostics and long-term compliance records serve different purposes. Do not indefinitely retain every prompt merely to make debugging convenient. Define who owns redaction rules, the expiry schedule and the process for investigating a sensitive trace.<\/p>\n A safe candidate release starts by freezing the model, instructions, tool definitions, retrieval index version, runtime identity and evaluation dataset. Run deterministic format and permission checks, then broader quality and safety evaluations. Compare results with the accepted baseline and inspect changes in hard cases: current-vs-legacy manual conflicts, private source requests, ambiguous approvals and partial tool completion. The review should explicitly record what changed and why the new configuration is expected to improve a real defect.<\/p>\nConstruct a reusable evaluation dataset from real requirements<\/h2>\n
Use evaluators as measurements, not as unquestioned judges<\/h2>\n
Instrument the whole transaction with traces<\/h2>\n
Investigate production behavior without contaminating it<\/h2>\n
Build a release gate that cannot be gamed by averages<\/h2>\n