{"id":4109,"date":"2026-10-12T17:53:41","date_gmt":"2026-10-12T17:53:41","guid":{"rendered":"https:\/\/www.examtopics.biz\/blog\/ab-100-foundry-model-router-small-model-prompts\/"},"modified":"2026-10-16T20:41:04","modified_gmt":"2026-10-16T20:41:04","slug":"ab-100-foundry-model-router-small-model-prompts","status":"publish","type":"post","link":"https:\/\/www.examtopics.biz\/blog\/ab-100-foundry-model-router-small-model-prompts\/","title":{"rendered":"Microsoft AB-100: Foundry Model Routing and Prompt Governance"},"content":{"rendered":"

The appropriate model for a business task depends on the measurable problem, not the most impressive model name. A short extraction job may be served well by a small model with strict schema validation, while a conflicting-policy investigation could require richer reasoning and additional verified sources. Routing requests between deployments changes the evaluation boundary: the team must inspect not only per-call token cost but also retries, latency, incorrect classifications, reviewer effort and the cost of a wrong business action. Build a holdout of simple, ambiguous and unauthorized requests and compare accepted task completions under a stable configuration. A model that is cheap but produces a confident unsupported approval can cost more per correct result than a cautious alternative. A useful independent AB-100 model-selection practice<\/a> scenario asks whether a proposed routing policy preserves data permissions, fallback authority and cost per correctly resolved case; it cannot verify the candidate\u2019s own Foundry configuration.<\/p>\n

Imagine a multinational warranty organization. Most incoming requests ask for a policy category or a structured extraction from a short claim. A smaller number require reasoning across repair history, product eligibility and geographic warranty terms. Treating every request as the hardest case increases costs and may create unnecessary latency. Treating every request as the easiest case may produce confident but wrong decisions. The useful goal is an explicit routing policy backed by evaluations and a safe exception path.<\/p>\n

Classify the work before selecting an endpoint<\/h2>\n

Start by separating request types, not by scoring vendors. A policy-labeling task has a constrained output and may be evaluated against labeled cases. A document extraction task needs field-level accuracy and a mechanism for reporting missing evidence. A complex claims question may need retrieval, comparisons and tool calls. A conversational explanation requires clarity but should not be allowed to issue refunds. Each task has a different acceptable error and latency envelope.<\/p>\n

Create an evaluation set<\/a> that preserves these distinctions: straightforward warranty categories, ambiguous jurisdictions, missing contract dates, an unreadable attachment, a customer requesting an unauthorized adjustment, and conflicting policy versions. Record what the system should do, including legitimate abstentions and handoffs. Model choice is only meaningful relative to these observable outcomes.<\/p>\n

After a business-agent build-or-buy and return-on-investment<\/a> decision sets the scope and acceptable operating cost, the architect still needs to classify which requests should reach which model. A routing policy should expose the task category, permissible data, fallback and required validation, rather than silently substitute a cheaper model whenever the system is busy.<\/p>\n

Use Foundry model routing as an implementation option, not a quality guarantee<\/h2>\n

Microsoft documents a Foundry model router<\/a> that directs compatible requests among supported underlying models. This can be useful when workload types and model economics vary, but it does not absolve the application of responsibility for output correctness, confidentiality or approvals. Availability, supported model lists, regions, API interfaces and versions change. As of the referenced Microsoft guidance, many supported models need no separate deployment for router use, but selected Claude models must be deployed before they can be included; verify the actual account, region, permitted subset and API contract rather than copying a static inventory into an operational rule.<\/p>\n

A router can make a good choice according to its available signals and still choose a model unsuited to a hidden business requirement. If a claims case involves legal-language nuance that the router cannot infer from the input, the application may need a separate classification step or explicit routing metadata. A model router should not be permitted to silently cross a data-residency boundary or send confidential information to an endpoint that is outside the authorized model set. Those controls belong in platform and application policy.<\/p>\n

Record the chosen route, model or deployment version when returned, request class, latency and cost signals, output schema version, and any exception state. Do not log raw protected claim details merely to explain routing. Without versioned telemetry, a regression after a model update becomes difficult to reproduce. Also distinguish a router that uses preexisting supported deployments from product-specific deployment requirements. The current Microsoft documentation explains where separate model deployment may still be needed.<\/p>\n

Determine when a small or specialized model is the better tool<\/h2>\n

A small language model or task-specialized model may be appropriate for a narrow, high-volume workload with stable labels, while a general-purpose model may be needed for nuanced explanations or multi-document reasoning. But \u201csmaller\u201d is not itself evidence of acceptable accuracy. Test for false acceptance of invalid cases, sensitivity to input format, rare exceptions, multilingual wording and missing-field behavior. Hardware, hosting, licensing, support and lifecycle constraints also affect whether a specialized model is economical.<\/p>\n

Suppose a warranty claim arrives with a product code, purchase date, service code and country. A deterministic rules engine can enforce the formal warranty window. A smaller classifier might identify the claim category. A retrieval component finds the currently applicable policy. A more capable model can explain an unusual exception in natural language without authorizing a payment. Splitting these responsibilities can reduce token spend and improve testability, but it increases integration and versioning responsibilities.<\/p>\n

Fine-tuning should not be the default answer to outdated facts. If a policy changes weekly, update the governed knowledge source. If a schema has changed, fix the consuming code and validation. Consider customization when you can state what adaptation is needed, maintain representative training\/evaluation data lawfully, and show that a simpler prompt, retrieval or post-processing approach fails to meet measured requirements. Always separate training data permission from a caller’s permission to retrieve operational records.<\/p>\n

Treat a prompt library as versioned product configuration<\/h2>\n

An enterprise prompt library is more than a collection of cleverly worded examples. It defines allowed tasks, expected input fields, instructions that constrain output, version ownership, and tests demonstrating what breaks when the prompt changes. The AB-100 skills map explicitly calls for prompt-library guidance; Microsoft also documents packaging prompts as solution components<\/a> for environment movement in applicable product workflows.<\/p>\n

For the warranty workflow, a claims-summary prompt might take claim_reference<\/code>, policy_evidence<\/code>, service_findings<\/code>, locale<\/code> and a structured allowed outcome set. The instructions must distinguish evidence from customer statements. It may describe a draft recommendation, but a backend service makes the actual entitlement decision. A useful negative example asks the model to override an expired policy based on a persuasive customer email; expected behavior is to identify the limitation, not comply.<\/p>\n

Use semantic versions or another deliberate naming convention for changes: a minor change might improve wording without altering the schema; a major change changes output fields or decision semantics and requires downstream compatibility tests. A shared prompt cannot be deployed independently of the application that interprets its output. The release manifest should associate prompt revision, model\/deployment version, retrieval index or data snapshot, application build and evaluator dataset revision.<\/p>\n

Prefer schema and business-rule validation to fluent confidence<\/h2>\n

A model output that parses as JSON can still be factually wrong. Consider warranty_eligible=true<\/code> generated from a missing purchase date. Syntactic validity tells us little about the eligibility decision. Validate required fields, date formats and known enums, then compare the proposed result with authoritative system records and policy rules. When the model cannot supply evidence, accept an explicit unknown state rather than coercing a guessed true\/false value.<\/p>\n

Build tests around the boundaries: conflicting product identifiers, expired dates, copied policy text from an untrusted customer attachment, malicious instructions embedded in a retrieved document, and a locale the selected model handles poorly. A model-selection experiment should not count a confident false entitlement as a success merely because the explanatory prose is persuasive.<\/p>\n

A practical metric set includes structured-field precision and recall on labeled cases, supported-answer rate, unauthorized-action count, abstention appropriateness, p50\/p95 latency and cost per correctly resolved legitimate claim. Avoid averaging away a high-severity failure: one unauthorized refund is not offset by dozens of good summaries. The evaluation set should be stable enough for regression checks while being refreshed when policy, model capabilities or workload distribution changes.<\/p>\n

Design exceptions and fallback paths that do not change authority<\/h2>\n

When the economical route fails validation, the system may re-run a better-suited model using the same allowed evidence. Such a retry is not permission to change tool scope or include private data previously excluded. A fallback model must be an approved deployment with adequate regional, contractual and technical controls. If no permitted model can handle the case, route it to manual review with the available evidence and explicit uncertainties.<\/p>\n

A safe result object for the claims workflow could be classification<\/code>, confidence_band<\/code>, source_ids<\/code>, policy_version<\/code>, validation_state<\/code>, review_reason<\/code> and correlation_id<\/code>. Confidence bands are descriptive diagnostics, not authorization. An external refund function should require independent authenticated authorization and a durable case state. The AI component may recommend review; it does not decide who can change a financial record.<\/p>\n

Protect against loops. If the first extraction lacks an invoice date, repeatedly asking similar prompts on the same scan may produce varied guesses, not the missing fact. Stop according to a documented retry budget, require better evidence, or escalate. Track the costs of failed and retried requests in the same total-cost model as successful requests.<\/p>\n

Compare operating cost and accuracy using a controlled holdout<\/h2>\n

Do not choose a route by one attractive benchmark. Collect a bounded sample of real, permission-cleared task shapes, and keep a held-out subset that developers cannot continually tune against. For each candidate, measure business-accepted outcomes, reviewer workload, escalations, tool latency, token use, inference cost, infrastructure costs and the cost of correcting errors. Total cost of ownership also includes observability, security review, support, model change management and specialized data maintenance.<\/p>\n

For illustration, an organization might calculate cost per properly classified claim rather than cost per thousand tokens. A cheaper model that produces more manual corrections may be more expensive at the business-process level. Conversely, the expensive route may be unnecessary for a narrow classification whose correct output is established by a few authoritative fields. Neither conclusion is automatic; the evaluation results decide.<\/p>\n

Document what changed between comparisons. If a second model receives a richer retrieval context, the experiment no longer isolates model quality. If a router is enabled but the prompts also change, attribute the observed improvement carefully. Business stakeholders need evidence that the selected configuration resolves the actual problem, not a claim that a newer model name represents progress.<\/p>\n

Establish acceptance and maintenance responsibilities<\/h2>\n

The final architecture decision should name the owners for router configuration, approved model catalog, prompt-library revisions, model evaluations, source data permissions, and incident response. Define release checks for output validity, accuracy on critical cases, authorized regional routing, known unsupported inputs, rollback and traceability. Revalidate the live Foundry platform capabilities before a deployment: model lists and product behavior may differ from the documented snapshot used in this article.<\/p>\n

A carefully chosen model stack is deliberately unglamorous: deterministic services own business rules, retrieval owns governed evidence, models own bounded interpretation and explanation, and the release process owns regression tests. That division makes model routing and customization relevant to AB-100 without mistaking model fluency for enterprise authority. The warranty example is a fictional design exercise; no Foundry deployment or real customer claim was executed for this article.<\/p>\n","protected":false},"excerpt":{"rendered":"

The appropriate model for a business task depends on the measurable problem, not the most impressive model name. A short extraction job may be served […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[29,47],"tags":[],"class_list":["post-4109","post","type-post","status-publish","format-standard","hentry","category-certifications","category-microsoft-certifications"],"_links":{"self":[{"href":"https:\/\/www.examtopics.biz\/blog\/wp-json\/wp\/v2\/posts\/4109","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.examtopics.biz\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.examtopics.biz\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.examtopics.biz\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.examtopics.biz\/blog\/wp-json\/wp\/v2\/comments?post=4109"}],"version-history":[{"count":5,"href":"https:\/\/www.examtopics.biz\/blog\/wp-json\/wp\/v2\/posts\/4109\/revisions"}],"predecessor-version":[{"id":4633,"href":"https:\/\/www.examtopics.biz\/blog\/wp-json\/wp\/v2\/posts\/4109\/revisions\/4633"}],"wp:attachment":[{"href":"https:\/\/www.examtopics.biz\/blog\/wp-json\/wp\/v2\/media?parent=4109"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.examtopics.biz\/blog\/wp-json\/wp\/v2\/categories?post=4109"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.examtopics.biz\/blog\/wp-json\/wp\/v2\/tags?post=4109"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}