Microsoft AI-103 (Developing AI Apps and Agents on Azure) Practice Test 27

Question 260

A user-acceptance test fails with the following observation: The writing agent receives raw salary records even though it needs only aggregate statistics. What is the best correction?

  1. Add a supervisor that gathers evidence, resolves disagreements, and approves final synthesis
  2. Pass sanitized aggregate outputs and enforce each agent's data entitlements
  3. Delimit external evidence and apply instruction-hierarchy and injection controls
  4. Classify request intent and dispatch to an authorized specialist with scoped capabilities

Explanation

Correct Answer

B:

The fault signal is that the writing agent receives raw salary records even though it needs only aggregate statistics; the targeted fix works because scoped messages and redacted handoff contracts minimize sensitive context propagation across multiple agents, restoring the ability to stop one agent from receiving private data that another agent needed only for its subtask.

Incorrect Answers

A:

If the failing function were to combine independent research subtasks into a coherent final response with oversight, this change could be relevant because a supervisor can assign independent tasks, collect outputs, and apply cross-checks before presenting an answer; the actual symptom is that the writing agent receives raw salary records even though it needs only aggregate statistics, which calls for a different repair.

C:

If the failing function were to keep untrusted customer-uploaded text from redefining the agent's permissions, this change could be relevant because trusted control instructions must have higher precedence than external content; retrieved pages are data, not agent policy; the actual symptom is that the writing agent receives raw salary records even though it needs only aggregate statistics, which calls for a different repair.

D:

If the failing function were to route finance, operations, and security questions to different bounded expert agents, this change could be relevant because a router delegates to an appropriate specialist with a narrow tool set; a single all-powerful agent is harder to govern; the actual symptom is that the writing agent receives raw salary records even though it needs only aggregate statistics, which calls for a different repair.

Question 261

The acceptance criteria for a customer-facing AI service require it to recover from a partially completed workflow when a later operation fails. Which design should be approved?

  1. Require approval before irreversible business actions
  2. Apply idempotency to write-capable tools
  3. Implement compensation for multi-step side effects
  4. Limit information passed between specialist agents

Explanation

Correct Answer

C:

For the stated requirement to recover from a partially completed workflow when a later operation fails, a compensating action undoes or offsets previously committed steps where a transaction cannot be atomic across services; this is the mechanism that the system must implement.

Incorrect Answers

A:

For a system whose objective is to prevent an autonomous agent from executing account closure without explicit review, a human approval checkpoint separates proposed actions from committed side effects and provides an auditable decision; this does not directly meet the required outcome to recover from a partially completed workflow when a later operation fails.

B:

For a system whose objective is to prevent double charging when an agent retries after a network timeout, an idempotency key ties a logical transaction to one effect even if the same request is retried; this does not directly meet the required outcome to recover from a partially completed workflow when a later operation fails.

D:

For a system whose objective is to stop one agent from receiving private data that another agent needed only for its subtask, scoped messages and redacted handoff contracts minimize sensitive context propagation across multiple agents; this does not directly meet the required outcome to recover from a partially completed workflow when a later operation fails.

Question 262

A design review proposes to analyze agent errors by failure class. Which measurable check would establish success on real inputs?

  1. Use a labeled error taxonomy and compare the distribution before and after each change
  2. Measure tool precision, forbidden-call rate, and unnecessary invocation count
  3. Locate the first failing step and measure time spent by workflow stage
  4. Measure field-level extraction accuracy on labeled scans and low-quality images

Explanation

Correct Answer

A:

The accepted change is to analyze agent errors by failure class; this evaluation checks whether it can prioritize corrections across retrieval misses, malformed tools, policy blocks, and model planning errors, since failure classification links observed symptoms to the component that can be improved; a single aggregate error rate obscures causes.

Incorrect Answers

B:

This test concerns the ability to detect unnecessary or dangerous tool calls even when its final prose looks plausible, for which tool-selection evaluation examines intermediate actions and authorization fit, not only the final natural-language answer; it does not establish whether the change to analyze agent errors by failure class works for the current release.

C:

This test concerns the ability to find where a lengthy agent plan diverges from intended workflow, for which per-step tracing captures model requests, tool parameters, statuses and transitions; a final response log hides the path taken; it does not establish whether the change to analyze agent errors by failure class works for the current release.

D:

This test concerns the ability to enable an agent to read key fields from photographed or scanned paperwork, for which document extraction tools convert visual text and layout into structured data suitable for subsequent agent reasoning; it does not establish whether the change to analyze agent errors by failure class works for the current release.

Question 263

Before approving the proposed action to implement compensation for multi-step side effects, which measurable acceptance check should be required?

  1. Force timeouts and confirm the payment ledger records one transaction
  2. Simulate unapproved requests and verify that no closure is committed
  3. Compare computed totals against independently verified reference calculations
  4. Inject mid-workflow failures and audit the final reconciled business state

Explanation

Correct Answer

D:

The accepted change is to implement compensation for multi-step side effects; this evaluation checks whether it can recover from a partially completed workflow when a later operation fails, since a compensating action undoes or offsets previously committed steps where a transaction cannot be atomic across services.

Incorrect Answers

A:

This test concerns the ability to prevent double charging when an agent retries after a network timeout, for which an idempotency key ties a logical transaction to one effect even if the same request is retried; it does not establish whether the change to implement compensation for multi-step side effects works for the current release.

B:

This test concerns the ability to prevent an autonomous agent from executing account closure without explicit review, for which a human approval checkpoint separates proposed actions from committed side effects and provides an auditable decision; it does not establish whether the change to implement compensation for multi-step side effects works for the current release.

C:

This test concerns the ability to let an agent aggregate a supplied numeric CSV where exact computation is important, for which a sandboxed execution tool performs calculations deterministically; executable code needs strict isolation and resource limits; it does not establish whether the change to implement compensation for multi-step side effects works for the current release.

Question 264

Stakeholders want a customer-facing AI service able to prevent double charging when an agent retries after a network timeout. Which choice implements that requirement?

  1. Define an agent's bounded role and objective
  2. Require approval before irreversible business actions
  3. Use a content-extraction tool for uploaded forms
  4. Implement compensation for multi-step side effects
  5. Apply idempotency to write-capable tools

Explanation

Correct Answer

E:

For the stated requirement to prevent double charging when an agent retries after a network timeout, an idempotency key ties a logical transaction to one effect even if the same request is retried; this is the mechanism that the system must implement.

Incorrect Answers

A:

For a system whose objective is to keep a support agent focused on diagnosing product issues rather than taking unrelated administrative action, a clear role and allowed-goal scope guide planning; enforceable permissions remain necessary at tool boundaries; this does not directly meet the required outcome to prevent double charging when an agent retries after a network timeout.

B:

For a system whose objective is to prevent an autonomous agent from executing account closure without explicit review, a human approval checkpoint separates proposed actions from committed side effects and provides an auditable decision; this does not directly meet the required outcome to prevent double charging when an agent retries after a network timeout.

C:

For a system whose objective is to enable an agent to read key fields from photographed or scanned paperwork, document extraction tools convert visual text and layout into structured data suitable for subsequent agent reasoning; this does not directly meet the required outcome to prevent double charging when an agent retries after a network timeout.

D:

For a system whose objective is to recover from a partially completed workflow when a later operation fails, a compensating action undoes or offsets previously committed steps where a transaction cannot be atomic across services; this does not directly meet the required outcome to prevent double charging when an agent retries after a network timeout.

Question 265

Production telemetry reveals this failure: An agent gives the right balance but unnecessarily calls a sensitive customer-export tool to obtain it. Which change most directly corrects it?

  1. Score chosen tool calls against permitted expected action traces
  2. Carry relevant prior tool results and user selections in session-scoped conversation state
  3. Enable correlated tracing across decision and tool spans with appropriate redaction
  4. Evaluate external side effects and final artifacts against labeled target states

Explanation

Correct Answer

A:

The fault signal is that an agent gives the right balance but unnecessarily calls a sensitive customer-export tool to obtain it; the targeted fix works because tool-selection evaluation examines intermediate actions and authorization fit, not only the final natural-language answer, restoring the ability to detect unnecessary or dangerous tool calls even when its final prose looks plausible.

Incorrect Answers

B:

If the failing function were to resolve 'those two records' by retaining the preceding selection context, this change could be relevant because conversation tracking gives referential continuity across turns, but the context should be bounded to the correct user session; the actual symptom is that an agent gives the right balance but unnecessarily calls a sensitive customer-export tool to obtain it, which calls for a different repair.

C:

If the failing function were to find where a lengthy agent plan diverges from intended workflow, this change could be relevant because per-step tracing captures model requests, tool parameters, statuses and transitions; a final response log hides the path taken; the actual symptom is that an agent gives the right balance but unnecessarily calls a sensitive customer-export tool to obtain it, which calls for a different repair.

D:

If the failing function were to determine whether the agent actually achieved a multi-step business goal, this change could be relevant because task-success tests compare final system state with expected outcomes; prose fluency does not prove work was completed; the actual symptom is that an agent gives the right balance but unnecessarily calls a sensitive customer-export tool to obtain it, which calls for a different repair.

Question 266

A production AI solution must consistently reduce unwarranted variability when an output must follow a fixed decision rubric. Which architecture decision is most defensible?

  1. Use explicit system and developer instructions for policy
  2. Lower sampling temperature for stable classifications
  3. Use explicit few-shot examples for format nuance
  4. Cache retrieved context with source versioning

Explanation

Correct Answer

B:

For the stated requirement to reduce unwarranted variability when an output must follow a fixed decision rubric, lower temperature can reduce sampling randomness where supported; it does not guarantee factual accuracy or replace schema validation; this is the mechanism that the system must implement.

Incorrect Answers

A:

For a system whose objective is to keep permanent application behavior separate from ordinary user requests, high-priority instructions establish the app's role and safety boundaries, but external authorization still requires programmatic enforcement; this does not directly meet the required outcome to reduce unwarranted variability when an output must follow a fixed decision rubric.

C:

For a system whose objective is to teach a model a specialized labeling convention that general instructions describe poorly, well-chosen examples illustrate the desired input-output mapping, including ambiguous edge cases; they must be representative; this does not directly meet the required outcome to reduce unwarranted variability when an output must follow a fixed decision rubric.

D:

For a system whose objective is to avoid repeated retrieval for unchanged documents while preventing stale grounding, versioned cache keys or invalidation preserve freshness when approved source content changes; this does not directly meet the required outcome to reduce unwarranted variability when an output must follow a fixed decision rubric.

Question 267

A customer-facing AI service is being scoped for production. The non-negotiable task is to prevent confident unsupported answers when a retrieval score or evidence threshold is weak. Which implementation fits?

  1. Batch independent embeddings for ingestion efficiency
  2. Constrain response length for concise summaries
  3. Use uncertainty-aware abstention
  4. Trim redundant conversation context

Explanation

Correct Answer

C:

For the stated requirement to prevent confident unsupported answers when a retrieval score or evidence threshold is weak, a calibrated no-answer path is safer than fabricating a missing fact; thresholds should be tuned on labeled questions; this is the mechanism that the system must implement.

Incorrect Answers

A:

For a system whose objective is to lower overhead when indexing many small documents without changing semantic meaning, batching compatible embedding requests can improve throughput within service rate limits; errors require granular retries; this does not directly meet the required outcome to prevent confident unsupported answers when a retrieval score or evidence threshold is weak.

B:

For a system whose objective is to avoid long generated replies when a client requires a short summary or fixed output budget, a sensible output-token cap and concise instruction bound generation size, but a cap that is too small may truncate content; this does not directly meet the required outcome to prevent confident unsupported answers when a retrieval score or evidence threshold is weak.

D:

For a system whose objective is to reduce token cost in long sessions while keeping current commitments and unresolved tasks, selective compression retains required state but excludes repeated old messages; simply raising token limits does not fix unnecessary spend; this does not directly meet the required outcome to prevent confident unsupported answers when a retrieval score or evidence threshold is weak.

Question 268

Engineers have implemented a change to use model routing by task complexity. What validation would expose a remaining defect?

  1. Evaluate task accuracy, p95 latency, and cost per successful output
  2. Inject rule-violating JSON fixtures and verify every violation is caught
  3. Measure task pass rate, spend, and latency per routing class
  4. Compare p95 latency by span and count budget overruns per stage

Explanation

Correct Answer

C:

The accepted change is to use model routing by task complexity; this evaluation checks whether it can send bounded simple queries to economical models and harder reasoning tasks to stronger ones, since a routing policy can select among models based on evaluated task difficulty and quality constraints.

Incorrect Answers

A:

This test concerns the ability to choose between smaller and larger models based on business quality and cost, for which representative evaluations reveal the trade-off among quality, latency and spend; model size is an imperfect proxy; it does not establish whether the change to use model routing by task complexity works for the current release.

B:

This test concerns the ability to catch schema-valid yet impossible business outputs before a workflow proceeds, for which a type-correct JSON value may still violate business invariants; deterministic post-validation enforces the rules; it does not establish whether the change to use model routing by task complexity works for the current release.

D:

This test concerns the ability to identify which dependent stage to optimize in a slow multi-step AI workflow, for which stage-specific latency budgets reveal the true critical path across inference, retrieval and tools; it does not establish whether the change to use model routing by task complexity works for the current release.

Question 269

During a regression test, the following unexpected result occurs: End-to-end timeouts rise, but engineers only measure the combined request duration. What should the engineer do?

  1. Implement a supported no-evidence response and escalation route for low-support cases
  2. Instrument separate deadlines and span timers for each major stage
  3. Tie cached contexts to source revision and invalidate on indexed-content changes
  4. Run a business-rule validator after schema checks and reject impossible state transitions

Explanation

Correct Answer

B:

The fault signal is that end-to-end timeouts rise, but engineers only measure the combined request duration; the targeted fix works because stage-specific latency budgets reveal the true critical path across inference, retrieval and tools, restoring the ability to identify which dependent stage to optimize in a slow multi-step AI workflow.

Incorrect Answers

A:

If the failing function were to prevent confident unsupported answers when a retrieval score or evidence threshold is weak, this change could be relevant because a calibrated no-answer path is safer than fabricating a missing fact; thresholds should be tuned on labeled questions; the actual symptom is that end-to-end timeouts rise, but engineers only measure the combined request duration, which calls for a different repair.

C:

If the failing function were to avoid repeated retrieval for unchanged documents while preventing stale grounding, this change could be relevant because versioned cache keys or invalidation preserve freshness when approved source content changes; the actual symptom is that end-to-end timeouts rise, but engineers only measure the combined request duration, which calls for a different repair.

D:

If the failing function were to catch schema-valid yet impossible business outputs before a workflow proceeds, this change could be relevant because a type-correct JSON value may still violate business invariants; deterministic post-validation enforces the rules; the actual symptom is that end-to-end timeouts rise, but engineers only measure the combined request duration, which calls for a different repair.

Question 270

Before approving the proposed action to batch independent embeddings for ingestion efficiency, which measurable acceptance check should be required?

  1. Evaluate correct abstention versus unnecessary refusal on labeled in-scope and out-of-scope inputs
  2. Track false-positive rate together with harmful-output escape rate during tuning
  3. Measure indexed chunks per minute, per-item failures, and sustained quota usage
  4. Check stale-citation incidence and cache hits after source updates

Explanation

Correct Answer

C:

The accepted change is to batch independent embeddings for ingestion efficiency; this evaluation checks whether it can lower overhead when indexing many small documents without changing semantic meaning, since batching compatible embedding requests can improve throughput within service rate limits; errors require granular retries.

Incorrect Answers

A:

This test concerns the ability to prevent confident unsupported answers when a retrieval score or evidence threshold is weak, for which a calibrated no-answer path is safer than fabricating a missing fact; thresholds should be tuned on labeled questions; it does not establish whether the change to batch independent embeddings for ingestion efficiency works for the current release.

B:

This test concerns the ability to reduce legitimate request blocks without weakening essential guardrails, for which precision and recall of safety classifications require policy-labeled evaluation rather than a blanket filter disablement; it does not establish whether the change to batch independent embeddings for ingestion efficiency works for the current release.

D:

This test concerns the ability to avoid repeated retrieval for unchanged documents while preventing stale grounding, for which versioned cache keys or invalidation preserve freshness when approved source content changes; it does not establish whether the change to batch independent embeddings for ingestion efficiency works for the current release.

Developing AI Apps and Agents on Azure (AI-103) Practice Tests