Microsoft AI-102 (Designing and Implementing a Microsoft Azure AI Solution) Practice Test 24

Question 230

An engineering team assumes model self-review eliminates the need for test cases. What is missing?

  1. Validate self-reviewed outputs against external ground truth and safety criteria
  2. Confirm production clients invoke the deployed fine-tuned model version
  3. Invalidate or version the retrieval cache when the indexed content changes
  4. Hold retrieval inputs and evaluation criteria consistent when isolating model differences

Explanation

Correct Answer

A:

A model can repeat or endorse its own mistakes; self-evaluation is not independent evidence. In the deployed enterprise chat and retrieval product, confirm the fix by retesting the affected behavior after validating self-reviewed outputs against external ground truth and safety criteria.

Incorrect Answers

B:

Training an adapted artifact does not alter a separate base-model deployment automatically in its own workflow, but that is a different correction from the present requirement to validate self-reviewed outputs against external ground truth and safety criteria. The reported issue would remain.

C:

Although useful elsewhere in the deployed enterprise chat and retrieval product, a stale application cache can mask successful index updates and keep outdated evidence in prompts; the present case is different and calls for the team to validate self-reviewed outputs against external ground truth and safety criteria.

D:

That change addresses another failure mode: comparable experiments change the intended variable while holding other significant conditions stable; it does not address this case, for which the relevant intervention is to validate self-reviewed outputs against external ground truth and safety criteria.

Question 231

A model tuned to reply in a new format shows no improvement in production. What rollout check matters most?

  1. Hold retrieval inputs and evaluation criteria consistent when isolating model differences
  2. Confirm production clients invoke the deployed fine-tuned model version
  3. Invalidate or version the retrieval cache when the indexed content changes
  4. Validate self-reviewed outputs against external ground truth and safety criteria

Explanation

Correct Answer

B:

Training an adapted artifact does not alter a separate base-model deployment automatically. In the deployed enterprise chat and retrieval product, confirm the fix by retesting the affected behavior after confirming production clients invoke the deployed fine-tuned model version.

Incorrect Answers

A:

Comparable experiments change the intended variable while holding other significant conditions stable in its own workflow, but that is a different correction from the present requirement to confirm production clients invoke the deployed fine-tuned model version. The reported issue would remain.

C:

That change addresses another failure mode: a stale application cache can mask successful index updates and keep outdated evidence in prompts; it does not address this case, for which the relevant intervention is to confirm production clients invoke the deployed fine-tuned model version.

D:

Although useful elsewhere in the deployed enterprise chat and retrieval product, a model can repeat or endorse its own mistakes; self-evaluation is not independent evidence; the present case is different and calls for the team to confirm production clients invoke the deployed fine-tuned model version.

Question 232

The team wants to compare groundedness across two model deployments, but one is tested on longer source passages. What must be controlled?

  1. Validate self-reviewed outputs against external ground truth and safety criteria
  2. Record sanitized tool arguments and outcomes linked to the relevant request trace
  3. Invalidate or version the retrieval cache when the indexed content changes
  4. Hold retrieval inputs and evaluation criteria consistent when isolating model differences

Explanation

Correct Answer

D:

Comparable experiments change the intended variable while holding other significant conditions stable. In the deployed enterprise chat and retrieval product, confirm the fix by retesting the affected behavior after holding retrieval inputs and evaluation criteria consistent when isolating model differences.

Incorrect Answers

A:

A model can repeat or endorse its own mistakes; self-evaluation is not independent evidence in its own workflow, but that is a different correction from the present requirement to hold retrieval inputs and evaluation criteria consistent when isolating model differences. The reported issue would remain.

B:

Although useful elsewhere in the deployed enterprise chat and retrieval product, privacy-conscious structured traces enable debugging without indiscriminately logging raw secrets; the present case is different and calls for the team to hold retrieval inputs and evaluation criteria consistent when isolating model differences.

C:

That change addresses another failure mode: a stale application cache can mask successful index updates and keep outdated evidence in prompts; it does not address this case, for which the relevant intervention is to hold retrieval inputs and evaluation criteria consistent when isolating model differences.

Question 233

A team proposes fine-tuning the assistant every morning with changing product prices. What is a better source of fresh facts?

  1. Evaluate with held-out queries and documents that represent production retrieval failures
  2. Trigger review selectively for outputs whose risk or complexity warrants its cost
  3. Retrieve the current authoritative price catalog at inference time
  4. Assign reliable correlation identifiers at request and conversation boundaries

Explanation

Correct Answer

C:

Parameter updates are inefficient and unreliable for frequently changing source facts compared with live grounding. In the deployed enterprise chat and retrieval product, confirm the fix by retesting the affected behavior after retrieving the current authoritative price catalog at inference time.

Incorrect Answers

A:

That change addresses another failure mode: realistic independent examples better measure generalization and avoid inflated retrieval metrics; it does not address this case, for which the relevant intervention is to retrieve the current authoritative price catalog at inference time.

B:

Conditional verification can control latency and spend while retaining checks on high-impact content in its own workflow, but that is a different correction from the present requirement to retrieve the current authoritative price catalog at inference time. The reported issue would remain.

D:

Although useful elsewhere in the deployed enterprise chat and retrieval product, correct trace correlation is required to reconstruct individual agent executions and avoid mixing users' actions; the present case is different and calls for the team to retrieve the current authoritative price catalog at inference time.

Question 234

In a deployed enterprise chat and retrieval product, engineers must diagnose latency and incorrect tool sequences during production conversations. The expected evidence is that investigators can see the request path and failing step. Which implementation is the best match?

  1. Instrument prompt and tool traces
  2. Use a model reflection or self-review pass
  3. Refresh retrieval grounding data
  4. Run representative regression evaluations

Explanation

Correct Answer

A:

Tracing records model calls, tool operations, and relevant workflow spans when configured. For this deployed enterprise chat and retrieval product, the intended evidence is that investigators can see the request path and failing step. It does not itself decide whether the model output is safe or accurate.

Incorrect Answers

B:

A reflection step asks the model or a separate evaluator to critique and revise a generated draft according to stated checks; it may be used to assess a draft response against explicit criteria before presenting the final answer in a deployed enterprise chat and retrieval product, but the current requirement is instead to diagnose latency and incorrect tool sequences during production conversations. That separate capability does not prove that investigators can see the request path and failing step; Self-critique may improve quality but does not guarantee truth or replace external evidence and evaluations, so selecting that alternative would not meet the stated acceptance check.

C:

Updating the indexed source corpus changes what the RAG retriever can supply on future queries; it may be used to make answers reflect updated internal policies without retraining the language model in a deployed enterprise chat and retrieval product, but the current requirement is instead to diagnose latency and incorrect tool sequences during production conversations. That separate capability does not prove that investigators can see the request path and failing step; It does not change the underlying model weights or solve a malformed JSON schema by itself, so selecting that alternative would not meet the stated acceptance check.

D:

Evaluations score responses against a sample dataset for qualities such as relevance, groundedness, or safety; it may be used to compare model or prompt changes against known expected behavior before releasing in a deployed enterprise chat and retrieval product, but the current requirement is instead to diagnose latency and incorrect tool sequences during production conversations. That separate capability does not prove that investigators can see the request path and failing step; A test score does not enforce real-time content filtering on future requests, so selecting that alternative would not meet the stated acceptance check.

Question 235

A team responsible for a deployed enterprise chat and retrieval product needs to adapt recurring domain-specific output style using approved example data and a supported tuning model; the acceptance check is that a separate tuned model artifact is evaluated before deployment. Which capability should be selected?

  1. Use a model reflection or self-review pass
  2. Fine-tune a supported language model
  3. Run representative regression evaluations
  4. Instrument prompt and tool traces

Explanation

Correct Answer

B:

Fine-tuning adjusts model behavior by training on examples rather than supplying the same context every turn. For this deployed enterprise chat and retrieval product, the intended evidence is that a separate tuned model artifact is evaluated before deployment. It is not the best means of keeping rapidly changing company policies current.

Incorrect Answers

A:

A reflection step asks the model or a separate evaluator to critique and revise a generated draft according to stated checks; it may be used to assess a draft response against explicit criteria before presenting the final answer in a deployed enterprise chat and retrieval product, but the current requirement is instead to adapt recurring domain-specific output style using approved example data and a supported tuning model. That separate capability does not prove that a separate tuned model artifact is evaluated before deployment; Self-critique may improve quality but does not guarantee truth or replace external evidence and evaluations, so selecting that alternative would not meet the stated acceptance check.

C:

Evaluations score responses against a sample dataset for qualities such as relevance, groundedness, or safety; it may be used to compare model or prompt changes against known expected behavior before releasing in a deployed enterprise chat and retrieval product, but the current requirement is instead to adapt recurring domain-specific output style using approved example data and a supported tuning model. That separate capability does not prove that a separate tuned model artifact is evaluated before deployment; A test score does not enforce real-time content filtering on future requests, so selecting that alternative would not meet the stated acceptance check.

D:

Tracing records model calls, tool operations, and relevant workflow spans when configured; it may be used to diagnose latency and incorrect tool sequences during production conversations in a deployed enterprise chat and retrieval product, but the current requirement is instead to adapt recurring domain-specific output style using approved example data and a supported tuning model. That separate capability does not prove that a separate tuned model artifact is evaluated before deployment; It does not itself decide whether the model output is safe or accurate, so selecting that alternative would not meet the stated acceptance check.

Question 236

Users report sporadic slow answers. The team knows total response time but cannot distinguish search versus generation. What should be added?

  1. Instrument correlated retrieval, prompt construction, model call, and tool spans
  2. Run a stable representative regression suite before promoting the prompt
  3. Verify that the chosen model supports tuning and prepare licensed high-quality example pairs
  4. Update and reindex the approved document corpus, including deletion or version invalidation

Explanation

Correct Answer

A:

Step-level traces attribute latency to actual pipeline stages and expose intermittent bottlenecks. In the deployed enterprise chat and retrieval product, confirm the fix by retesting the affected behavior after instrumenting correlated retrieval, prompt construction, model call, and tool spans.

Incorrect Answers

B:

A handful of successes cannot establish overall improvement or catch regressions across known scenarios in its own workflow, but that is a different correction from the present requirement to instrument correlated retrieval, prompt construction, model call, and tool spans. The reported issue would remain.

C:

Although useful elsewhere in the deployed enterprise chat and retrieval product, fine-tuning is not universally supported and requires curated examples appropriate to the target behavior; the present case is different and calls for the team to instrument correlated retrieval, prompt construction, model call, and tool spans.

D:

That change addresses another failure mode: grounding quality depends on current indexed evidence, not only model weights; it does not address this case, for which the relevant intervention is to instrument correlated retrieval, prompt construction, model call, and tool spans.

Question 237

A team enables detailed AI traces during a production incident. Traces may include confidential prompts and retrieved documents. Which TWO controls are appropriate? (Choose TWO.)

  1. Invalidate or version the retrieval cache when the indexed content changes
  2. Hold retrieval inputs and evaluation criteria consistent when isolating model differences
  3. Limit access to traces using a role-appropriate observability policy
  4. Redact or minimize confidential content in recorded trace attributes
  5. Validate self-reviewed outputs against external ground truth and safety criteria

Explanation

Correct Answers

C:

Telemetry can reveal personal and organizational information, so access to it should follow the same least-privilege principles as other sensitive data.

D:

Trace detail should support troubleshooting without unnecessarily copying sensitive user and source content into the logging store.

Incorrect Answers

A:

That change addresses another failure mode: a stale application cache can mask successful index updates and keep outdated evidence in prompts; it does not address this case, for which the relevant intervention is to redact or minimize confidential content in recorded trace attributes and limit access to traces using a role-appropriate observability policy.

B:

Although useful elsewhere in the deployed enterprise chat and retrieval product, comparable experiments change the intended variable while holding other significant conditions stable; the present case is different and calls for the team to redact or minimize confidential content in recorded trace attributes and limit access to traces using a role-appropriate observability policy.

E:

A model can repeat or endorse its own mistakes; self-evaluation is not independent evidence in its own workflow, but that is a different correction from the present requirement to redact or minimize confidential content in recorded trace attributes and limit access to traces using a role-appropriate observability policy. The reported issue would remain.

Question 238

Traces from different sessions are combined under one ID, making user journeys impossible to analyze. What instrumentation should change?

  1. Evaluate with held-out queries and documents that represent production retrieval failures
  2. Trigger review selectively for outputs whose risk or complexity warrants its cost
  3. Assign reliable correlation identifiers at request and conversation boundaries
  4. Break down quality metrics by relevant subgroups and inspect high-risk failures

Explanation

Correct Answer

C:

Correct trace correlation is required to reconstruct individual agent executions and avoid mixing users' actions. In the deployed enterprise chat and retrieval product, confirm the fix by retesting the affected behavior after assigning reliable correlation identifiers at request and conversation boundaries.

Incorrect Answers

A:

Realistic independent examples better measure generalization and avoid inflated retrieval metrics in its own workflow, but that is a different correction from the present requirement to assign reliable correlation identifiers at request and conversation boundaries. The reported issue would remain.

B:

That change addresses another failure mode: conditional verification can control latency and spend while retaining checks on high-impact content; it does not address this case, for which the relevant intervention is to assign reliable correlation identifiers at request and conversation boundaries.

D:

Although useful elsewhere in the deployed enterprise chat and retrieval product, averages conceal concentrated error patterns that matter for production suitability; the present case is different and calls for the team to assign reliable correlation identifiers at request and conversation boundaries.

Question 239

Evaluation shows good overall accuracy but serious failure on a regulated-language subset. What is the proper analysis?

  1. Trigger review selectively for outputs whose risk or complexity warrants its cost
  2. Break down quality metrics by relevant subgroups and inspect high-risk failures
  3. Assign reliable correlation identifiers at request and conversation boundaries
  4. Evaluate with held-out queries and documents that represent production retrieval failures

Explanation

Correct Answer

B:

Averages conceal concentrated error patterns that matter for production suitability. In the deployed enterprise chat and retrieval product, confirm the fix by retesting the affected behavior after breaking down quality metrics by relevant subgroups and inspect high-risk failures.

Incorrect Answers

A:

Conditional verification can control latency and spend while retaining checks on high-impact content in its own workflow, but that is a different correction from the present requirement to break down quality metrics by relevant subgroups and inspect high-risk failures. The reported issue would remain.

C:

Although useful elsewhere in the deployed enterprise chat and retrieval product, correct trace correlation is required to reconstruct individual agent executions and avoid mixing users' actions; the present case is different and calls for the team to break down quality metrics by relevant subgroups and inspect high-risk failures.

D:

That change addresses another failure mode: realistic independent examples better measure generalization and avoid inflated retrieval metrics; it does not address this case, for which the relevant intervention is to break down quality metrics by relevant subgroups and inspect high-risk failures.

Question 240

The review pass doubles latency on every request, even trivial greetings. What optimization fits?

  1. Evaluate with held-out queries and documents that represent production retrieval failures
  2. Retrieve the current authoritative price catalog at inference time
  3. Break down quality metrics by relevant subgroups and inspect high-risk failures
  4. Trigger review selectively for outputs whose risk or complexity warrants its cost

Explanation

Correct Answer

D:

Conditional verification can control latency and spend while retaining checks on high-impact content. In the deployed enterprise chat and retrieval product, confirm the fix by retesting the affected behavior after triggering review selectively for outputs whose risk or complexity warrants its cost.

Incorrect Answers

A:

Realistic independent examples better measure generalization and avoid inflated retrieval metrics in its own workflow, but that is a different correction from the present requirement to trigger review selectively for outputs whose risk or complexity warrants its cost. The reported issue would remain.

B:

That change addresses another failure mode: parameter updates are inefficient and unreliable for frequently changing source facts compared with live grounding; it does not address this case, for which the relevant intervention is to trigger review selectively for outputs whose risk or complexity warrants its cost.

C:

Although useful elsewhere in the deployed enterprise chat and retrieval product, averages conceal concentrated error patterns that matter for production suitability; the present case is different and calls for the team to trigger review selectively for outputs whose risk or complexity warrants its cost.

Designing and Implementing a Microsoft Azure AI Solution (AI-102) Practice Tests