Question 100
A user-acceptance test fails with the following observation: Customer complaints rise after a model refresh but traces do not identify which deployment revision produced each answer. What is the best correction?
- Align query embeddings with index embeddings and re-embed stored content when migrating models
- Tag outputs by model and prompt version and run side-by-side regression cases
- Cache validated FAQ outputs by normalized question and content version with proper expiration
- Instrument each tool call with duration, status, and bounded retry decisions
Explanation
Correct Answer
B:The fault signal is that customer complaints rise after a model refresh but traces do not identify which deployment revision produced each answer; the targeted fix works because version-tagged test results separate model changes from request-mix changes and allow a reliable rollback decision, restoring the ability to detect regression when a newly deployed model changes how it answers the same business questions.
Incorrect Answers
A:If the failing function were to catch incompatible document and query vectors when retrieval quality suddenly collapses, this change could be relevant because embedding dimensionality and model compatibility must be maintained between indexed passages and query vectors; vector space changes require a coordinated migration; the actual symptom is that customer complaints rise after a model refresh but traces do not identify which deployment revision produced each answer, which calls for a different repair.
C:If the failing function were to avoid repeating the same approved reference answer for unchanged public content, this change could be relevant because an application cache can reduce repeated inference requests when answers and policy versions are safe to reuse; invalidation and access scope remain essential; the actual symptom is that customer complaints rise after a model refresh but traces do not identify which deployment revision produced each answer, which calls for a different repair.
D:If the failing function were to find why agents fail to complete workflows despite healthy model endpoints, this change could be relevant because tool telemetry records HTTP failures, permission denials and timeouts at action boundaries; model token metrics will not diagnose a downstream API error; the actual symptom is that customer complaints rise after a model refresh but traces do not identify which deployment revision produced each answer, which calls for a different repair.
Question 101
A system now includes a component to measure groundedness with cited source evidence. Which evaluation should accompany the rollout?
- Score unsupported claim rate and citation fidelity on a curated RAG test set
- Confirm no requests use the retired credential and access continues during rotation
- Track policy category, severity, false-positive review, and block rate by version
- Compare indexed versions and document counts with the authoritative document repository
Explanation
Correct Answer
A:The accepted change is to measure groundedness with cited source evidence; this evaluation checks whether it can detect fluent answers that introduce claims unsupported by retrieved passages, since groundedness evaluations compare generated claims with supplied evidence; fluency or user satisfaction alone cannot establish factual support.
Incorrect Answers
B:This test concerns the ability to replace an exposed secret while keeping a production inference endpoint available, for which credential rollover with overlapping validity and centralized secret distribution enables staged updates; longer-term managed identity can reduce secret use; it does not establish whether the change to measure groundedness with cited source evidence works for the current release.
C:This test concerns the ability to identify spikes in unsafe or blocked prompts and unsafe generated material, for which guardrail and content-filter telemetry reveals policy-related events; it should be segmented without exposing sensitive payloads unnecessarily; it does not establish whether the change to measure groundedness with cited source evidence works for the current release.
D:This test concerns the ability to detect when newly approved source documents are not available to the grounding pipeline, for which indexer execution status, document counts, and source timestamps show whether retrieval data is current; a good model response cannot compensate for missing content; it does not establish whether the change to measure groundedness with cited source evidence works for the current release.
Question 102
A pilot has a functional gap: Most tokens in every call come from irrelevant retrieved passages appended to a short question. How should the team address that gap?
- Implement bounded retry-after-aware backoff with jitter and limit client concurrency
- Trim and rerank context, bound retrieval count, and retain only task-relevant evidence
- Introduce a new credential through a secret store, roll replicas, then revoke the exposed credential
- Cache validated FAQ outputs by normalized question and content version with proper expiration
Explanation
Correct Answer
B:The fault signal is that most tokens in every call come from irrelevant retrieved passages appended to a short question; the targeted fix works because input tokens contribute to context limits and usage costs; pruning irrelevant evidence can improve efficiency without changing the model deployment, restoring the ability to reduce inference spend caused by oversized repeated document context.
Incorrect Answers
A:If the failing function were to recover from temporary HTTP 429 responses without amplifying load on the model deployment, this change could be relevant because exponential backoff with jitter and respect for retry hints reduces synchronized retries; extra retries cannot replace insufficient sustained capacity; the actual symptom is that most tokens in every call come from irrelevant retrieved passages appended to a short question, which calls for a different repair.
C:If the failing function were to replace an exposed secret while keeping a production inference endpoint available, this change could be relevant because credential rollover with overlapping validity and centralized secret distribution enables staged updates; longer-term managed identity can reduce secret use; the actual symptom is that most tokens in every call come from irrelevant retrieved passages appended to a short question, which calls for a different repair.
D:If the failing function were to avoid repeating the same approved reference answer for unchanged public content, this change could be relevant because an application cache can reduce repeated inference requests when answers and policy versions are safe to reuse; invalidation and access scope remain essential; the actual symptom is that most tokens in every call come from irrelevant retrieved passages appended to a short question, which calls for a different repair.
Question 103
The support ticket describes this behavior: A long-running agent stays accurate but inference expense accelerates sharply after many follow-up turns. Which engineering change should come first?
- Track input and output tokens per step and trim stale context without dropping required facts
- Trim and rerank context, bound retrieval count, and retain only task-relevant evidence
- Instrument each tool call with duration, status, and bounded retry decisions
- Align query embeddings with index embeddings and re-embed stored content when migrating models
Explanation
Correct Answer
A:The fault signal is that a long-running agent stays accurate but inference expense accelerates sharply after many follow-up turns; the targeted fix works because per-turn token accounting reveals input-context inflation; controlled summarization or trimming can reduce cost while preserving vital state, restoring the ability to identify waste when conversation history and tool outputs grow over successive exchanges.
Incorrect Answers
B:If the failing function were to reduce inference spend caused by oversized repeated document context, this change could be relevant because input tokens contribute to context limits and usage costs; pruning irrelevant evidence can improve efficiency without changing the model deployment; the actual symptom is that a long-running agent stays accurate but inference expense accelerates sharply after many follow-up turns, which calls for a different repair.
C:If the failing function were to find why agents fail to complete workflows despite healthy model endpoints, this change could be relevant because tool telemetry records HTTP failures, permission denials and timeouts at action boundaries; model token metrics will not diagnose a downstream API error; the actual symptom is that a long-running agent stays accurate but inference expense accelerates sharply after many follow-up turns, which calls for a different repair.
D:If the failing function were to catch incompatible document and query vectors when retrieval quality suddenly collapses, this change could be relevant because embedding dimensionality and model compatibility must be maintained between indexed passages and query vectors; vector space changes require a coordinated migration; the actual symptom is that a long-running agent stays accurate but inference expense accelerates sharply after many follow-up turns, which calls for a different repair.
Question 104
Engineers have implemented a change to use caching for reusable deterministic outputs. What validation would expose a remaining defect?
- Measure cache-hit ratio, stale-answer rate, and token savings after invalidation tests
- Compare sustained token throughput and throttling against the new allocation
- Track 429 incidence, retry amplification, and successful completion time under load
- Compare indexed versions and document counts with the authoritative document repository
Explanation
Correct Answer
A:The accepted change is to use caching for reusable deterministic outputs; this evaluation checks whether it can avoid repeating the same approved reference answer for unchanged public content, since an application cache can reduce repeated inference requests when answers and policy versions are safe to reuse; invalidation and access scope remain essential.
Incorrect Answers
B:This test concerns the ability to serve a stable higher request rate when the deployment repeatedly reaches its assigned token budget, for which tokens-per-minute and request limits constrain throughput; allocation or capacity decisions must match the deployment's actual measured demand; it does not establish whether the change to use caching for reusable deterministic outputs works for the current release.
C:This test concerns the ability to recover from temporary HTTP 429 responses without amplifying load on the model deployment, for which exponential backoff with jitter and respect for retry hints reduces synchronized retries; extra retries cannot replace insufficient sustained capacity; it does not establish whether the change to use caching for reusable deterministic outputs works for the current release.
D:This test concerns the ability to detect when newly approved source documents are not available to the grounding pipeline, for which indexer execution status, document counts, and source timestamps show whether retrieval data is current; a good model response cannot compensate for missing content; it does not establish whether the change to use caching for reusable deterministic outputs works for the current release.
Question 105
A user-acceptance test fails with the following observation: End-to-end latency rises but operators cannot tell whether search or generation consumes the extra time. What is the best correction?
- Run source-grounded evaluations and inspect failing claim-to-passage alignments
- Collect and classify guardrail outcomes with privacy-aware security telemetry
- Tag outputs by model and prompt version and run side-by-side regression cases
- Add correlated request tracing across retrieval, tools, and model calls
Explanation
Correct Answer
D:The fault signal is that end-to-end latency rises but operators cannot tell whether search or generation consumes the extra time; the targeted fix works because distributed traces expose timing and relationships among retrieval, tool and inference spans; aggregate CPU graphs alone cannot isolate these steps, restoring the ability to locate whether a slow RAG response is delayed by search, tool calls, or model generation.
Incorrect Answers
A:If the failing function were to detect fluent answers that introduce claims unsupported by retrieved passages, this change could be relevant because groundedness evaluations compare generated claims with supplied evidence; fluency or user satisfaction alone cannot establish factual support; the actual symptom is that end-to-end latency rises but operators cannot tell whether search or generation consumes the extra time, which calls for a different repair.
B:If the failing function were to identify spikes in unsafe or blocked prompts and unsafe generated material, this change could be relevant because guardrail and content-filter telemetry reveals policy-related events; it should be segmented without exposing sensitive payloads unnecessarily; the actual symptom is that end-to-end latency rises but operators cannot tell whether search or generation consumes the extra time, which calls for a different repair.
C:If the failing function were to detect regression when a newly deployed model changes how it answers the same business questions, this change could be relevant because version-tagged test results separate model changes from request-mix changes and allow a reliable rollback decision; the actual symptom is that end-to-end latency rises but operators cannot tell whether search or generation consumes the extra time, which calls for a different repair.
Question 106
The planned deployment includes a mechanism to separate review and execution responsibilities. Which outcome should the QA team measure?
- Test cross-role queries for unauthorized citations and verify least-privilege results
- Test that execution is blocked without approval and that every approval is attributable
- Audit that proposer and approver identities are distinct on completed decisions
- Replay a labeled policy suite against old and new guardrail versions
Explanation
Correct Answer
C:The accepted change is to separate review and execution responsibilities; this evaluation checks whether it can stop an agent from approving its own high-risk recommendation, since distinct approval identity or review service introduces independent oversight; a second prompt in the same privileged agent is not separation of duties.
Incorrect Answers
A:This test concerns the ability to prevent an agent from citing restricted HR content to ordinary staff, for which security trimming or permission-aware retrieval applies user access rules before private passages reach generation; it does not establish whether the change to separate review and execution responsibilities works for the current release.
B:This test concerns the ability to keep payment or account-changing tool calls under an explicit human authorization step, for which human-in-the-loop approval gates restrict irreversible operations even if the agent's reasoning or external evidence is wrong; it does not establish whether the change to separate review and execution responsibilities works for the current release.
D:This test concerns the ability to explain changes in moderation behavior across releases and support rollback, for which versioned guardrail policies make thresholds, blocklists and exception decisions reproducible in evaluations and incidents; it does not establish whether the change to separate review and execution responsibilities works for the current release.
Question 107
A cloud engineering team needs its AI workflow to discover harmful outputs and instruction-bypass failures across diverse user and tool inputs. What technical approach most directly enables that?
- Limit knowledge access to the caller's entitlements
- Use an adversarial safety test suite before release
- Enforce per-tool authorization independently
- Record provenance for retrieved evidence
Explanation
Correct Answer
B:For the stated requirement to discover harmful outputs and instruction-bypass failures across diverse user and tool inputs, red-team cases exercise guardrail boundaries under realistic attacks, unlike happy-path functional tests alone; this is the mechanism that the system must implement.
Incorrect Answers
A:For a system whose objective is to prevent an agent from citing restricted HR content to ordinary staff, security trimming or permission-aware retrieval applies user access rules before private passages reach generation; this does not directly meet the required outcome to discover harmful outputs and instruction-bypass failures across diverse user and tool inputs.
C:For a system whose objective is to stop an agent's broad language permissions from turning into unrestricted business-system privileges, tools must authorize each action using the caller's allowed scope, regardless of what the LLM requests; natural-language rules are insufficient; this does not directly meet the required outcome to discover harmful outputs and instruction-bypass failures across diverse user and tool inputs.
D:For a system whose objective is to show exactly which source version supported a regulated answer, document IDs, timestamps, and citation spans let reviewers reconstruct the evidence used at response time; later source edits should not erase the audit trail; this does not directly meet the required outcome to discover harmful outputs and instruction-bypass failures across diverse user and tool inputs.
Question 108
After implementing a workflow to use an adversarial safety test suite before release, the team needs a robust release gate. Which check should it choose?
- Red-team malicious instructions inside retrieved pages and record detected unsafe tool intentions
- Measure attack success rate and incident severity across protected versions
- Reconstruct historical answer citations using retained source references and version records
- Attempt denied write calls and confirm enforced rejection plus audit records
Explanation
Correct Answer
B:The accepted change is to use an adversarial safety test suite before release; this evaluation checks whether it can discover harmful outputs and instruction-bypass failures across diverse user and tool inputs, since red-team cases exercise guardrail boundaries under realistic attacks, unlike happy-path functional tests alone.
Incorrect Answers
A:This test concerns the ability to prevent an untrusted document from instructing an agent to ignore its governing rules, for which prompt Shields and related controls can detect indirect instructions embedded in lower-trust content; document text remains evidence, not authority; it does not establish whether the change to use an adversarial safety test suite before release works for the current release.
C:This test concerns the ability to show exactly which source version supported a regulated answer, for which document IDs, timestamps, and citation spans let reviewers reconstruct the evidence used at response time; later source edits should not erase the audit trail; it does not establish whether the change to use an adversarial safety test suite before release works for the current release.
D:This test concerns the ability to stop an agent's broad language permissions from turning into unrestricted business-system privileges, for which tools must authorize each action using the caller's allowed scope, regardless of what the LLM requests; natural-language rules are insufficient; it does not establish whether the change to use an adversarial safety test suite before release works for the current release.
Question 109
Stakeholders want a Foundry-based application able to keep traces useful without retaining customer identifiers or secrets in operational logs. Which choice implements that requirement?
- Record provenance for retrieved evidence
- Apply input and output harm filters
- Remove or mask sensitive personal data before logging
- Require human approval for high-impact agent actions
Explanation
Correct Answer
C:For the stated requirement to keep traces useful without retaining customer identifiers or secrets in operational logs, PII-aware redaction and minimal logging reduce privacy risk; collecting full prompts in diagnostic systems is not inherently necessary; this is the mechanism that the system must implement.
Incorrect Answers
A:For a system whose objective is to show exactly which source version supported a regulated answer, document IDs, timestamps, and citation spans let reviewers reconstruct the evidence used at response time; later source edits should not erase the audit trail; this does not directly meet the required outcome to keep traces useful without retaining customer identifiers or secrets in operational logs.
B:For a system whose objective is to limit harmful user prompts and unsafe assistant responses according to an approved safety policy, content filters classify supported harm categories at the inference boundary; thresholds require evaluation for false positives and misses; this does not directly meet the required outcome to keep traces useful without retaining customer identifiers or secrets in operational logs.
D:For a system whose objective is to keep payment or account-changing tool calls under an explicit human authorization step, human-in-the-loop approval gates restrict irreversible operations even if the agent's reasoning or external evidence is wrong; this does not directly meet the required outcome to keep traces useful without retaining customer identifiers or secrets in operational logs.
Question 110
A solution architect is deciding how to explain changes in moderation behavior across releases and support rollback. Which Azure AI pattern addresses that task?
- Limit knowledge access to the caller's entitlements
- Apply input and output harm filters
- Separate review and execution responsibilities
- Enforce per-tool authorization independently
- Use documented safety-policy versioning
Explanation
Correct Answer
E:For the stated requirement to explain changes in moderation behavior across releases and support rollback, versioned guardrail policies make thresholds, blocklists and exception decisions reproducible in evaluations and incidents; this is the mechanism that the system must implement.
Incorrect Answers
A:For a system whose objective is to prevent an agent from citing restricted HR content to ordinary staff, security trimming or permission-aware retrieval applies user access rules before private passages reach generation; this does not directly meet the required outcome to explain changes in moderation behavior across releases and support rollback.
B:For a system whose objective is to limit harmful user prompts and unsafe assistant responses according to an approved safety policy, content filters classify supported harm categories at the inference boundary; thresholds require evaluation for false positives and misses; this does not directly meet the required outcome to explain changes in moderation behavior across releases and support rollback.
C:For a system whose objective is to stop an agent from approving its own high-risk recommendation, distinct approval identity or review service introduces independent oversight; a second prompt in the same privileged agent is not separation of duties; this does not directly meet the required outcome to explain changes in moderation behavior across releases and support rollback.
D:For a system whose objective is to stop an agent's broad language permissions from turning into unrestricted business-system privileges, tools must authorize each action using the caller's allowed scope, regardless of what the LLM requests; natural-language rules are insufficient; this does not directly meet the required outcome to explain changes in moderation behavior across releases and support rollback.