Question 1
Business leaders reject an activity-only metric for this process: A retraining release changes the tokenizer and label order. They need to detect the issue where downstream apps interpret correct predictions as the wrong category. Which performance indicator fits?
- Contract-valid predictions after custom-model updates
- True performance drift detected before critical SLO breach
- Critical-field extraction accuracy across document classes
- Worst-performing segment error rate
Explanation
Correct Answer
A:For the use case where a retraining release changes the tokenizer and label order, the indicator contract-valid predictions after custom-model updates is worth baselining against the earlier issue where downstream apps interpret correct predictions as the wrong category.
Incorrect Answers
B:A change in true performance drift detected before critical SLO breach is informative for customer terminology changes over time; it does not establish whether the current process achieves contract-valid predictions after custom-model updates.
C:The alternative tracks critical-field extraction accuracy across document classes, arising from the mechanism where stratified field metrics protect high-impact values; its movement alone says little about the failure that downstream apps interpret correct predictions as the wrong category.
D:The alternative tracks worst-performing segment error rate, arising from the mechanism where segmented evaluations reveal uneven model behavior; its movement alone says little about the failure that downstream apps interpret correct predictions as the wrong category.
Question 2
A policy review covers an agent for which an AI triage model under-serves one customer language. Its most relevant exposure is that one language group might face materially worse decisions without appeal. What safeguard should be required before release?
- Require mitigation and human appeal for consequential differences
- Provide accessible human escalation paths
- Disable the candidate for a cohort on critical regression
- Require a tested human escalation path for unmatched cases
Explanation
Correct Answer
A:The business setting is an AI triage model under-serves one customer language, with the exposure that one language group might face materially worse decisions without appeal; the enforceable protection must correspond to the path where segment-aware evaluations surface fairness risks.
Incorrect Answers
B:The safeguard for chat and voice agents have different interaction patterns deals with a different threat: voice users might be abandoned without an accessible path to a human; even after it is enforced, the exposure that one language group might face materially worse decisions without appeal remains possible in the target workflow where segment-aware evaluations surface fairness risks.
C:This restriction is associated with a conversational change affects only one support geography and addresses the exposure that a regional pilot might produce a severe local regression hidden by global averages; it does not specifically protect the target workflow against one language group might face materially worse decisions without appeal.
D:For the use case where a support agent changes how it classifies priority intents, the control addresses the threat that a new support agent might route urgent cases to a general queue; the specified agent must be protected against one language group might face materially worse decisions without appeal.
Question 3
The acceptance board asks how to verify a corrected design. The setting is: A regulated workload must keep data in permitted locations. Last time, an external model call exports sensitive prompts to an unapproved region. Which scenario should the team execute?
- Recreate an evaluation from the recorded dataset snapshot
- Trace synthetic sensitive requests through each configured service
- Submit a document without classification and verify it stays excluded
- Rebuild a test index and switch traffic under observation
Explanation
Correct Answer
B:For the required workflow where a regulated workload must keep data in permitted locations, the test needs to reproduce or prevent the failure where an external model call exports sensitive prompts to an unapproved region; tracing synthetic sensitive requests through each configured service exercises the path where data-flow architecture identifies processing, storage and telemetry locations.
Incorrect Answers
A:In a solution where model retraining consumes periodically revised business records, recreating an evaluation from the recorded dataset snapshot could catch the defect that no one can identify which dataset produced a model; the current release gate must challenge the failure where an external model call exports sensitive prompts to an unapproved region.
C:The verification concerning new documents enter a production source automatically checks ingestion gates check classification, freshness and ownership against unreviewed confidential drafts appear in agent answers; its result says little about the observed issue where an external model call exports sensitive prompts to an unapproved region.
D:For a search index changes embedding model and chunking approach, the alternate test is about whether explicit index versions support compatible retrieval and rollback; it cannot prove the outcome needed after the reported regression where an external model call exports sensitive prompts to an unapproved region.
Question 4
A proposed deployment supports this business case: Service recommends a price adjustment requiring finance approval. Reviewers also warn that a suggested customer concession might post before delegated finance approval. Which restriction mitigates that concrete risk?
- Do not allow autonomous purchasing without approved limits
- Require human sign-off on critical test oracles
- Treat generated expectations as suggestions until verified
- Forbid direct mutation before recorded delegated approval
Explanation
Correct Answer
D:Service recommends a price adjustment requiring finance approval; its risk scenario is that a suggested customer concession might post before delegated finance approval; the safeguard fits because transaction-path tests confirm recommendation, review and posting records and the restriction addresses that specific exposure.
Incorrect Answers
A:An AI-driven replenishment proposal influences procurement; its guardrail addresses the possibility that autonomous replenishment proposals might place purchases beyond approved limits; the present architecture must constrain the different threat that a suggested customer concession might post before delegated finance approval.
B:This restriction is associated with Copilot writes a test expecting a plausible but wrong answer and addresses the exposure that a fabricated test oracle might approve a flawed agent behavior; it does not specifically protect the target workflow against a suggested customer concession might post before delegated finance approval.
C:Another process could need this guardrail because a hallucinated expected outcome might become a trusted automated oracle; the design being assessed supports service recommends a price adjustment requiring finance approval and has the explicit exposure that a suggested customer concession might post before delegated finance approval.
Question 5
A design adds autonomous behavior to a process where agent must remember order context while asking for missing details. This creates a documented risk: An agent might act before collecting mandatory identity and form fields. Which enforced control is the best fit?
- Remove personal details before using feedback as test data
- Block protected actions unless the trusted user and backend authorize
- Prevent actions before identity and missing fields are verified
- Prevent duplicate case creation on repeated event delivery
Explanation
Correct Answer
C:When an agent might act before collecting mandatory identity and form fields, the use case agent must remember order context while asking for missing details requires a guardrail aligned with the behavior that multi-turn tests assess state tracking, entity separation and follow-up questions.
Incorrect Answers
A:Where support tickets reveal an unusual multistep conversation failure, the guard can address real customer details might leak into evaluation corpora derived from tickets; the exposure that an agent might act before collecting mandatory identity and form fields remains relevant to the present assessment.
B:This restriction is associated with a retrieved email contains instructions to bypass approval and addresses the exposure that injected text might persuade a model to invoke a protected API; it does not specifically protect the target workflow against an agent might act before collecting mandatory identity and form fields.
D:The alternate guard addresses duplicate Dynamics events might create redundant onboarding cases for a won opportunity should create a properly owned onboarding case; the present threat, an agent might act before collecting mandatory identity and form fields, needs a different enforcement boundary.
Question 6
A quality engineer is writing a regression case after the incident where unreviewed confidential drafts appear in agent answers. The underlying business scenario is: New documents enter a production source automatically. What should the test exercise?
- Reindex a modified document and verify traceability
- Reconstruct one released model's source and threshold decisions
- Submit a document without classification and verify it stays excluded
- Ask about identical case names from isolated tenants
Explanation
Correct Answer
C:Submitting a document without classification and verifying it stays excluded is aligned with the failure that unreviewed confidential drafts appear in agent answers; its causal relevance is that ingestion gates check classification, freshness and ownership for the situation where new documents enter a production source automatically.
Incorrect Answers
A:The tested behavior is that content lineage records update, transformation and index revision; the necessary evidence here concerns ingestion gates check classification, freshness and ownership and the previously observed unreviewed confidential drafts appear in agent answers.
B:In a solution where data scientists cannot explain why a newer classifier behaves differently, reconstructing one released model's source and threshold decisions could catch the defect that models move to production without an audit of training inputs; the current release gate must challenge the failure where unreviewed confidential drafts appear in agent answers.
D:The tested behavior is that partitioning and authorized filters constrain relevant candidates; the necessary evidence here concerns ingestion gates check classification, freshness and ownership and the previously observed unreviewed confidential drafts appear in agent answers.
Question 7
A policy review covers an agent for which a tuned model claims higher accuracy but adds latency. Its most relevant exposure is that a new classifier might replace its safe baseline before comparisons finish. What safeguard should be required before release?
- Limit retroactive alteration of approved QA evidence
- Keep rollback available until all critical criteria pass
- Set cost budgets and investigate anomalous context expansion
- Trigger retraining review when input distribution shifts
Explanation
Correct Answer
B:The business setting is a tuned model claims higher accuracy but adds latency, with the exposure that a new classifier might replace its safe baseline before comparisons finish; the enforceable protection must correspond to the path where candidate comparison evaluates quality, latency and safety under matched load.
Incorrect Answers
A:The safeguard for an updated agent performs differently months after release deals with a different threat: old evaluation evidence might be overwritten to conceal a failed release gate; even after it is enforced, the exposure that a new classifier might replace its safe baseline before comparisons finish remains possible in the target workflow where candidate comparison evaluates quality, latency and safety under matched load.
C:This restriction is associated with model bills increase although request volume is stable and addresses the exposure that a prompt might accumulate costly context with no consumption guardrail; it does not specifically protect the target workflow against a new classifier might replace its safe baseline before comparisons finish.
D:The alternate boundary mitigates input drift might undermine accuracy without triggering a review, which is relevant to production documents differ from clean training samples; it leaves the current workflow vulnerable to a new classifier might replace its safe baseline before comparisons finish.
Question 8
Operations has not established the cause of this defect: Surface quality scores appear healthy despite wrong documents. The associated setting is: An assistant responds fluently but cites unrelated support articles. Which inspection should precede remediation?
- Correlate request rates with throttles and queue depth
- Review threshold breaches against user-impacting failure times
- Compare query, filters and top retrieved source identifiers
- Compare retrieved document IDs for multiple test principals
Explanation
Correct Answer
C:The reported fault is that surface quality scores appear healthy despite wrong documents in a system where an assistant responds fluently but cites unrelated support articles; evidence from comparing query, filters and top retrieved source identifiers can show whether retrieval telemetry distinguishes evidence search from answer generation.
Incorrect Answers
A:The proposed probe concerns a regional launch multiplies concurrent agent requests, where capacity metrics and throttling errors show saturation risk; this incident instead requires evidence about retrieval telemetry distinguishes evidence search from answer generation and the failure that surface quality scores appear healthy despite wrong documents.
B:Teams discover failures through complaints several days later; examining that other path does not determine whether retrieval telemetry distinguishes evidence search from answer generation, which is what distinguishes the issue in the stated use case.
D:For the failure that agent cites a private HR document to unauthorized staff, comparing retrieved document IDs for multiple test principals is relevant; this question instead reports that surface quality scores appear healthy despite wrong documents, so different evidence is needed.
Question 9
A staging test has to detect the specific defect that logs cannot reconstruct which API mutation followed an answer. In the workflow where a request crosses Copilot Studio, a flow and the CRM backend, which exercise verifies the corrective behavior?
- Import a test solution and verify expected agent behavior
- Promote to staging and confirm correct connection targets
- Interrupt processing after the first app commits
- Inject timeout after commit and verify a single write
- Force a downstream error and verify cross-system traceability
Explanation
Correct Answer
E:For the required workflow where a request crosses Copilot Studio, a flow and the CRM backend, the test needs to reproduce or prevent the failure where logs cannot reconstruct which API mutation followed an answer; forcing a downstream error and verifying cross-system traceability exercises the path where correlation identifiers link the distributed operational path.
Incorrect Answers
A:In the alternate setting where the same agent must move from development to test and production, the test exercises solution-based ALM transports supported components under controlled releases; even a pass would not show that logs cannot reconstruct which API mutation followed an answer has been prevented.
B:The suggested test serves a flow moved to production still points to a test endpoint, in which agent writes real customer data into the sandbox; the present scenario instead concerns the reported problem where logs cannot reconstruct which API mutation followed an answer.
C:The suggested test serves a blocked invoice triggers a hold on related procurement actions, in which a network fault leaves the two apps inconsistent; the present scenario instead concerns the reported problem where logs cannot reconstruct which API mutation followed an answer.
D:In a solution where agents modify records through external APIs, injecting timeout after commit and verifying a single write could catch the defect that successful chat sessions conceal repeated backend writes; the current release gate must challenge the failure where logs cannot reconstruct which API mutation followed an answer.
Question 10
The fault affects an AI-powered workflow in which agents suggest payment adjustments and inventory holds. Specifically, a new connector allows a service identity to post directly. Which diagnostic investigation best distinguishes the relevant failure path?
- Inspect signatures, hashes and approved source records
- Compare action privileges before and after package import
- Trace source exception through target state and error queue
- Review endpoint access logs and allowed network paths
Explanation
Correct Answer
B:A new connector allows a service identity to post directly; to attribute the defect rather than guess at a fix, comparing action privileges before and after package import examines the dependency where authorization regression gates verify posting authority remains constrained.
Incorrect Answers
A:In the process where deployment automation pulls model artifacts from unverified origins, the diagnostic procedure could explain a replaced model package introduces unknown behavior; this particular incident instead turns on whether authorization regression gates verify posting authority remains constrained and why a new connector allows a service identity to post directly.
C:For the failure that a network fault leaves the two apps inconsistent, tracing source exception through target state and error queue is relevant; this question instead reports that a new connector allows a service identity to post directly, so different evidence is needed.
D:In the separate scenario where a sensitive model endpoint is reachable using a broadly shared key, the same investigation could explain untrusted clients submit requests and exfiltrate outputs; it does not isolate the reported cause in a system where agents suggest payment adjustments and inventory holds.