Question 100
An assistant supports employees and external customers. What should requirements specify before exposing the same tools to both?
- That external users receive shorter answers.
- Which operations and records each authenticated role may access.
- That every user receives the same retrieved document set.
- That users describe their own privileges in prompts.
Explanation
Correct Answer
B:Role-specific authorization defines the actual capability boundary and informs tool exposure and backend checks.
Incorrect Answers
A:Answer length does not restrict the underlying operations or records they can access.
C:Uniform retrieval can expose documents beyond individual entitlements; access must follow the authenticated user.
D:Self-asserted authority is untrusted input and cannot establish actual entitlements.
Question 101
A pilot succeeds on English messages, but production includes Urdu and mixed-language text. What is missing?
- Representative evaluation coverage for the production language mix.
- A larger English-only demonstration set.
- A claim that multilingual capability guarantees equal performance.
- Identical output formatting for every language.
Explanation
Correct Answer
A:The pilot’s evidence does not establish performance on materially different inputs encountered in deployment. Separate important input categories so strong performance on common easy cases cannot conceal failure on a required minority case.
Incorrect Answers
B:More samples from the same restricted slice do not directly test the missing languages.
C:General capability does not prove this task’s measured quality across languages and writing styles.
D:Formatting consistency cannot establish that the source message was understood correctly.
Question 102
A fraud decision must return during a live checkout. What must API selection prioritize?
- A synchronous or streamed path meeting the checkout response deadline.
- Only the maximum requests allowed per batch.
- Batch discounts regardless of completion timing.
- An email containing the answer the next day.
Explanation
Correct Answer
A:The decision is on the interactive critical path; asynchronous deferred processing cannot substitute for timely completion.
Incorrect Answers
B:Batch capacity does not determine suitability for a live response deadline.
C:Lower cost does not satisfy an immediate decision requirement when results arrive after checkout.
D:Delayed notification changes the required user flow rather than meeting its timing constraint.
Question 103
A product promises deletion of a user’s stored AI history. What scope must the design identify?
- All data copies covered by the deletion policy.
- Only the visible chat title.
- Only the model’s last response.
- An instruction asking future generations to forget.
Explanation
Correct Answer
A:The promise requires a defined lifecycle across relevant storage layers, not just removal from one interface. Removing the primary row is incomplete if a searchable index or cache can still reproduce the deleted content under the applicable policy.
Incorrect Answers
B:Deleting a label leaves underlying messages and derived records intact.
C:Earlier inputs, outputs, and application copies can remain after one response is removed.
D:A prompt does not delete persisted application storage or enforce retention policy.
Question 104
An assistant's availability target is 99.9%. One team excludes planned maintenance from its calculation; another counts it. What must the requirement settle before their measurements can be compared?
- Use the larger reported number as the official result.
- Compare the percentages after rounding both to the same precision.
- Define the measurement window and the downtime included in the calculation.
- Use the arithmetic mean of the two published availability percentages.
Explanation
Correct Answer
C:The teams must use the same eligible service-time window and maintenance treatment. Otherwise their percentages measure different definitions of availability even if both calculations are internally correct.
Incorrect Answers
A:Selecting the favorable value does not resolve the incompatible accounting rules and makes compliance depend on reporting preference.
B:Display precision does not resolve whether maintenance counts as downtime. Equal formatting can still conceal different measurement definitions.
D:Averaging incompatible measurements conceals the denominator difference. Define included downtime and the measurement window before aggregation.
Question 105
Stakeholders request an assistant that drafts emails but must never send them. What should the capability design reflect?
- Expose the send tool but hide its button in the interface.
- Allow sending after a content-quality classifier approves the draft.
- Provide drafting capability without granting a send operation.
- Expose the send tool with a description recommending draft-only use.
Explanation
Correct Answer
C:The explicit exclusion should shape available execution authority, not merely the wording of generated drafts.
Incorrect Answers
A:Hiding a UI control does not remove the agent's callable send capability. The exclusion must be reflected at the execution boundary.
B:Content approval establishes neither product scope nor permission to send. Even a well-written draft remains subject to the explicit exclusion.
D:A recommendation in the description leaves the prohibited operation available. Removing send authority expresses the requirement directly.
Question 106
A business must explain why an automated refund was issued. Which evidence requirement is most relevant?
- Store only a generic “processed” flag.
- Record inputs, policy version, authorization, action parameters, and execution outcome.
- Keep only the customer’s preferred language.
- Require the model’s private internal reasoning transcript.
Explanation
Correct Answer
B:These records connect the decision to its governing rules and actual effect without requiring hidden model reasoning.
Incorrect Answers
A:A status flag lacks the evidence needed to reconstruct which policy and action produced the refund.
C:Language preference does not explain authorization, refund amount, or the rule applied.
D:Operational accountability should use observable evidence and concise justifications, not depend on inaccessible internal reasoning.
Question 107
Developers repeatedly tune a prompt against the same examples. What protects the final quality estimate?
- Report only the examples used during tuning.
- Evaluate on a separate representative holdout set.
- Treat prompt length as the final quality score.
- Remove all difficult cases from testing.
Explanation
Correct Answer
B:Unseen cases reduce the risk that reported improvement reflects tuning to known examples rather than general task performance.
Incorrect Answers
A:Those examples are influenced by iterative optimization and can overstate generalization.
C:Length does not measure whether the application meets its task requirements.
D:Excluding failures makes the score less representative of production rather than improving the system.
Question 108
A revised prompt passes offline tests but may affect production behavior. What deployment limits initial exposure?
- Deploy to every user but compare only the first few responses.
- Immediate replacement for every user without monitoring.
- A monitored canary with explicit rollback thresholds.
- Deleting the previous prompt before launch.
Explanation
Correct Answer
C:Compare the limited rollout against agreed success and failure thresholds before increasing exposure, and retain a way to stop the rollout.
Incorrect Answers
A:Observing a small sample does not limit deployment exposure; a canary constrains which traffic receives the candidate behavior.
B:This maximizes exposure before production behavior is understood.
D:Removing the known configuration makes recovery harder if the new behavior regresses.
Question 109
A team wants production-like evaluation without letting the candidate agent execute real changes. Which approach fits?
- Skip recording proposed actions.
- Run shadow evaluations with side effects disabled or safely simulated.
- Let both old and new agents perform every write.
- Publish candidate responses as authoritative immediately.
Explanation
Correct Answer
B:Record what the candidate would do while the established path remains authoritative; otherwise experimental behavior can create real side effects.
Incorrect Answers
A:Without observations, shadow execution provides little evidence about candidate behavior.
C:Duplicate execution can change real state and defeats the purpose of nonimpacting evaluation.
D:That exposes unvalidated behavior rather than collecting shadow evidence.
Question 110
Old and new application versions must run simultaneously during deployment. A shared structured field is being renamed. Which migration preserves compatibility during the overlap?
- Switch producers immediately and update consumers only when they fail.
- Remove the old field before deploying any new consumers.
- Deploy the new producer alone and require all running clients to infer the rename.
- Introduce compatibility first, migrate consumers, and remove the old field afterward.
Explanation
Correct Answer
D:An expand-and-contract sequence gives both consumer versions a supported representation while they coexist. The old contract is retired only after its remaining users have migrated.
Incorrect Answers
A:That exposes still-running old consumers to an incompatible response during the overlap. Compatibility must exist before either side relies on the new-only contract.
B:Existing clients still require that field, so the first step breaks the active version before a replacement path is available.
C:A consumer's parser follows its implemented contract. It cannot be assumed to reinterpret an unfamiliar field without a compatible code change.