Question 30
A safety classifier blocks 98% of a labeled attack set, but a new threshold also blocks 20% of legitimate account-recovery requests. The team must retain useful service while controlling attack risk. What evidence should guide the next mitigation?
- Keep the threshold because aggregate attack detection is the only release measure
- Lower the threshold until legitimate refusals disappear, then release without attack retesting
- Choose a cutoff from the score distribution without labeling blocked requests
- Compare attack misses and legitimate blocks by slice before selecting a mitigation
Explanation
Correct Answer
D:A labeled, sliced evaluation makes the trade-off between false positives and missed attacks visible. The team can then test threshold, classifier, or workflow changes against explicit security and usability criteria.
Incorrect Answers
A:Attack detection is important, but unintended blocking also affects whether the service works for its intended users. The release decision needs evidence on both error types.
B:Reducing false positives may increase missed attacks. A candidate threshold must be evaluated on malicious cases as well as the legitimate traffic it unblocks.
C:A score histogram can show how often the classifier fires, but cannot distinguish harmful requests from legitimate ones. Selecting a cutoff requires outcome labels and an explicit trade-off between the two error types.
Question 31
An assistant explains eligibility decisions from approved records. Review shows its answers are accurate overall, but it invents denial reasons more often for one language group. What should the release review prioritize?
- Add a translation layer and expand deployment before measuring group-specific errors
- Report only groups with large volumes, combining smaller groups into the overall average
- Approve based on the overall score and investigate subgroup differences after expansion
- Investigate subgroup error patterns, then evaluate targeted mitigations before expanding deployment
Explanation
Correct Answer
D:Segmented evaluation can distinguish data coverage, translation, retrieval, and decision-rule problems. Mitigations should be checked against the affected group and the broader population to avoid shifting the failure elsewhere.
Incorrect Answers
A:Translation could address one possible cause, but the evidence does not yet establish that cause. Expanding before measuring the effect risks preserving or worsening the disparity.
B:Aggregation may be necessary for privacy or statistical reliability, but it must not erase a known problem. The team needs an analysis capable of explaining the observed unsupported denials.
C:A passing overall score can coexist with harmful concentrated errors. The already observed disparity merits investigation before exposing more users to it.
Question 32
An internal review must reconstruct which configuration produced a disputed answer and which policy sources were supplied. Which records are necessary? Choose TWO.
- The request’s prompt and model configuration revision identifiers
- The final answer text and its confidence statement, without source provenance
- The current deployment settings, captured after the incident investigation starts
- The document index creation time, without identifiers for retrieved passages
- The request’s retrieved source identifiers and preserved revision references
Explanation
Correct Answers
A:Request-linked revisions establish which configuration was actually used. They should resolve to preserved artifacts; identifying a version helps reconstruction but does not guarantee identical nondeterministic output.
E:Source versions identify the evidence made available for this particular response. Without them, a later policy edit could make a reconstruction use different facts.
Incorrect Answers
B:The final answer is useful evidence of the symptom, but it does not establish which configuration or sources produced it. A confidence statement cannot supply that missing provenance.
C:Current settings can differ from the deployment at the time of the incident. Reconstruction requires versions linked to the disputed request, not a later snapshot.
D:An index timestamp identifies a broad processing event, not the particular evidence supplied for an answer. It cannot distinguish relevant source revisions or retrieved passages.
Question 33
A sponsor requests an autonomous purchasing assistant but cannot state the allowed spending limit, approval owner, or acceptable error rate. What should the architect do before committing to a production design?
- Commit to a read-only prototype and treat its assumptions as the production requirements
- Ask the delivery team to choose default spending limits from a previous project
- Agree decision boundaries, measurable success criteria, and accountable owners through structured discovery
- Estimate delivery from a working demo, leaving business approvals for the final sprint
Explanation
Correct Answer
C:The missing answers determine autonomy, controls, evaluation, and operating responsibilities. A recorded discovery outcome gives the design a concrete acceptance basis.
Incorrect Answers
A:A read-only prototype can support discovery, but its assumptions still need stakeholder validation. A production commitment cannot safely treat temporary prototype choices as approved requirements.
B:Prior projects can suggest questions and ranges, but they cannot authorize spending for this sponsor. Business owners must decide the applicable boundaries.
D:Late resolution of authority and acceptance criteria can materially change the design. A demo-based estimate before those decisions conceals significant delivery uncertainty.
Question 34
A retailer must choose between an overnight batch assistant and a real-time assistant. Both are technically feasible, but the real-time option has higher operational cost. What should the architecture decision record emphasize?
- Document decision drivers, measured trade-offs, the selected option, and conditions for reconsideration
- A detailed deployment diagram and library inventory without the selection rationale
- The chosen approach’s benefits and implementation steps without recording rejected alternatives
- A record of stakeholder approval without the assumptions behind the decision
Explanation
Correct Answer
A:This captures why the option fits current requirements and what future changes would alter the conclusion. It helps successors distinguish an intentional trade-off from an accidental implementation constraint.
Incorrect Answers
B:A diagram and inventory describe the implementation, but leave future maintainers unable to assess why it was selected. They do not capture the relevant timeliness-versus-cost trade-off.
C:Implementation guidance is useful, but omitting alternatives and assumptions makes the decision harder to reassess. The record should explain why the selected benefits justified its costs.
D:Approval establishes that a decision was accepted, not the circumstances that made it suitable. Future owners need those circumstances to recognize when reconsideration is warranted.
Question 35
The customer interprets a 3-second SLA as time until the complete answer, while engineering measures time until the first streamed token. What should happen before signing off?
- Average the first-token and complete-answer times into one reported latency
- Use first-token latency for all reporting because it is easier to measure
- Adopt a complete-answer target but leave the request population and percentile undefined
- Agree timing boundaries, covered requests, percentiles, and failure treatment with the customer
Explanation
Correct Answer
D:The parties need one observable service commitment. Explicit measurement boundaries prevent a fast first token from being reported as compliance with a complete-answer promise.
Incorrect Answers
A:Combining two different events produces a number that matches neither party’s original expectation. A service measure needs a defined meaning, not a blended compromise.
B:Instrumentation convenience does not establish the customer’s intended experience. First-token timing alone cannot demonstrate compliance with a complete-answer commitment.
C:Choosing the end event resolves only part of the ambiguity. The parties still need to specify which requests count and how slow or failed requests affect compliance.
Question 36
A pilot was approved to draft policy answers for staff review. After a demonstration, the sponsor asks it to automatically make policy exceptions for customers using the same release date. What is the best response?
- Assess the new decision authority, risk, acceptance criteria, and effect on delivery
- Retain staff-drafting tests because both workflows produce the same answer format
- Add an approval sentence to the prompt and keep the original release commitment
- Deploy exception decisions to a small group before agreeing who can authorize them
Explanation
Correct Answer
A:The architect should make the added responsibility and work visible before committing. The outcome may be a revised schedule, narrower scope, or separate release based on informed agreement.
Incorrect Answers
B:Output format does not capture the difference between advice and authority. Tests must examine whether exception decisions are permitted and correct under the new responsibility.
C:A prompt change alone does not establish accountable authorization or evaluate the new consequences. Keeping the date without impact analysis hides required design and testing work.
D:A small rollout limits exposure but does not resolve who may make exceptions. The decision boundary should be agreed before users are subject to the new behavior.
Question 37
A Claude service passes acceptance tests, but only its original architect knows how to recover from an expired external-service credential. What remains necessary for a supportable handoff?
- Document the architecture and close the project without exercising credential recovery
- Assign an operator and test the recovery runbook and escalation path
- Transfer monitoring access but leave credential renewal dependent on the original architect
- Add automated credential-expiry alerts without assigning a responder or recovery procedure
Explanation
Correct Answer
B:The receiving team needs both authority and a reproducible procedure for routine recovery. Exercising the procedure verifies that the handoff works beyond a written document.
Incorrect Answers
A:Architecture documentation explains the system but does not demonstrate that the receiving team can recover it. A handoff needs practiced procedures and the authority to execute them.
C:Visibility helps detect the failure but does not give the receiving team the ability to resolve it. The original single-person dependency remains.
D:Alerts can create useful warning time, but someone must be accountable for acting on them. A notification without an owner or procedure does not complete the handoff.
Question 38
A development team wants Claude Code to use the repository's documented test command and coding conventions consistently. These instructions are suitable for all contributors and contain no secrets. Which approach is appropriate?
- Store the project guidance only in each developer’s user-level instruction file
- Maintain reviewed test commands and coding conventions in the shared project CLAUDE.md
- Rely on each developer’s automatically learned local preferences for project conventions
- Record the guidance only in the opening chat message of each development task
Explanation
Correct Answer
B:A project instruction file gives contributors a common reviewable place for conventions and commands. Keeping it concise and current improves its practical usefulness; it does not replace automated enforcement.
Incorrect Answers
A:User-level files can guide an individual, but separate copies can drift and are not the repository’s shared source. Project guidance should follow the project’s review process.
C:Individual learned preferences do not provide a reviewed common project contract. The team needs an explicit maintained source for its conventions.
D:Session-specific instructions can work temporarily but require repeated manual setup and are easy to omit. A shared project file gives the conventions a durable location.
Question 39
Claude Code proposes a refactor that appears clean and passes formatting checks. The affected module handles currency rounding, and no behavioral tests have run. What should the engineer do before merging?
- Review the explanatory summary and merge if it describes unchanged behavior
- Run a syntax check and formatter, treating a clean result as sufficient evidence
- Generate tests from the new implementation and accept its current outputs as expected values
- Review the actual diff and test rounding boundaries against the established behavior contract
Explanation
Correct Answer
D:Refactoring can accidentally alter behavior even when the intended change is structural. Tests around rounding boundaries and a human diff review provide evidence about the module's actual contract.
Incorrect Answers
A:A summary describes intended behavior and can miss discrepancies in the code. The engineer needs evidence from the actual patch and relevant execution cases.
B:Syntax and formatting checks do not verify arithmetic behavior. They cannot establish that boundary values still follow the required currency rounding rules.
C:Deriving expected values from the changed code can encode its bug as the test oracle. Tests should compare behavior with independently established requirements and trusted examples.
Question 40
Claude Code reports passing tests locally, but CI fails on the same commit. The engineer has not compared the CI test command, working directory, or configuration with the local run. What should the engineer do first?
- Reproduce the CI test invocation with its working directory and effective configuration
- Increase the model context window while keeping the failing command unchanged
- Rewrite the tests to accept the output currently produced by CI
- Copy the developer’s package cache into CI before checking which command is run
Explanation
Correct Answer
A:The discrepancy points to the execution environment rather than a need for a new model answer. Matching the actual CI invocation and configuration makes the failure reproducible and exposes assumptions hidden by the local run.
Incorrect Answers
B:Context capacity affects what the assistant can consider, but does not correct an incorrect command or environment. The failure must first be reproduced under the conditions that produced it.
C:Changing the expected output before diagnosing the difference risks turning a defect into accepted behavior. The test contract should be preserved while the environment and invocation are investigated.
D:Dependency differences are one possibility, but copying a cache changes the environment without identifying the cause. Establishing the exact failing invocation is the more controlled first step.