Question 350
A pilot succeeds with expert volunteers who know every system limitation. The intended rollout includes occasional users with little domain knowledge. What evidence is missing before extrapolating the result?
- More repetitions of the same tasks by the original expert volunteers
- Results from representative users performing the intended production tasks
- A larger synthetic benchmark scored without user interaction
- Higher throughput measurements using the expert volunteers' saved prompts
- A longer satisfaction survey sent only to the expert volunteers
Explanation
Correct Answer
B:Expert volunteers may compensate for weaknesses that occasional users cannot recognize. The rollout decision needs evidence about the actual user population and workflow.
Incorrect Answers
A:Repetition can improve estimates for that cohort but does not address the population gap. The planned users differ in relevant knowledge and behavior.
C:A benchmark may test model quality while omitting how inexperienced users interpret and act on results. The identified uncertainty concerns the human workflow.
D:Throughput evidence addresses capacity rather than user representativeness. It cannot show whether occasional users can complete tasks correctly.
E:Detailed opinions from the same cohort do not establish usability for occasional users. The evaluation needs the missing population represented.
Question 351
A sponsor asks to add another language one week before release. Existing tests, support staffing, and policy review cover only the original language. What should guide the decision?
- Translate the interface and assume the existing acceptance results transfer
- Keep the date by removing the new language from all quality metrics
- Reject every late request without considering its impact or value
- Assess the added acceptance, support, and risk work before committing to the date
Explanation
Correct Answer
D:A language expansion can affect more than translation quality. The decision should expose the additional work and uncertainty before promising the same release date.
Incorrect Answers
A:Interface translation does not establish task quality or operational readiness in the new language. The current evidence covers a narrower scope.
B:Excluding the added scope from measurement would hide its readiness. The release decision still needs evidence for the users being included.
C:A change process should evaluate the actual trade-off. A categorical rejection ignores the possibility of a feasible phased or rescheduled approach.
Question 352
Three architecture choices are blocked by uncertainty about OCR quality, retrieval recall, and user tolerance for delay. How should the next learning work be organized?
- Run one generic benchmark and use its aggregate score for every decision
- Create targeted experiments ranked by decision impact and cost of learning
- Implement all three alternatives fully and compare them only after production
- Choose the most familiar architecture and remove the uncertainties from the plan
Explanation
Correct Answer
B:Each experiment should resolve an uncertainty that affects a concrete decision. Prioritizing by impact and effort helps the team learn before committing to expensive paths.
Incorrect Answers
A:Different uncertainties require different evidence. A single aggregate measure may not explain OCR failure, retrieval limits, or user tolerance.
C:Full implementation can be unnecessarily costly when smaller experiments resolve the decisive uncertainty. It delays learning until commitments are harder to reverse.
D:Familiarity may influence implementation effort but does not answer the unresolved questions. The assumptions still affect whether the design works.
Question 353
A pilot shows that most value comes from preparing evidence packs, while autonomous final decisions add little benefit and substantial risk. What roadmap adjustment is justified?
- Hide the evidence-pack results to avoid confusing the original success narrative
- Continue toward autonomy because it appeared in the original concept deck
- Cancel all AI work because the autonomous decision feature underperformed
- Prioritize evidence preparation and reassess whether final-decision autonomy is worthwhile
Explanation
Correct Answer
D:The roadmap should respond to measured value and risk. The pilot provides a reason to invest in the useful capability and reconsider the less beneficial expansion.
Incorrect Answers
A:Selective reporting undermines the roadmap decision. Stakeholders need the actual distribution of value and risk across capabilities.
B:An initial vision is a hypothesis, not a commitment immune to evidence. The new findings materially change the value and risk trade-off.
C:The evidence identifies a valuable capability within the project. Abandoning it would ignore the demonstrated benefit of evidence preparation.
Question 354
Two accountable teams disagree about a release requirement, and neither has authority over the other. Repeated technical meetings have not resolved the conflict. What should the architect do?
- Present the alternatives, evidence, and consequences to the designated decision authority
- Choose the preferred option privately and let implementation settle the dispute
- Average the two requirements even if the result satisfies neither team's constraint
- Keep scheduling identical meetings until one team stops attending
Explanation
Correct Answer
A:The unresolved issue needs an accountable trade-off decision. A concise record of options and consequences enables escalation without pretending the conflict is purely technical.
Incorrect Answers
B:A hidden decision bypasses the relevant authority and conceals the trade-off. It can create a later governance or delivery conflict.
C:A numerical compromise is not necessarily a valid or authorized solution. The alternatives and consequences need an explicit decision.
D:Repeated discussion without a decision path does not resolve authority. The issue should be escalated through the agreed process.
Question 355
A vendor product meets current needs, but the organization requires the ability to migrate its configuration and records later. What should procurement validate before commitment?
- The product's current interface similarity to a competing vendor
- Only the introductory subscription discount and first-year usage allowance
- A verbal assurance that migration is usually easy for other customers
- The vendor's ability to generate a sample migration checklist
- Export formats, access to required records, transition assistance, and termination conditions
Explanation
Correct Answer
E:Exit capability depends on practical and contractual details, not merely present functionality. These terms determine whether migration remains feasible when the relationship changes.
Incorrect Answers
A:Visual similarity does not imply compatible data or configuration exports. Exit feasibility must be checked at the actual integration and contract boundaries.
B:Initial price does not establish whether data and configurations can be moved later. The stated requirement concerns the end of the relationship as well as its start.
C:A general assurance does not specify this organization's access, formats, or obligations. The relevant capabilities and terms need concrete verification.
D:A checklist can describe tasks without providing the necessary export or contractual rights. Procurement should validate the underlying capability and terms.
Question 356
A business owner says, 'The assistant must never recommend an expired offer.' Which artifact best converts this statement into implementable acceptance criteria?
- Defined offer-validity rules, authoritative timestamps, boundary examples, and expected failure behavior
- Use the original statement unchanged as the sole test specification
- A generic accuracy target averaged across every recommendation type
- A demonstration using only offers with expiry dates several years away
Explanation
Correct Answer
A:The requirement needs precise semantics for expiration and unavailable evidence. Concrete rules and examples allow implementation and testing to agree on the same behavior.
Incorrect Answers
B:The statement leaves expiration semantics and missing-data behavior unresolved. Tests need explicit rules, boundary conditions, and authoritative time inputs.
C:An aggregate target can conceal expired-offer errors. The specific requirement needs its own rules and relevant cases.
D:Easy examples do not clarify the decisive boundary conditions. The acceptance criteria need cases near expiration and situations with missing or conflicting data.
Question 357
A small pilot reports 95% task success from 20 cases. A stakeholder wants to advertise that as established production accuracy. What is the appropriate communication?
- Report the observed result with sample size, scope, and uncertainty
- Round the result to 100% because only one case failed
- Report only the percentage because sample size is a technical detail
- Refuse to share any result until every production case has been observed
Explanation
Correct Answer
A:The evidence is useful but limited by the small pilot and tested conditions. Clear qualification prevents the observation from becoming an unsupported production guarantee.
Incorrect Answers
B:Rounding away the failure materially misrepresents the observed outcome. It also conceals uncertainty rather than explaining it.
C:The sample size affects how the result should be interpreted. Omitting it can imply stronger evidence than the pilot provides.
D:Complete enumeration is not necessary for useful communication. The team can share bounded evidence without presenting it as certainty.
Question 358
Monthly active usage doubles after a mandatory rollout, but staff handling time and error rates are unchanged. What conclusion is supported?
- Usage increased, while the intended operational benefit remains unproven
- The assistant necessarily made the workflow worse because errors did not fall
- The team should stop measuring outcomes because usage is easier to track
- The doubled usage proves that operational productivity also doubled
Explanation
Correct Answer
A:Mandatory exposure can increase activity without improving outcomes. The observed metrics establish adoption but do not demonstrate the claimed efficiency or quality benefit.
Incorrect Answers
B:Unchanged outcomes do not establish deterioration. The data supports a narrower conclusion that the intended improvement has not yet been demonstrated.
C:Convenient measurement does not replace the business objective. Outcome tracking remains necessary to evaluate whether the rollout creates value.
D:Activity and productivity are different measures. The unchanged handling time and error rates do not support that proportional benefit claim.
Question 359
A feature performs well for routine cases but fails acceptance for a distinct high-complexity segment. The segments can be reliably identified before execution. What rollout proposal best uses the evidence?
- Release only to the passing segment with explicit routing and expansion criteria
- Release to everyone and average the failing segment into the overall score
- Reject all deployment permanently because one segment needs more work
- Allow the model to silently decide which cases were probably routine afterward
- Change the complex segment's acceptance threshold after observing its failures
Explanation
Correct Answer
A:A bounded rollout can capture demonstrated value while retaining the established process for unsupported cases. Reliable segmentation and clear expansion criteria are essential.
Incorrect Answers
B:An aggregate score can hide a known unsupported population. The failing segment should not be included merely because routine cases are more numerous.
C:The evidence supports a usable subset and an identifiable boundary. A permanent blanket rejection ignores the possibility of a controlled segmented rollout.
D:Post hoc labeling does not enforce the pre-execution boundary. The proposal relies on reliable routing before unsupported cases reach the feature.
E:A relaxed retrospective threshold would require a justified and authorized requirement change. It does not itself demonstrate that the segment is ready.
Question 360
A release runbook lists technical rollback commands but does not say who can order rollback or which symptoms warrant it. What addition is needed?
- Decision authority, trigger conditions, and communication responsibilities
- More screenshots of the deployment interface without changing the decision process
- A statement that rollback should be avoided whenever possible
- A requirement that every user approve rollback individually
Explanation
Correct Answer
A:Commands explain how to act but not when or by whose authority. The runbook needs an operational decision process so responders can use those commands promptly and appropriately.
Incorrect Answers
B:Screenshots may help execution but do not resolve authority or trigger conditions. The missing information concerns the decision to initiate rollback.
C:A general preference provides no actionable criteria. Responders need conditions and authority for deciding when rollback is warranted.
D:That process is impractical for incident response and does not assign accountable authority. The organization needs a defined operational owner.