Question 10
An engineer receives this incident report: A support agent treats every follow-up pronoun as a new topic because earlier exchanges are not supplied. What should be changed?
- Persist only allowed preference fields behind identity checks and expiration controls
- Enable semantic ranking over searchable human-readable fields after initial retrieval
- Register an approved order-status API tool and scope its credentials to read-only lookup
- Carry forward the relevant conversation context with the agent's dialogue state
Explanation
Correct Answer
D:The fault signal is that a support agent treats every follow-up pronoun as a new topic because earlier exchanges are not supplied; the targeted fix works because conversation state preserves necessary prior turns for reference resolution; it differs from a searchable repository of long-lived institutional documents, restoring the ability to resolve a follow-up such as 'What about the second option?' using the user's previous turn.
Incorrect Answers
A:If the failing function were to personalize repeated interactions without placing all old transcripts in every prompt, this change could be relevant because explicitly persisted, consent-governed preference data supports bounded long-term personalization; transient chat state should not become an uncontrolled permanent profile; the actual symptom is that a support agent treats every follow-up pronoun as a new topic because earlier exchanges are not supplied, which calls for a different repair.
B:If the failing function were to improve ordering of an already retrieved set by how directly it answers a user's question, this change could be relevant because semantic reranking uses language understanding to reorder candidate search results; it does not replace indexing or generate new source documents; the actual symptom is that a support agent treats every follow-up pronoun as a new topic because earlier exchanges are not supplied, which calls for a different repair.
C:If the failing function were to let an agent retrieve a live order status from the organization's transactional system, this change could be relevant because an authenticated tool call obtains current application state at request time; stale embeddings cannot guarantee a transaction's latest status; the actual symptom is that a support agent treats every follow-up pronoun as a new topic because earlier exchanges are not supplied, which calls for a different repair.
Question 11
The support ticket describes this behavior: The general text summarizer returns prose descriptions instead of executable Python for API helpers. Which engineering change should come first?
- Run vector retrieval on document chunks using the same embedding space as queries
- Connect governed document retrieval and pass cited passages to the answer step
- Pass the actual image to a vision-capable model and ground its response in visible evidence
- Use a model evaluated for coding tasks and validate outputs with an isolated test runner
- Embed the queries and chunks with a compatible model and add vector retrieval
Explanation
Correct Answer
D:The fault signal is that the general text summarizer returns prose descriptions instead of executable Python for API helpers; the targeted fix works because a code-capable generative model can reason over programming context; repository access, tool execution, and tests still need explicit integration, restoring the ability to produce and review Python functions with useful syntax and unit-test coverage.
Incorrect Answers
A:If the failing function were to find related guidance when employees use very different wording from the source documents, this change could be relevant because vector search compares embeddings to retrieve semantically close text even when terms differ, provided index and query embeddings are compatible; the actual symptom is that the general text summarizer returns prose descriptions instead of executable Python for API helpers, which calls for a different repair.
B:If the failing function were to answer policy questions using the latest approved internal manuals instead of model recollection, this change could be relevant because a connected retrieval source gives an agent current authorized reference content that can be cited and updated separately from model weights; the actual symptom is that the general text summarizer returns prose descriptions instead of executable Python for API helpers, which calls for a different repair.
C:If the failing function were to answer natural-language questions about photographed equipment where the answer depends on visible details, this change could be relevant because a multimodal model accepts image evidence alongside text; a text-only model cannot inspect visual input unless a separate vision pipeline supplies it; the actual symptom is that the general text summarizer returns prose descriptions instead of executable Python for API helpers, which calls for a different repair.
E:If the failing function were to map queries and reference passages into vectors for semantic retrieval, this change could be relevant because embeddings encode text into vectors used by a similarity search system; they are not themselves a response-generating chatbot; the actual symptom is that the general text summarizer returns prose descriptions instead of executable Python for API helpers, which calls for a different repair.
Question 12
The engineering change request is to expose an authenticated business API as an agent tool. What is the most direct acceptance criterion?
- Measure exact-identifier match recall and precision on known part-number queries
- Test multi-turn follow-ups for correct reference resolution without cross-user leakage
- Verify preference recall, deletion behavior, and strict user-by-user isolation
- Trace tool invocations and compare returned statuses with authorized API ground truth
Explanation
Correct Answer
D:The accepted change is to expose an authenticated business API as an agent tool; this evaluation checks whether it can let an agent retrieve a live order status from the organization's transactional system, since an authenticated tool call obtains current application state at request time; stale embeddings cannot guarantee a transaction's latest status.
Incorrect Answers
A:This test concerns the ability to retrieve entries containing exact product part numbers and uncommon policy codes, for which keyword retrieval matches specific lexical tokens precisely, important when identifiers lack semantic meaning to an embedding model; it does not establish whether the change to expose an authenticated business API as an agent tool works for the current release.
B:This test concerns the ability to resolve a follow-up such as 'What about the second option?' using the user's previous turn, for which conversation state preserves necessary prior turns for reference resolution; it differs from a searchable repository of long-lived institutional documents; it does not establish whether the change to expose an authenticated business API as an agent tool works for the current release.
C:This test concerns the ability to personalize repeated interactions without placing all old transcripts in every prompt, for which explicitly persisted, consent-governed preference data supports bounded long-term personalization; transient chat state should not become an uncontrolled permanent profile; it does not establish whether the change to expose an authenticated business API as an agent tool works for the current release.
Question 13
An engineering team has a requirement to personalize repeated interactions without placing all old transcripts in every prompt. Which capability should be in the initial design?
- Ground responses in a search knowledge source
- Store approved long-term user preferences separately
- Combine keyword and vectors in hybrid search
- Expose an authenticated business API as an agent tool
Explanation
Correct Answer
B:For the stated requirement to personalize repeated interactions without placing all old transcripts in every prompt, explicitly persisted, consent-governed preference data supports bounded long-term personalization; transient chat state should not become an uncontrolled permanent profile; this is the mechanism that the system must implement.
Incorrect Answers
A:For a system whose objective is to answer policy questions using the latest approved internal manuals instead of model recollection, a connected retrieval source gives an agent current authorized reference content that can be cited and updated separately from model weights; this does not directly meet the required outcome to personalize repeated interactions without placing all old transcripts in every prompt.
C:For a system whose objective is to support queries mixing specific document codes with natural-language intent, hybrid retrieval runs lexical and vector queries together and merges ranked results; it helps preserve exact tokens while matching semantic concepts; this does not directly meet the required outcome to personalize repeated interactions without placing all old transcripts in every prompt.
D:For a system whose objective is to let an agent retrieve a live order status from the organization's transactional system, an authenticated tool call obtains current application state at request time; stale embeddings cannot guarantee a transaction's latest status; this does not directly meet the required outcome to personalize repeated interactions without placing all old transcripts in every prompt.
Question 14
During a regression test, the following unexpected result occurs: Relevant chunks are already in the candidate pool but weakly related snippets occupy the first positions. What should the engineer do?
- Run vector retrieval on document chunks using the same embedding space as queries
- Submit combined text and vector queries to a hybrid-capable search index
- Enable semantic ranking over searchable human-readable fields after initial retrieval
- Embed the queries and chunks with a compatible model and add vector retrieval
Explanation
Correct Answer
C:The fault signal is that relevant chunks are already in the candidate pool but weakly related snippets occupy the first positions; the targeted fix works because semantic reranking uses language understanding to reorder candidate search results; it does not replace indexing or generate new source documents, restoring the ability to improve ordering of an already retrieved set by how directly it answers a user's question.
Incorrect Answers
A:If the failing function were to find related guidance when employees use very different wording from the source documents, this change could be relevant because vector search compares embeddings to retrieve semantically close text even when terms differ, provided index and query embeddings are compatible; the actual symptom is that relevant chunks are already in the candidate pool but weakly related snippets occupy the first positions, which calls for a different repair.
B:If the failing function were to support queries mixing specific document codes with natural-language intent, this change could be relevant because hybrid retrieval runs lexical and vector queries together and merges ranked results; it helps preserve exact tokens while matching semantic concepts; the actual symptom is that relevant chunks are already in the candidate pool but weakly related snippets occupy the first positions, which calls for a different repair.
D:If the failing function were to map queries and reference passages into vectors for semantic retrieval, this change could be relevant because embeddings encode text into vectors used by a similarity search system; they are not themselves a response-generating chatbot; the actual symptom is that relevant chunks are already in the candidate pool but weakly related snippets occupy the first positions, which calls for a different repair.
Question 15
After implementing a workflow to apply semantic reranking to candidate passages, the team needs a robust release gate. Which check should it choose?
- Compare classification macro-F1, p95 latency, and per-request inference cost on representative tickets
- Track normalized discounted cumulative gain and top-rank relevance on judged query results
- Assess recall for paraphrases and meaning-preserving rewordings of known questions
- Measure exact-identifier match recall and precision on known part-number queries
Explanation
Correct Answer
B:The accepted change is to apply semantic reranking to candidate passages; this evaluation checks whether it can improve ordering of an already retrieved set by how directly it answers a user's question, since semantic reranking uses language understanding to reorder candidate search results; it does not replace indexing or generate new source documents.
Incorrect Answers
A:This test concerns the ability to categorize short support tickets under strict response-time and inference-budget limits, for which a compact model can meet narrow classification needs with fewer inference resources when validated against representative labeled examples; it does not establish whether the change to apply semantic reranking to candidate passages works for the current release.
C:This test concerns the ability to find related guidance when employees use very different wording from the source documents, for which vector search compares embeddings to retrieve semantically close text even when terms differ, provided index and query embeddings are compatible; it does not establish whether the change to apply semantic reranking to candidate passages works for the current release.
D:This test concerns the ability to retrieve entries containing exact product part numbers and uncommon policy codes, for which keyword retrieval matches specific lexical tokens precisely, important when identifiers lack semantic meaning to an embedding model; it does not establish whether the change to apply semantic reranking to candidate passages works for the current release.
Question 16
The team plans to select a small language model for bounded classification. Which test provides the clearest evidence that this change works?
- Measure image-grounded answer accuracy against a labeled set of equipment photos
- Check recall at k for paraphrased questions against known relevant passages
- Track normalized discounted cumulative gain and top-rank relevance on judged query results
- Compare classification macro-F1, p95 latency, and per-request inference cost on representative tickets
Explanation
Correct Answer
D:The accepted change is to select a small language model for bounded classification; this evaluation checks whether it can categorize short support tickets under strict response-time and inference-budget limits, since a compact model can meet narrow classification needs with fewer inference resources when validated against representative labeled examples.
Incorrect Answers
A:This test concerns the ability to answer natural-language questions about photographed equipment where the answer depends on visible details, for which a multimodal model accepts image evidence alongside text; a text-only model cannot inspect visual input unless a separate vision pipeline supplies it; it does not establish whether the change to select a small language model for bounded classification works for the current release.
B:This test concerns the ability to map queries and reference passages into vectors for semantic retrieval, for which embeddings encode text into vectors used by a similarity search system; they are not themselves a response-generating chatbot; it does not establish whether the change to select a small language model for bounded classification works for the current release.
C:This test concerns the ability to improve ordering of an already retrieved set by how directly it answers a user's question, for which semantic reranking uses language understanding to reorder candidate search results; it does not replace indexing or generate new source documents; it does not establish whether the change to select a small language model for bounded classification works for the current release.
Question 17
The acceptance criteria for a production AI solution require it to produce and review Python functions with useful syntax and unit-test coverage. Which design should be approved?
- Use an embedding model to represent meaning
- Use vector similarity search for paraphrases
- Deploy a multimodal model for visual reasoning
- Deploy a code-capable model for code-focused assistance
Explanation
Correct Answer
D:For the stated requirement to produce and review Python functions with useful syntax and unit-test coverage, a code-capable generative model can reason over programming context; repository access, tool execution, and tests still need explicit integration; this is the mechanism that the system must implement.
Incorrect Answers
A:For a system whose objective is to map queries and reference passages into vectors for semantic retrieval, embeddings encode text into vectors used by a similarity search system; they are not themselves a response-generating chatbot; this does not directly meet the required outcome to produce and review Python functions with useful syntax and unit-test coverage.
B:For a system whose objective is to find related guidance when employees use very different wording from the source documents, vector search compares embeddings to retrieve semantically close text even when terms differ, provided index and query embeddings are compatible; this does not directly meet the required outcome to produce and review Python functions with useful syntax and unit-test coverage.
C:For a system whose objective is to answer natural-language questions about photographed equipment where the answer depends on visible details, a multimodal model accepts image evidence alongside text; a text-only model cannot inspect visual input unless a separate vision pipeline supplies it; this does not directly meet the required outcome to produce and review Python functions with useful syntax and unit-test coverage.
Question 18
A new customer-facing AI service must improve ordering of an already retrieved set by how directly it answers a user's question. Which approach should the team select first?
- Use vector similarity search for paraphrases
- Combine keyword and vectors in hybrid search
- Apply semantic reranking to candidate passages
- Deploy a multimodal model for visual reasoning
Explanation
Correct Answer
C:For the stated requirement to improve ordering of an already retrieved set by how directly it answers a user's question, semantic reranking uses language understanding to reorder candidate search results; it does not replace indexing or generate new source documents; this is the mechanism that the system must implement.
Incorrect Answers
A:For a system whose objective is to find related guidance when employees use very different wording from the source documents, vector search compares embeddings to retrieve semantically close text even when terms differ, provided index and query embeddings are compatible; this does not directly meet the required outcome to improve ordering of an already retrieved set by how directly it answers a user's question.
B:For a system whose objective is to support queries mixing specific document codes with natural-language intent, hybrid retrieval runs lexical and vector queries together and merges ranked results; it helps preserve exact tokens while matching semantic concepts; this does not directly meet the required outcome to improve ordering of an already retrieved set by how directly it answers a user's question.
D:For a system whose objective is to answer natural-language questions about photographed equipment where the answer depends on visible details, a multimodal model accepts image evidence alongside text; a text-only model cannot inspect visual input unless a separate vision pipeline supplies it; this does not directly meet the required outcome to improve ordering of an already retrieved set by how directly it answers a user's question.
Question 19
During a regression test, the following unexpected result occurs: The assistant correctly handles typed questions but invents the state of controls shown only in a photo. What should the engineer do?
- Pass the actual image to a vision-capable model and ground its response in visible evidence
- Use a model evaluated for coding tasks and validate outputs with an isolated test runner
- Submit combined text and vector queries to a hybrid-capable search index
- Embed the queries and chunks with a compatible model and add vector retrieval
Explanation
Correct Answer
A:The fault signal is that the assistant correctly handles typed questions but invents the state of controls shown only in a photo; the targeted fix works because a multimodal model accepts image evidence alongside text; a text-only model cannot inspect visual input unless a separate vision pipeline supplies it, restoring the ability to answer natural-language questions about photographed equipment where the answer depends on visible details.
Incorrect Answers
B:If the failing function were to produce and review Python functions with useful syntax and unit-test coverage, this change could be relevant because a code-capable generative model can reason over programming context; repository access, tool execution, and tests still need explicit integration; the actual symptom is that the assistant correctly handles typed questions but invents the state of controls shown only in a photo, which calls for a different repair.
C:If the failing function were to support queries mixing specific document codes with natural-language intent, this change could be relevant because hybrid retrieval runs lexical and vector queries together and merges ranked results; it helps preserve exact tokens while matching semantic concepts; the actual symptom is that the assistant correctly handles typed questions but invents the state of controls shown only in a photo, which calls for a different repair.
D:If the failing function were to map queries and reference passages into vectors for semantic retrieval, this change could be relevant because embeddings encode text into vectors used by a similarity search system; they are not themselves a response-generating chatbot; the actual symptom is that the assistant correctly handles typed questions but invents the state of controls shown only in a photo, which calls for a different repair.
Question 20
A cloud engineering team must answer policy questions using the latest approved internal manuals instead of model recollection. What is the most appropriate Azure AI implementation?
- Use conversation state to retain dialogue context
- Deploy a code-capable model for code-focused assistance
- Store approved long-term user preferences separately
- Ground responses in a search knowledge source
Explanation
Correct Answer
D:For the stated requirement to answer policy questions using the latest approved internal manuals instead of model recollection, a connected retrieval source gives an agent current authorized reference content that can be cited and updated separately from model weights; this is the mechanism that the system must implement.
Incorrect Answers
A:For a system whose objective is to resolve a follow-up such as 'What about the second option?' using the user's previous turn, conversation state preserves necessary prior turns for reference resolution; it differs from a searchable repository of long-lived institutional documents; this does not directly meet the required outcome to answer policy questions using the latest approved internal manuals instead of model recollection.
B:For a system whose objective is to produce and review Python functions with useful syntax and unit-test coverage, a code-capable generative model can reason over programming context; repository access, tool execution, and tests still need explicit integration; this does not directly meet the required outcome to answer policy questions using the latest approved internal manuals instead of model recollection.
C:For a system whose objective is to personalize repeated interactions without placing all old transcripts in every prompt, explicitly persisted, consent-governed preference data supports bounded long-term personalization; transient chat state should not become an uncontrolled permanent profile; this does not directly meet the required outcome to answer policy questions using the latest approved internal manuals instead of model recollection.