Question 180
Reusable pipeline parts must remain stable as teams improve their ML workflows. An operator is tasked to stop new jobs from using an obsolete feature extractor while preserving traceability for historic runs. Which mechanism provides that capability?
- Pinned component reference
- Component contract change
- Component retirement plan
- Pipeline code versioning
Explanation
Correct Answer
C:Deprecation and reference management reduce new dependency on old assets. Deleting history blindly can undermine auditing of previous model versions.
Incorrect Answers
A:A versioned component reference avoids unreviewed behavior changes; it supports deterministic rollout separately from data version management; the outstanding need is to stop new jobs from using an obsolete feature extractor while preserving traceability for historic runs.
B:Explicit input and output contract changes let consumers adapt deliberately; silent schema changes may break downstream training at runtime; this approach leaves the team unable to stop new jobs from using an obsolete feature extractor while preserving traceability for historic runs.
D:Source revision tracking distinguishes execution topology changes from mere data refreshes; a run timestamp alone cannot fully reproduce its implementation; the requirement, by contrast, is to stop new jobs from using an obsolete feature extractor while preserving traceability for historic runs.
Question 181
A review board compares candidates trained on the same prediction task. The immediate design goal is to choose between model candidates using an untouched evaluation set rather than only training performance. Which approach addresses that goal?
- Segment-level evaluation
- Cross-validation summary
- Held-out test evaluation
- Baseline-model comparison
Explanation
Correct Answer
C:A held-out set tests generalization to unseen examples. Reusing it repeatedly for tuning can leak selection information and bias reported performance.
Incorrect Answers
A:Segment metrics expose uneven performance across populations or operating conditions; aggregate averages can mask concentrated harm; the requirement, by contrast, is to choose between model candidates using an untouched evaluation set rather than only training performance.
B:Cross-validation examines performance across several partitions; it can reveal unstable models that look strong on one favorable split; the team would still need to choose between model candidates using an untouched evaluation set rather than only training performance.
D:A baseline gives a meaningful reference for improvement; high absolute accuracy alone may be unhelpful for imbalanced labels; the implementation under review needs to choose between model candidates using an untouched evaluation set rather than only training performance.
Question 182
A review board compares candidates trained on the same prediction task. The team is required to compare average quality and variability across training folds before selecting a robust candidate. Which option is the best fit?
- Confidence interval reporting
- Cross-validation summary
- Segment-level evaluation
- Held-out test evaluation
- Baseline-model comparison
Explanation
Correct Answer
B:Cross-validation examines performance across several partitions. It can reveal unstable models that look strong on one favorable split.
Incorrect Answers
A:Intervals characterize statistical uncertainty around an estimate when calculated appropriately; a tiny point-estimate gain should not automatically trigger promotion; the implementation under review needs to compare average quality and variability across training folds before selecting a robust candidate.
C:Segment metrics expose uneven performance across populations or operating conditions; aggregate averages can mask concentrated harm; this alternative does not achieve the requirement to compare average quality and variability across training folds before selecting a robust candidate.
D:A held-out set tests generalization to unseen examples; reusing it repeatedly for tuning can leak selection information and bias reported performance; the actual design criterion is to compare average quality and variability across training folds before selecting a robust candidate.
E:A baseline gives a meaningful reference for improvement; high absolute accuracy alone may be unhelpful for imbalanced labels; the outstanding need is to compare average quality and variability across training folds before selecting a robust candidate.
Question 183
A review board compares candidates trained on the same prediction task. An operator is tasked to require a sophisticated model to outperform a simple established baseline on the same dataset and metric. Which mechanism provides that capability?
- Latency-aware model selection
- Segment-level evaluation
- Confidence interval reporting
- Baseline-model comparison
Explanation
Correct Answer
D:A baseline gives a meaningful reference for improvement. High absolute accuracy alone may be unhelpful for imbalanced labels.
Incorrect Answers
A:Production selection often balances accuracy with serving latency and resource cost; a leaderboard based only on validation accuracy can miss operational infeasibility; the actual design criterion is to require a sophisticated model to outperform a simple established baseline on the same dataset and metric.
B:Segment metrics expose uneven performance across populations or operating conditions; aggregate averages can mask concentrated harm; the outstanding need is to require a sophisticated model to outperform a simple established baseline on the same dataset and metric.
C:Intervals characterize statistical uncertainty around an estimate when calculated appropriately; a tiny point-estimate gain should not automatically trigger promotion; this approach leaves the team unable to require a sophisticated model to outperform a simple established baseline on the same dataset and metric.
Question 184
A review board compares candidates trained on the same prediction task. The immediate design goal is to assess whether a small measured improvement could plausibly reflect evaluation sampling noise. Which approach addresses that goal?
- Held-out test evaluation
- Confidence interval reporting
- Segment-level evaluation
- Latency-aware model selection
Explanation
Correct Answer
B:Intervals characterize statistical uncertainty around an estimate when calculated appropriately. A tiny point-estimate gain should not automatically trigger promotion.
Incorrect Answers
A:A held-out set tests generalization to unseen examples; reusing it repeatedly for tuning can leak selection information and bias reported performance; the needed capability is to assess whether a small measured improvement could plausibly reflect evaluation sampling noise.
C:Segment metrics expose uneven performance across populations or operating conditions; aggregate averages can mask concentrated harm; the team would still need to assess whether a small measured improvement could plausibly reflect evaluation sampling noise.
D:Production selection often balances accuracy with serving latency and resource cost; a leaderboard based only on validation accuracy can miss operational infeasibility; the specified outcome instead requires the team to assess whether a small measured improvement could plausibly reflect evaluation sampling noise.
Question 185
A review board compares candidates trained on the same prediction task. The team is required to prefer a slightly less accurate candidate when it alone meets strict production response-time limits. Which option is the best fit?
- Cross-validation summary
- Latency-aware model selection
- Confidence interval reporting
- Segment-level evaluation
Explanation
Correct Answer
B:Production selection often balances accuracy with serving latency and resource cost. A leaderboard based only on validation accuracy can miss operational infeasibility.
Incorrect Answers
A:Cross-validation examines performance across several partitions; it can reveal unstable models that look strong on one favorable split; the specified outcome instead requires the team to prefer a slightly less accurate candidate when it alone meets strict production response-time limits.
C:Intervals characterize statistical uncertainty around an estimate when calculated appropriately; a tiny point-estimate gain should not automatically trigger promotion; the requested task remains to prefer a slightly less accurate candidate when it alone meets strict production response-time limits.
D:Segment metrics expose uneven performance across populations or operating conditions; aggregate averages can mask concentrated harm; this alternative does not achieve the requirement to prefer a slightly less accurate candidate when it alone meets strict production response-time limits.
Question 186
A review board compares candidates trained on the same prediction task. An operator is tasked to check whether an aggregate improvement conceals substantial degradation for an important customer subgroup. Which mechanism provides that capability?
- Cross-validation summary
- Held-out test evaluation
- Latency-aware model selection
- Segment-level evaluation
Explanation
Correct Answer
D:Segment metrics expose uneven performance across populations or operating conditions. Aggregate averages can mask concentrated harm.
Incorrect Answers
A:Cross-validation examines performance across several partitions; it can reveal unstable models that look strong on one favorable split; this alternative does not achieve the requirement to check whether an aggregate improvement conceals substantial degradation for an important customer subgroup.
B:A held-out set tests generalization to unseen examples; reusing it repeatedly for tuning can leak selection information and bias reported performance; the actual design criterion is to check whether an aggregate improvement conceals substantial degradation for an important customer subgroup.
C:Production selection often balances accuracy with serving latency and resource cost; a leaderboard based only on validation accuracy can miss operational infeasibility; the outstanding need is to check whether an aggregate improvement conceals substantial degradation for an important customer subgroup.
Question 187
A predictive modeling project must produce unbiased performance estimates. The immediate design goal is to preserve minority-class proportions across training and test splits for a categorical prediction task. Which approach addresses that goal?
- Group-aware split
- Stratified sampling
- Label leakage audit
- Preprocessing fitted on training only
Explanation
Correct Answer
B:Stratification helps create representative partitions when labels are imbalanced. It cannot resolve leakage from duplicate entities in both splits.
Incorrect Answers
A:Group-aware splitting reduces entity-level leakage when many rows share a source; ordinary random sampling can overestimate generalization to unseen customers; this approach leaves the team unable to preserve minority-class proportions across training and test splits for a categorical prediction task.
C:Post-outcome features can make evaluation unrealistically strong while being unavailable at inference time; feature provenance and availability timing must be checked; the requirement, by contrast, is to preserve minority-class proportions across training and test splits for a categorical prediction task.
D:Fit-on-train preprocessing avoids importing held-out distribution information into model construction; fitting on the full dataset creates a subtle leakage path; the team would still need to preserve minority-class proportions across training and test splits for a categorical prediction task.
Question 188
A predictive modeling project must produce unbiased performance estimates. The team is required to train on earlier observations and evaluate on later dates for a genuine forward-looking forecast. Which option is the best fit?
- Label leakage audit
- Stratified sampling
- Preprocessing fitted on training only
- Time-based split
Explanation
Correct Answer
D:Temporal splitting reflects future deployment conditions and avoids seeing later events during training. Random row splits can leak future information.
Incorrect Answers
A:Post-outcome features can make evaluation unrealistically strong while being unavailable at inference time; feature provenance and availability timing must be checked; the team would still need to train on earlier observations and evaluate on later dates for a genuine forward-looking forecast.
B:Stratification helps create representative partitions when labels are imbalanced; it cannot resolve leakage from duplicate entities in both splits; the specified outcome instead requires the team to train on earlier observations and evaluate on later dates for a genuine forward-looking forecast.
C:Fit-on-train preprocessing avoids importing held-out distribution information into model construction; fitting on the full dataset creates a subtle leakage path; the implementation under review needs to train on earlier observations and evaluate on later dates for a genuine forward-looking forecast.
Question 189
A predictive modeling project must produce unbiased performance estimates. An operator is tasked to keep records for the same customer in one partition to prevent correlated examples leaking across train and validation. Which mechanism provides that capability?
- Immutable test set
- Group-aware split
- Preprocessing fitted on training only
- Stratified sampling
- Label leakage audit
Explanation
Correct Answer
B:Group-aware splitting reduces entity-level leakage when many rows share a source. Ordinary random sampling can overestimate generalization to unseen customers.
Incorrect Answers
A:A locked test set provides a less biased estimate after model selection; using it iteratively for development turns it into another validation set; the requested task remains to keep records for the same customer in one partition to prevent correlated examples leaking across train and validation.
C:Fit-on-train preprocessing avoids importing held-out distribution information into model construction; fitting on the full dataset creates a subtle leakage path; the actual design criterion is to keep records for the same customer in one partition to prevent correlated examples leaking across train and validation.
D:Stratification helps create representative partitions when labels are imbalanced; it cannot resolve leakage from duplicate entities in both splits; the outstanding need is to keep records for the same customer in one partition to prevent correlated examples leaking across train and validation.
E:Post-outcome features can make evaluation unrealistically strong while being unavailable at inference time; feature provenance and availability timing must be checked; this approach leaves the team unable to keep records for the same customer in one partition to prevent correlated examples leaking across train and validation.
Question 190
A predictive modeling project must produce unbiased performance estimates. The immediate design goal is to learn normalization or imputation statistics only from training data before applying them to validation. Which approach addresses that goal?
- Time-based split
- Preprocessing fitted on training only
- Label leakage audit
- Group-aware split
Explanation
Correct Answer
B:Fit-on-train preprocessing avoids importing held-out distribution information into model construction. Fitting on the full dataset creates a subtle leakage path.
Incorrect Answers
A:Temporal splitting reflects future deployment conditions and avoids seeing later events during training; random row splits can leak future information; the outstanding need is to learn normalization or imputation statistics only from training data before applying them to validation.
C:Post-outcome features can make evaluation unrealistically strong while being unavailable at inference time; feature provenance and availability timing must be checked; the needed capability is to learn normalization or imputation statistics only from training data before applying them to validation.
D:Group-aware splitting reduces entity-level leakage when many rows share a source; ordinary random sampling can overestimate generalization to unseen customers; the requirement, by contrast, is to learn normalization or imputation statistics only from training data before applying them to validation.