EXAMTOPICS / TECHNOLOGY INSIGHTS

PMI-CPMAI: How to Evaluate an AI Project Idea

An AI project should earn the right to become an AI project. A strong team does not begin with a model, a vendor demonstration, or a mandate to “add AI.” It begins with a business problem, a measurable outcome, and an honest test of whether machine learning or generative AI is actually the best way to improve that outcome. The current PMI-CPMAI approach makes that discipline explicit by starting with business alignment and feasibility before teams commit heavily to data preparation and model development.

This early decision matters because AI can make an attractive prototype long before it proves it can operate reliably, legally, securely, and economically. A feasibility assessment therefore has to test more than technical possibility. It must examine the problem definition, the availability and rights of the data, the level of uncertainty the organization can tolerate, the integration burden, the cost of running the solution, and the decision that will be made if the evidence is disappointing.

Start with the business decision, not the model

The first question is not “Which model should we use?” It is “What decision, workflow, or customer outcome are we trying to improve?” That distinction prevents teams from turning a technology capability into a vague business case. A service organization might want to reduce time spent triaging support tickets. A manufacturer might want to detect defects earlier. A finance team might want to improve the quality of a forecast. Each statement describes a business result that can be evaluated before an AI technique is selected.

A useful problem statement identifies the current process, the people affected, the baseline performance, the consequence of error, and the intended improvement. “Build an AI assistant” says almost nothing about value. “Reduce the median time an experienced analyst spends locating approved policy guidance from eight minutes to two without increasing incorrect answers” gives the team something testable.

This framing also exposes non-AI alternatives. Better search, simpler rules, workflow redesign, a conventional statistical method, or a database query may solve the problem more cheaply and with less operational risk. Rejecting AI when a simpler method is better is a successful feasibility result, not a failed innovation effort.

Match the AI pattern to the kind of work

Once the problem is clear, the team can ask what kind of AI capability could plausibly help. Prediction, classification, anomaly detection, recommendation, optimization, document extraction, language generation, computer vision, and conversational retrieval have different data needs and failure modes. Treating them as one generic “AI” category hides the real engineering and governance requirements.

A predictive maintenance idea, for example, depends on historical sensor patterns and a usable definition of failure. A generative support assistant depends on trustworthy source material, retrieval quality, instruction design, and safeguards against unsupported answers. A fraud model may tolerate more false positives than a medical or safety decision. The same model family can be appropriate in one context and irresponsible in another because the cost of mistakes is different.

The feasibility review should therefore identify what the system is expected to infer or generate, how often that task occurs, how quickly the result must arrive, and what human review is available. The goal is not to choose a specific product too early. It is to see whether the proposed AI pattern fits the decision environment.

Data feasibility is often the real go-or-no-go gate

An attractive use case can collapse when the data is examined. Teams need to know what data exists, who owns it, how it was collected, whether it is representative, whether it can legally be used for the intended purpose, and how much work is required to make it reliable enough for development and operations.

Availability is not the same as fitness. A large dataset may contain missing values, inconsistent labels, hidden selection bias, duplicate records, or information that would not be available at prediction time. A document collection may include obsolete policies or conflicting versions. A customer dataset may include personal information that changes the acceptable architecture and retention rules. The operational consequences of those issues are why AI data privacy belongs in feasibility, not only in a late compliance review.

Teams should also ask whether the data pipeline can be sustained. A pilot that depends on one analyst exporting a spreadsheet manually each week is not evidence that a production system has a reliable data supply. Feasibility includes source stability, lineage, refresh rate, permissions, monitoring, and a named owner for data-quality problems.

Define success before the prototype creates enthusiasm

AI prototypes can look impressive because they demonstrate possibility. That makes objective success criteria especially important. The team should define what performance is needed, what business outcome that performance must enable, and how the result compares with the current process or a simpler baseline.

Technical metrics and business metrics serve different purposes. Precision, recall, error rate, latency, retrieval quality, or groundedness may describe model behavior. Cost per case, time saved, conversion, defect reduction, risk avoided, or customer satisfaction describes organizational value. A project needs a defensible connection between the two. A model can improve a benchmark without creating a meaningful outcome, while a modest technical improvement can be valuable if it removes a costly operational bottleneck.

Success criteria should also include failure thresholds. Teams need to know what result would cause them to redesign, narrow the use case, keep the human process, or stop the project. Without those thresholds, every disappointing result can be reframed as a reason to spend more.

Test technical feasibility in the real operating environment

Technical feasibility is broader than model accuracy. The solution has to connect to the systems that provide context and receive the result. It needs an authentication model, acceptable latency, capacity planning, logging, version control, and a way to recover when a dependency fails.

Integration can be the dominant cost. A classifier that performs well in a notebook may be difficult to embed in a legacy case-management platform. A retrieval-augmented assistant may require document ingestion, access-aware retrieval, source synchronization, and a user interface that makes uncertainty visible. An image model may create network or storage requirements that the original budget ignored.

The feasibility phase should therefore include a thin end-to-end path, not only an isolated model experiment. That does not mean building production infrastructure early. It means proving the hardest assumptions at the smallest responsible scale. Technology-driven project management is strongest when technical discovery changes the plan before sunk cost makes change politically difficult.

Risk, ethics, and security change the design

AI risk is not a single checklist item. Different systems can create privacy, bias, intellectual-property, cybersecurity, safety, regulatory, reputational, and operational risks. The feasibility question is whether those risks can be reduced to an acceptable level with controls that still preserve the business value.

A high-impact decision may require human approval, explanation, auditability, conservative confidence thresholds, or a restricted population. A generative system may need source grounding, prompt-injection defenses, content filtering, access controls, and an escalation path for uncertain outputs. A low-risk internal drafting tool may justify a lighter control model.

Risk should be analyzed alongside value because controls have cost. If the only safe architecture makes the system slower and more expensive than the business case can support, the correct decision may be not to proceed. Broad concerns about AI risks and limitations become useful only when translated into concrete failure modes, owners, and design choices for the proposed system.

Build the economic case around total cost, not model price

AI economics include more than training or API charges. Data acquisition, labeling, integration, evaluation, security, observability, human review, model serving, vendor contracts, change management, compliance, support, and retraining can dominate total cost. The project may also create variable consumption costs that increase as adoption grows.

A useful ROI case identifies the unit of value. If the system saves analyst time, estimate how much time can actually be redeployed rather than multiplying seconds saved by a theoretical hourly wage. If it increases conversion, distinguish causal improvement from normal variation. If it reduces risk, document the event probability and loss assumptions instead of hiding them in a single large benefit number.

Scenario ranges are often more credible than a single forecast. The team can model optimistic, expected, and conservative adoption; different inference volumes; or different levels of human review. The purpose is not to manufacture certainty. It is to understand which assumptions the investment depends on most.

Use a pilot to answer the riskiest questions

A feasibility pilot should be designed as an evidence-producing experiment. Its scope should be narrow enough to run quickly but realistic enough to test the assumptions that could kill the project: data quality, acceptable error, user behavior, integration, operational cost, or governance.

The pilot needs an entry condition, a measurement plan, and a decision at the end. “Successful demo” is not a decision. “Proceed if the system reaches the minimum accuracy and latency thresholds, users complete the workflow without bypass behavior, and projected unit economics remain within the approved range” is.

The strongest outcome may be a smaller use case, a revised process, or a stop decision. That is why feasibility belongs before major commitment. Across the broader PMI certification family, disciplined project decisions are about choosing work that can create value under real constraints. AI does not remove that responsibility; it makes it more important.