Comparing ERP Options and Recording Your Decision. Compare evidence, discuss differences and record the reasons.
A defensible ERP selection preserves the reasons behind the recommendation. It makes critical constraints visible, records what evaluators observed independently, and shows whether the preferred option changes when uncertain assumptions move.
The purpose is to give the decision maker a trustworthy account of the tradeoffs. A precise-looking total can be misleading when it combines missing evidence, incompatible judgments, and requirements that should never have been averaged together.
Use the evaluation protocol below when a shortlist is narrowing and stakeholder preferences are becoming strong. It is designed to keep evidence, interpretation, and decision authority connected without forcing every participant to agree.
Establish the rules before the room has a favorite
Before demonstrations begin, agree what the selection is meant to decide. Include the business outcomes, release boundary, constraints, evaluation criteria, and people authorized to accept tradeoffs.
Divide the decision into two layers.
The first contains non-negotiable conditions: outcomes or constraints that must be satisfied for the proposed scope to proceed. The responsible business, security, legal, or control owner should validate each condition and the evidence required. Do not classify a preference as mandatory merely to protect it from competition.
The second contains tradeable attributes: differences the organization is willing to balance, such as routine operating effort, maintenance complexity, deployment workload, or convenience. Define how these will be assessed before seeing the options.
A candidate with an unresolved mandatory condition should have a visible decision status. It should not become acceptable because attractive features elsewhere lift its weighted average. The organization may change its scope or requirement, but that is a separate, accountable decision.
Assign judgment to people who can make it
Create an evaluator map rather than inviting everyone to score every topic.
For each criterion, record:
- The person who performs or owns the affected work
- The specialist needed to examine technical, control, or commercial implications
- The evidence each evaluator should inspect
- The person who can resolve conflicting interpretations
- The executive who has authority to accept the resulting tradeoff
An accounts-payable specialist can assess exception handling they perform regularly. An integration lead can examine how a failure is detected and recovered. A sponsor can judge whether the overall recommendation supports the organization’s direction. These contributions are complementary.
Participants should be able to select “not assessed” when they lack the evidence or expertise. Forcing a score converts absence into apparent knowledge.
Set attendance around the decision being examined. Executives may join a detailed process session when their perspective is needed, but agree that observations will be recorded before senior participants express a preference. Leadership involvement is more useful when it protects candid evidence as well as providing direction.
Collect independent observations before discussion
Give evaluators time to record their findings before the group debrief. Ask for three separate entries: what happened, what it means for the requirement, and what evidence is still missing.
For example:
“An operator recovered the rejected transaction through an exception screen. The session did not show who is notified when the rejection occurs. Recovery was observed; detection and ownership remain unverified.”
That entry is more useful than “integration looks good.” It also gives another evaluator something specific to challenge or confirm.
Preserve the initial record. Discussion should improve the evaluation, and people should be free to change their minds. A revised assessment should explain the new evidence or reasoning. Overwriting the original makes it difficult to distinguish learning from pressure to converge.
Provide a private route for raising a concern when hierarchy or a potential conflict of interest makes an open discussion difficult. The facilitator should bring the substance into the decision process with appropriate discretion. Private feedback is not permission to hide material issues from the accountable decision maker.
Keep fit and confidence in separate fields
A useful scorecard has two panels.
Panel one describes fit. What outcome does the proposed approach achieve, under which conditions, and with what operating burden? Record the requirement, observed result, workaround, additional work, and residual limitation.
Panel two describes confidence. How well does the evidence support that assessment? Record whether the complete scenario was observed, only part was observed, the answer is a proposal, or the claim remains unverified. Identify the evidence reference and the next proof needed.
A low-confidence answer should not silently become a poor fit score. The capability may exist, but the selection team does not yet know enough. Equally, high confidence can support a negative conclusion: a clearly demonstrated limitation is useful evidence.
Avoid multiplying fit by confidence and treating the result as a new fact. That can conceal whether an option appears weak because it fails the need or because the evaluation is incomplete. Keep uncertainty visible so the next action is clear.
For the tradeable attributes only, a simple scoring scale may help summarize judgments. Define the anchors in terms the evaluators can apply. One illustrative scale is zero for an outcome not achieved, one for material operating concessions, two for an acceptable bounded constraint, and three for the agreed outcome under the tested conditions. Use “not assessed” outside the numeric scale.
The exact scale matters less than consistent interpretation. Calibrate it on a sample observation before formal scoring, and document any material disagreement about the anchors.
Treat disagreement as work to investigate
When scores differ, do not begin by averaging them. First ask what kind of disagreement exists.
An observation disagreement means evaluators remember different events. Return to the recorded evidence or repeat the scenario.
An interpretation disagreement means they agree on what happened but disagree about its consequences. Ask the process owner to clarify the operating requirement and investigate downstream effects.
A preference disagreement means both interpretations are reasonable, but the stakeholders value the tradeoff differently. That belongs with the accountable decision maker.
An evidence gap means one or both assessments rest on assumptions. Assign the missing proof rather than negotiating a compromise score.
For example, a finance evaluator may favor a detailed approval process while an operations evaluator sees unacceptable delay. They may both have observed the same behavior accurately. The next step is to examine decision timing, exception routes, transaction exposure, and authority. A midpoint score would describe none of that.
Test how easily the recommendation changes
Weighted scores can help organize tradeable attributes, provided the group understands their limitations. A sensitivity check asks whether reasonable changes in priorities alter the ranking.
Consider this entirely hypothetical example. Two candidates have already cleared the defined mandatory conditions. The team evaluates operating simplicity, maintainability, and rollout effort using the illustrative zero-to-three scale.
Candidate A receives three, one, and two respectively. Candidate B receives two, three, and one. These are invented judgments for explaining the calculation, not product assessments.
With weights of 40%, 35%, and 25%, Candidate A’s weighted score is 2.05 and Candidate B’s is 2.10. With weights of 50%, 25%, and 25%, Candidate A scores 2.25 and Candidate B scores 2.00.
The arithmetic reveals that the apparent winner depends on the balance between operating simplicity and maintainability. It does not reveal that one set of weights is correct, or that the small initial difference is meaningful.
The decision maker now has a better question: which tradeoff reflects the organization’s actual operating model and capacity? If maintainability has not been demonstrated well, additional proof may be more useful than further debate over percentages.
Test only plausible alternatives and document why they are plausible. Do not change weights repeatedly until the desired option wins. Also examine uncertain scores: if a reasonable reassessment of one incompletely tested attribute reverses the recommendation, name that dependency explicitly.
Preserve a record of deliberate overrides
The final choice need not mechanically follow a score. A decision maker may legitimately choose a different option because of a documented commercial, organizational, or strategic consideration. The integrity problem arises when the rationale is hidden or the scoring is rewritten to justify a decision already made.
Use a compact decision record:
- Decision and proposed scope: ______
- Mandatory conditions and their evidence status: ______
- Strongest reasons for the recommendation: ______
- Material disadvantages being accepted: ______
- Important dissent and the response: ______
- Unresolved assumptions or conditions: ______
- Sensitivity findings that could change the choice: ______
- Accountable decision maker and approval date: ______
- Events that require reconsideration: ______
Keep conflicts of interest visible. If an evaluator has a commercial relationship or a stake in a particular outcome, record it and decide how their contribution should be reviewed. Disclosure is part of interpreting evidence; it is not automatically a reason to discard expertise.
Finish with conditions that somebody owns
A conditional recommendation is useful only when its conditions are specific. “Proceed subject to integration confirmation” is too broad. Identify the scenario, environment, expected result, proof owner, reviewer, and latest decision point before an irreversible commitment.
Likewise, a concern accepted for later resolution needs a funded owner and a clear boundary. The selection decision should not create an unowned implementation problem simply by moving it to a different document.
Before approving the recommendation, ask an informed person who missed the demonstrations to read the record. Can they explain why the chosen option is preferred, what evidence is incomplete, and which disadvantages the organization is accepting? If they cannot, strengthen the record before polishing the presentation.
A useful selection process can accommodate disagreement and executive judgment. Its standard is that important evidence survives, uncertainty remains visible, and the person making the decision can explain the consequences they are choosing.
