A project can meet its launch date and approved budget while leaving the business with slower decisions, persistent manual work and weak data. It can also exceed its original plan while creating substantial operational value. Budget and schedule matter, but they answer delivery questions. They cannot by themselves establish whether the investment improved the business.
ERP success requires a second measurement system: one that connects the capabilities delivered to changes in work and then to outcomes the organization values. That system should be designed before implementation, while the baseline still exists and the promised benefits can be challenged.
The executive question is not simply whether the platform went live. It is what changed, for whom, at what ongoing cost, with what evidence, and whether the improvement will endure.
Keep delivery, operation and value distinct
Delivery measures describe the program: scope completed, defects resolved, spending and milestones. Operational measures describe how the new service works: transaction completion, availability, exception queues, data quality and support demand. Value measures describe business consequences: better customer service, lower avoidable cost, improved cash timing or the ability to support a new business model.
These categories are related but not interchangeable. A low defect count may reflect effective delivery, weak testing or reluctant reporting. High transaction volume may show adoption or simply unavoidable use. A shorter close may reflect better reconciliation or a decision to leave more issues unresolved.
For each benefit, describe the causal chain. A common receipt process may improve the timeliness of inventory information, reduce unmatched invoices and release accounts-payable capacity. Each link needs evidence. If the receipt process is inconsistent, the downstream benefit may fail even when the software functions as designed.
GAO’s 2010 review of Department of Defense ERP modernization recommended quantitative performance measures tied to intended business capabilities. The relevant lesson is the need to assess the capabilities the investment was supposed to provide. It is not a commercial benchmark or a claim that one metric set fits all ERP programs.
Establish a baseline that can survive scrutiny
A baseline needs more than a number in a business case. Record the definition, population, period, data source and calculation. State exclusions and known quality issues. Explain whether the period was normal, seasonal or affected by an unusual event.
Measure distributions where they matter. The average time to resolve an order exception can improve while a small set of high-value orders remains stuck for weeks. A median, a high-percentile measure and a count of overdue cases may together provide a more useful picture than the mean alone.
Normalize thoughtfully. Cost per order, exceptions per invoice and hours per close can reveal changes that raw totals conceal. But denominators can also hide changes in complexity. A shift from large customized orders to small standard orders may improve the average without any system benefit.
Preserve enough underlying data to restate the comparison if definitions change. If historical data is unreliable, say so. Use a prospective observation period or a carefully designed sample rather than presenting an invented precise baseline. Measurement uncertainty belongs in the decision, not in a footnote that nobody reads.
A hypothetical order process shows why definitions matter
Imagine a distributor measuring whether its new ERP reduced order corrections. All figures and circumstances are illustrative.
Before implementation, it processes 10,000 orders in a month and corrects 400, a 4% correction rate. After stabilization, it processes 12,000 orders and corrects 360, a 3% rate. The rate has improved by one percentage point, and the raw number of corrections has also fallen despite higher volume.
If the old 4% rate had continued at the new volume, the company would expect 480 corrections. The observed difference is 120 corrections. At an assumed ten minutes per correction, that represents 20 hours of capacity per month. The calculation is a normalized comparison, not proof that ERP caused the entire change.
The team checks whether the order mix, correction definition and observation window are comparable. It finds that the sales department also introduced a new validation checklist during the same period. The improvement may reflect the combined process and system change. Claiming that the software alone created the result would overstate the evidence.
The team also examines unfinished work. If employees stopped labeling problems as corrections and instead left orders in a pending queue, the metric could improve while customers waited longer. It therefore pairs correction rate with pending-order age and on-time confirmation. It samples records to verify that classifications remain consistent.
Finally, leadership decides what to do with the 20 hours. If the team uses them to absorb growth without additional overtime, the realized benefit may include avoided overtime expense. If the hours support better customer follow-up, the benefit is capacity redeployment with a different outcome to measure. Neither should automatically be recorded as payroll savings.
The example produces a useful conclusion: the combined operating change appears to have improved correction performance, subject to stated comparability checks, and the business can investigate how released capacity is used. It avoids a more impressive but unsupported ROI claim.
Distinguish observation from attribution
A before-and-after change can be real without being caused by the implementation. Demand, staffing, supplier behavior, policy changes and seasonality may all affect results. Leadership needs an attribution approach proportionate to the size and consequence of the benefit claim.
GAO’s Designing Evaluations: 2012 Revision distinguishes ongoing performance measurement from evaluation that examines context and, in some designs, causal effects. That distinction is valuable for ERP benefits: a dashboard can show that performance changed, while additional analysis is needed to explain why.
Where practical, compare similar units introduced to the new process at different times. Check whether their trends were comparable before rollout and whether other changes affected them differently. A comparison group is not automatically valid simply because it uses the old system.
Where a credible comparison is unavailable, document the mechanism, competing explanations and supporting evidence. Transaction traces, staff observations and process measures can help establish a reasoned contribution claim. State the limits. A transparent conclusion about contribution is more useful than false certainty about causation.
Do not delay every operational improvement until a formal study is complete. The degree of rigor should match the decision. A small workflow adjustment needs less evidence than a claim used to justify a major expansion or a significant workforce commitment.
Measure adoption through completed work
Logins and training completion are weak proxies for productive adoption. Employees may log in because they must, then perform the real work elsewhere. Training attendance does not show whether users can handle exceptions safely.
Assess whether intended work is completed through the designed process, whether users understand consequential choices and whether shadow processes persist. Observe representative tasks and ask employees to explain their decisions. Include occasional users and complex cases rather than testing only expert champions.
Pair adoption measures with usability and control evidence. Fast processing is not desirable if users bypass approvals or choose default codes that undermine reporting. Conversely, a temporary increase in support questions can indicate that staff are surfacing problems rather than concealing them.
Measure the burden of use as well. If a new process moves data entry from finance to sales, evaluate the total effort and its effect on customer-facing work. A department-level productivity gain can conceal an enterprise-wide loss.
Create a small benefits record with an accountable owner
For each material benefit, maintain a practical record containing the intended outcome, baseline, target or acceptable range, operational mechanism, measure definition, data source, owner and review date. Add dependencies, ongoing cost and the evidence required to claim realization.
This is a working management tool, not a validated benefits-scoring framework. Its purpose is to support decisions. Keep the set small enough that owners can investigate deviations rather than merely populate a dashboard.
Targets should come from the business case and operational feasibility, not borrowed benchmark percentages. Use a range when uncertainty is material. Define a balancing measure that would reveal an undesirable side effect, such as faster invoice processing paired with duplicate-payment detection and unresolved exceptions.
Benefits also need a decision rule. If the outcome is below expectation, who decides whether to improve training, change the process, modify the system or revise the original assumption? A red indicator without an owner or response path is a reporting artifact rather than management.
Review value after the project team leaves
Some outcomes should improve quickly after stabilization; others require several business cycles. Match the review cadence to the mechanism. Daily incident monitoring is appropriate during transition, while a seasonal inventory benefit may require a longer observation period.
Record the cost of maintaining the improvement. New administration, support, integration and data-governance work may offset part of the gross benefit. A capability can still be worthwhile, but the organization should understand its net operating effect.
Keep the original case visible without treating it as untouchable. A strategy change may make an old benefit irrelevant and create a new priority. Revise the forward-looking plan transparently while preserving the original commitment for accountability. Quietly replacing targets after launch makes success impossible to interpret.
There is also a risk of excessive measurement. Detailed tracking can cost more than it reveals, encourage gaming or distract staff from service. Retire measures that no longer inform decisions. Retain enough evidence to verify material claims and detect deterioration.
A successful ERP investment leaves the business able to perform important work more reliably, economically or flexibly. The measurement system should make that improvement visible and challenge it when the evidence is weak. On-time delivery is worth recognizing. Lasting operational value must be demonstrated separately.
