Supplier performance scorecards: the metrics, the weights, and the arithmetic

How to score each metric: targets, floors, and the points formula

Raw percentages are not comparable across metrics. 96% on-time and 96% document accuracy are not the same quality of performance, because the useful range differs. Nobody runs 60% document accuracy and stays approved, but 88% on-time is depressingly common. So convert every metric to points on a shared 0 to 100 scale using two numbers: a target that earns full marks, and a floor that earns zero.

For metrics where higher is better, points are the distance from the floor as a share of the floor-to-target range. For inverted metrics where lower is better — lead time variance in days, response time in hours, defects in PPM — swap the terms.

Adjusting the weights for your operation

The default weighting is built for a manufacturer or distributor with a mix of domestic and imported inbound. It is a reasonable starting point and a bad finishing point. The weights should reflect what a failure on that metric actually costs you, and that is entirely a function of your operation.

If you run sequenced or just-in-time assembly, lead time variance and on-time dominate, because a two-day swing consumes buffer you do not carry. Line stoppage cost is the reason. Published figures for automotive line-down run into the tens of thousands of dollars per minute at OEM final assembly, and typically the hundreds to low thousands at a tier-one plant, driven by takt time, unit contribution margin, and whether the shift can be recovered on overtime. Use your own number — your finance team already has it, and the point is only that whatever it is, it makes variance expensive.

If you supply a graded retailer, in-full weighs as heavily as on-time, because you are being scored on both and penalised for either. Retailer penalty programs commonly run 1 to 3 percent of order value per non-compliant order, and a short-shipped line from your supplier becomes a short-shipped line to your customer. That is the same arithmetic covered in the OTIF guide, one tier upstream.

If you operate under regulatory scope — pharma, defense, aerospace, food — document accuracy and defects carry the weight, because a missing certificate of analysis or certificate of origin does not slow a shipment down, it stops it. A customs hold on incorrect paperwork typically costs two to five days plus per-diem charges, and demurrage and detention at North American ports commonly runs $150 to $400 per container per day after free time, escalating in tiers.

And if you are the 3PL or contract manufacturer rather than the shipper, run two scorecards from the same record: one on the vendors you buy from, and one per client on the inbound they direct to you. The second is the one that ends the argument about whose miss it was, because a client-nominated supplier landing late is a fact you can produce at line level instead of a claim you have to defend.

Pick the profile closest to your operation and adjust from there. The only hard rule: the weights must sum to 100, and you must publish them to your suppliers before the period they apply to.

Where the data actually lives, and which joins break

The metrics above are not hard to define. They are hard to populate. A single PO line's story is spread across at least three systems, and the joins between them break in specific, predictable ways — the same seven ways, at almost every company.

The promise date lives in the ERP. The arrival lives in the WMS. The transport event lives in the TMS or the carrier's feed. The quantity exists in three places — PO line, receipt, invoice — and they disagree. The ASN arrives by EDI and is never diffed against what turned up. Defects surface in the QMS weeks after the receipt was closed.

The single most common defect in supplier on-time reporting is this: most ERPs overwrite the confirmed delivery date on the PO schedule line when a supplier reschedules. By the time the goods arrive, the PO says the supplier was on time, because the PO says the promise was the date they eventually hit. Unless you are capturing the change log and snapshotting the first confirmed date, your on-time number is measuring nothing. Track reschedules as a separate metric — schedule change rate — so the supplier cannot quietly move the goalposts and score full marks.

The second most common defect is unit of measure. The PO is in cases, the receipt is keyed in eaches, and the conversion factor is maintained in a third place. A 96% line fill becomes 100% or 12% depending on which direction the conversion goes wrong, and it will not be obvious from the summary.

The third is receipt timestamps. A WMS goods receipt records when a receiver keyed the transaction, not when the truck hit the gate. A Friday 18:00 arrival receipted Monday 08:15 reads as a three-day miss. If you have a gate or yard log, join to that. If you do not, at minimum agree a business-day rule with your suppliers and apply it consistently.

And one that is structurally unfixable if you get the terms wrong: under Ex Works or FCA terms, the supplier's obligation ends at their dock. If you grade them on delivery date, you are grading your own forwarder. Measure ready-to-ship date and booking notification accuracy instead, and grade the carrier separately. The same logic applies in reverse under DDP or DAP, where delivery date is fair game.

Running the review without a spreadsheet argument

The scorecard is the easy part. The review is where these programs live or die, and they die in a very specific way: forty minutes of the meeting spent disputing whether line 4417 was actually late, and five minutes on what changes next quarter. Four rules stop that.

First, the pre-read rule. Publish the score and the full line-level detail five business days before the meeting. Disputes must be filed in writing at least two business days before, citing PO line numbers. Anything not disputed in advance is accepted for the period and can be corrected in the next one. This single rule reclaims most of the meeting, because a supplier who has to reconcile line numbers in writing usually finds you were right.

Second, publish your own performance back to them. Average PO lead time actually given versus the quoted lead time, forecast accuracy at the horizon they plan against, schedule change rate, and payment on time. If your average PO gives 8 days of notice against a 21-day quoted lead time, their on-time score is your problem and you are about to pay a corrective-action consultant to discover that. This reciprocal page is the fastest way to turn an adversarial review into a working one.

Third, bring cost, not percentages. “You were 90% on time” is a debate. The Halvorsen quarter, costed out: 41 late lines drove 6 expedited air moves at an average $4,200 premium over the planned ocean lane, which is $25,200. Premiums vary widely by lane and consignment, from a few hundred dollars for a regional truck upgrade to five figures for an intercontinental charter, so use your own actuals. Add roughly 3 hours of planner rework per exception — 41 lines, 123 hours — at a $65 fully loaded hourly rate (planner rates realistically run $45 to $85 loaded depending on geography and seniority), which is $7,995. Add the $13,855 of working capital locked in extra safety stock on one SKU from the variance calculation. That is $33,195 of expensed cost plus $13,855 of tied-up capital, from one supplier, in one quarter. Nobody argues with that the way they argue with a percentage.

Where this gets hard, and what Orkestra actually does about it

Section six is the bottleneck, and it is not a method problem. One PO line's story sits in four places: the promise and its change history in the ERP, the receipt and the received quantity in the WMS, the transport events in the TMS or the carrier feed, the ASN in the EDI stream. Assembling that by hand is a month-end exercise, which is why most scorecard programs end up grading last quarter and spending the review arguing about the data instead of the supplier.

Orkestra is a layer over the ERP, TMS, and WMS you already own. It does not replace any of them. What it does for this problem specifically is hold one record per PO line — first confirmed date and its change history, ready-to-ship, transport events, arrival, received quantity by line, ASN versus actual, document status — so the number assembles continuously instead of being rebuilt in a workbook the week before the review. Order management holds the PO-to-receipt lifecycle. Analytics runs your metric definitions against that record, so when a supplier disputes line 4417 you open the line, not the workbook. And because the record is live rather than retrospective, the Exception Monitoring Agent can flag an inbound line drifting past its confirmed date while there is still a week to expedite, split, or re-promise — which over a quarter is worth more than the score itself.

Supplier on-time in-full is the same measurement problem as customer OTIF, one tier upstream, and it breaks the same way: the data existed in time to act, but it lived in four systems with no owner and no workflow. The customer-facing half of it is in the OTIF guide.

DBW, a global automotive supplier, replaced spreadsheet-driven coordination with real-time visibility and cut supply chain costs 18%. Matalco unified supply chain data across its systems and partners. Neither replaced an ERP, TMS, or WMS to do it.