A concrete example

A grocery distribution planner needs to decide tonight how many pallets to route to each of 40 stores for tomorrow. The demand at each store is uncertain, so the planner first runs a gradient-boosted tree model that predicts tomorrow’s demand at each store from features: day of week, weather, local events, recent POS (Point-of-Sale) data. Those predictions are then handed to a vehicle routing solver as fixed demand parameters, and the solver produces a pallet-routing plan.

The ML model is trained to minimise mean squared forecast error across all 40 stores. It does this well. But the routing solver is very sensitive to under-predictions at stores near the capacity limit of a route — a 10-pallet shortfall at one store can cascade into two extra vehicle journeys. The same 10-pallet shortfall at a store with slack capacity costs almost nothing. Training on MSE treats all prediction errors as equally bad. The result is a routing plan that performs fine in expectation but generates expensive recourse trips far more often than a decision-aware model would.

The fix is to train the demand model not on MSE but on a loss that measures how much each forecast error degrades the routing plan — a decision loss. This is harder to implement but directly optimises the objective the business actually cares about.

Why practitioners misread this

It is not a critique of using ML in optimisation pipelines. Predict-then-Optimize is the dominant architecture precisely because it is modular, interpretable, and easy to maintain. The critique is narrower: that MSE-trained predictors are not calibrated to the sensitivity structure of the downstream solver. Swapping to a decision-aware loss, or at minimum understanding which forecast errors are costly versus cheap, recovers most of the gap without abandoning the pipeline.

The problem is worst when solver sensitivity is asymmetric and concentrated. If the solver responds roughly equally to under- and over-prediction across all inputs, Predict-then-Optimize works well. It breaks down when there are a small number of ‘hinge’ inputs where the solver is highly sensitive in one direction (e.g. capacity limits in vehicle routing, budget limits in portfolio allocation, lead-time commitments in scheduling). Identifying those hinge inputs is the practical diagnostic step before deciding whether decision-aware training is worth the implementation cost.

The gap is real but often small in practice. Multiple empirical studies find that decision-aware training closes 10–30 % of the optimality gap attributable to forecast error in routing and inventory settings. For most operations, that is material but not transformative. The business case for decision-aware training rises sharply when hinge inputs are concentrated (routing capacity limits, power-grid ramp constraints, pharmaceutical batch yields) and falls when errors are diffuse across many small inputs.

Where this shows up in practice

Predict-then-Optimize is the standard architecture for any ML-guided supply chain, logistics, energy dispatch, or workforce scheduling system. It appears wherever a demand or cost forecast feeds directly into a planning or routing solver. The decision-awareness gap is most consequential in vehicle routing (capacity hinge), unit commitment (ramp-rate hinge), portfolio rebalancing (risk-budget hinge), and pharmaceutical batch scheduling (yield hinge). Today’s CVRP paper by Tomazella et al. addresses the downstream consequence of demand uncertainty in routing; the Predict-then-Optimize framing explains why the upstream forecasting step needs to be designed with that sensitivity in mind.