The hardest forecasting decision is not which model to use!
When you have many entities to forecast, don't just build a better model. Build a better routing system.
TL;DR: Most forecasting pipelines invest heavily in model selection but apply it uniformly. Before asking which model is best, ask what state your series is in. Build a routing system, not just a better model.
The Problem
In a multi-entity time series forecasting problem, I spent more time on routing than on model tuning. The series were too heterogeneous to treat uniformly:
- Some were decelerating
- Some were accelerating
- Some were near-zero
- Some were noisy and ambiguous
Applying the same model to all would produce structurally wrong forecasts. The same model optimized for an accelerating trajectory will systematically overforecast a decelerating one.
The Classification Layer
Before fitting anything, I built a classification layer:
- Compute an elasticity score across two rolling windows
- Classify each entity into a trajectory regime
- Route to a different forecasting strategy per regime
Decelerating
Pick the most conservative model
Accelerating
Pick the median prediction for robustness
Ambiguous
Let recent holdout validation error decide
Key Takeaway
Design insight: Most forecasting pipelines invest heavily in model selection but apply it uniformly. Before asking which model is best, ask what state your series is in. Build a routing system, not just a better model.
Comments
FAQ
What is the key takeaway from "The hardest forecasting decision is not which model to use!"?
Most forecasting pipelines invest heavily in model selection but apply it uniformly. Before asking which model is best, ask what state your series is in. Build a routing system, not just a better model.
Who wrote this and what is it about?
This was written by Mahmoud Trigui, Senior Data Scientist. In multi-entity forecasting, routing each entity to the right strategy based on its trajectory regime matters more than model selection.