COVID-19 Global Spread Prediction

Trajectory-aware pipeline forecasting cumulative COVID-19 deaths for ~180 countries across a multi-week horizon. The pipeline does not apply one strategy to all — it first classifies each country by its death trajectory, then routes it to a matched forecasting approach.

🏆 Rank 4 / 884 competitors Only 47 succeeded to submit Zindi · Apr 2020 R · ETS · ARIMA · TBATS

TL;DR: When forecasting many series at once, the first modeling decision is not "which algorithm?" — it is "are all these series actually the same kind of problem?" Classification before forecasting is often the highest-leverage step.

The Problem

When the competition opened, every team reached for the same approach: fit one model to all countries. But COVID-19 doesn't behave the same everywhere. A country with 200 deaths decelerating toward zero has nothing in common with one that just crossed 10 deaths and is doubling every three days. Treating them as the same forecasting problem felt fundamentally wrong to me.

The data came from Johns Hopkins CSSE — cumulative death counts for approximately 180 countries. We had to forecast multiple weeks ahead, and the evaluation penalized large errors heavily. One bad prediction for a single country could destroy your entire score. The data was also messy: countries reported irregularly, creating artificial spikes when they submitted catch-up corrections after missing several days.

Out of 884 registered competitors, only 47 managed to submit a valid forecast. The data complexity and the short timeline filtered out most participants. This told me the real challenge wasn't modeling — it was engineering a system that could handle 180 different forecasting contexts without manual intervention.

🏆
Rank 4 out of 884 registered competitors Only 47 teams succeeded in submitting a valid forecast — the data complexity and short timeline filtered out most participants.

My Approach

I decided early that the first modeling decision should not be "which algorithm?" but rather "are all these series actually the same kind of problem?" So I built a classification layer before any forecasting happened. For each country, I computed an elasticity score — the rate-of-change in daily deaths across two time windows. This gave me a per-country slope signal that I could use to sort countries into trajectory classes.

Four classes emerged naturally: DOWN (decelerating, past peak), UP (accelerating with enough data), NEAR_ZERO (fewer than 7 data points), and REST (everything else). Each class got its own forecasting strategy. For DOWN countries, I used damped exponential smoothing and deliberately selected the most conservative model — one that projected the fewest future deaths. The epidemiological logic is clear: a decelerating country should not suddenly spike in the forecast.

For UP countries, I used undamped models and selected the median prediction rather than the best-fitting one, which avoided both over-optimistic and over-pessimistic extremes. For NEAR_ZERO countries with barely any data, I validated against held-out days before committing to a model. And for the REST — the ambiguous middle — I ran validation-based model selection across ETS, ARIMA, and TBATS variants.

Before any model touched the data, I applied spike smoothing. When a country missed reporting for several days and then submitted a catch-up correction, standard time series models treated that spike as real signal. I built two levels of smoothing (thresholds 0.5 and 0.33) and let validation decide which version worked best for each country. Nothing was applied uniformly — every decision was made per-country.

Key Decisions

Classify Before Forecasting

Rather than applying one global model, I invested the time to build a trajectory classification system that routes each country to the strategy matching its epidemiological phase. This single architectural choice produced more improvement than any hyperparameter tuning possibly could.

Conservative Prior for Decelerating Countries

For countries past their peak, I deliberately selected the model that projected the fewest future deaths. A slightly low forecast is far less damaging than one that artificially inflates projections for a country already on its way down. This encoded domain knowledge directly into the model selection criterion.

Spike Smoothing as Preprocessing

Irregular reporting created artificial spikes that destabilized standard time series models. I applied redistribution smoothing at two thresholds and let per-country validation choose the best version. The key insight: data quality issues are not uniform, so the fix cannot be uniform either.

Multiple Transformations in the Inner Loop

Three transformation strategies — raw values, log, and diff(log) — were tested per country inside the selection loop. The right transformation depends on the trajectory class and smoothing version, so it cannot be decided globally before seeing each series.

Key Takeaway

This project taught me that heterogeneous forecasting — routing different series to different strategies — consistently outperforms applying a single "best" model globally. The time I spent on trajectory classification and routing logic produced more lift than any amount of hyperparameter tuning would have. When you have many entities to forecast simultaneously, the first question is never "which algorithm?" — it is "are these actually the same kind of problem?"

Design insight: When forecasting many series at once, the first modeling decision is not "which algorithm?" — it is "are all these series actually the same kind of problem?" Classification before forecasting is often the highest-leverage step.

FAQ

What is the key takeaway from "COVID-19 Global Spread Prediction"?

When forecasting many series at once, the first modeling decision is not "which algorithm?" — it is "are all these series actually the same kind of problem?" Classification before forecasting is often the highest-leverage step.

Who wrote this and what is it about?

This was written by Mahmoud Trigui, Senior Data Scientist. Trajectory-aware forecasting pipeline for COVID-19 cumulative deaths across 180 countries. Elasticity scoring, country classification into 4 trajectory regimes (UP, DOWN, NEAR_ZERO, REST), spike smoothing, validation-based model selection. ETS, ARIMA, TBATS in R. Ranked 4th out of 884 competitors on Zindi, April 2020.

Comments

Machine Learning Feature Engineering MLForecast Time Series Decomposition Forecasting LightGBM XGBoost Catboost Clustering Segmentation NLP LLMs Web App R Markdown SQL Oracle DB SAS-Guide SAS E-Miner Dataiku BigQuery GCP Python R CRISP-DM Hypothesis Testing ANOVA Data Analytics Dimensionality Reduction Recommendation System Network Analysis Geospace Analysis Spatial Data Embedding Sampling Techniques Decision Rules Data Storytelling CVM Churn Fraud Detection Sentiment Analysis Topic Modeling IBM Watson PowerBI Looker Studio VBA Statistical Learning Ensemble Modeling Stacking Cross-Validation Profiling ABT Construction Plumber Tidyverse Shiny Prophet Deep Learning Scikit-Learn JSON SAS Programming Git VS Code CSS Styling Automated Reporting Outlier Detection Temporal Clustering Startup Survival Pre-Valuation Modeling K-Means Decision Trees Data Science Predictive Modeling SVM LDA Text Classification Weight Prediction Pattern Recognition Real-Time Detection Community Detection Pipeline Automation Data Quality Checks Data Reliability Specification Mapping Business Strategy Marketing Campaigns Try & Buy Frameworks KPI Dashboards Network Quality Sales Analytics Mentoring Statistics Lecturer Remote Work Hybrid Work Consulting Contract Full-Time Freelance Sofrecom Orange Group Tunisia Telecom Kiota Intelligence VC Analytics Series A Prediction Pre-Valuation Modeling Production ML Applied AI Prompt Engineering Business Forecasting Decision Systems Graph Analytics Household Detection Multi-SIM Detection FTTH Forecasting Audit Extraction Infrastructure Classification Pydantic GPT-4 OpenAI API Base64 Classification Zindi Codementor LAAS-CNRS ESSAI MIT xPRO Tunisia ML Competition Cell Tower Analysis Uber Logistics Uber Cape Town Necessary Condition Analysis Behavioral Signals Spike Smoothing Observation Unit Design Dendrogram Ward Clustering VIF Target Encoding Machine Learning Feature Engineering MLForecast Time Series Decomposition Forecasting LightGBM XGBoost Catboost Clustering Segmentation NLP LLMs Web App R Markdown SQL Oracle DB SAS-Guide SAS E-Miner Dataiku BigQuery GCP Python R CRISP-DM Hypothesis Testing ANOVA Data Analytics Dimensionality Reduction Recommendation System Network Analysis Geospace Analysis Spatial Data Embedding Sampling Techniques Decision Rules Data Storytelling CVM Churn Fraud Detection Sentiment Analysis Topic Modeling IBM Watson PowerBI Looker Studio VBA Statistical Learning Ensemble Modeling Stacking Cross-Validation Profiling ABT Construction Plumber Tidyverse Shiny Prophet Deep Learning Scikit-Learn JSON SAS Programming Git VS Code CSS Styling Automated Reporting Outlier Detection Temporal Clustering Startup Survival Pre-Valuation Modeling K-Means Decision Trees Data Science Predictive Modeling SVM LDA Text Classification Weight Prediction Pattern Recognition Real-Time Detection Community Detection Pipeline Automation Data Quality Checks Data Reliability Specification Mapping Business Strategy Marketing Campaigns Try & Buy Frameworks KPI Dashboards Network Quality Sales Analytics Mentoring Statistics Lecturer Remote Work Hybrid Work Consulting Contract Full-Time Freelance Sofrecom Orange Group Tunisia Telecom Kiota Intelligence VC Analytics Series A Prediction Pre-Valuation Modeling Production ML Applied AI Prompt Engineering Business Forecasting Decision Systems Graph Analytics Household Detection Multi-SIM Detection FTTH Forecasting Audit Extraction Infrastructure Classification Pydantic GPT-4 OpenAI API Base64 Classification Zindi Codementor LAAS-CNRS ESSAI MIT xPRO Tunisia ML Competition Cell Tower Analysis Uber Logistics Uber Cape Town Necessary Condition Analysis Behavioral Signals Spike Smoothing Observation Unit Design Dendrogram Ward Clustering VIF Target Encoding