DataDrive2030 Early Learning Predictors Challenge

The Problem

Early childhood development outcomes depend on programme characteristics that aren't always obvious. The task was to find which features of early learning programmes actually predict better developmental outcomes for children — and then use those features to predict outcomes for new programmes. This sits at the intersection of prediction and explanation: you need an accurate model, but you also need it to reveal which programme characteristics matter.

The dataset contained child development assessment scores alongside household characteristics, programme features, and learning environment descriptors. The challenge was that the feature space was large, highly correlated, and full of missing values. Many features measured similar constructs from slightly different angles — creating multicollinearity that could destabilize models and make feature importance rankings unreliable.

The evaluation required predicting continuous outcome scores, but the real value was identifying which programme features are actionable predictors. A black-box model that predicts well but can't tell you what to change is less useful than one that achieves good accuracy while revealing the relationships between programme design choices and child outcomes. This dual objective shaped every pipeline decision.

My Approach

I built a 15-step pipeline that prioritizes data quality before model complexity. The first seven steps focus entirely on feature preparation: loading and basic preprocessing, derived feature construction, correlation removal (eliminating one feature from every highly-correlated pair), missing value imputation, factor level consolidation (grouping rare categories to prevent overfitting), ordinal encoding that respects variable structure, and variable clustering for dimensionality reduction. Only after these seven steps does modeling begin.

This preprocessing-heavy approach reflects a belief that models can't compensate for dirty features. Highly correlated inputs create unstable coefficient estimates and unreliable feature importance rankings. Missing values handled inconsistently create artificial patterns. Rare factor levels create features that appear predictive in-sample but don't generalize. Addressing all of these before training gives every model in the ensemble a clean, structured input space.

The modeling phase deployed five distinct algorithmic families: CatBoost, LightGBM, H2O AutoML, SuperLearner (with NNLS metalearner), and tidymodels stacking. Each family has different inductive biases — CatBoost handles categoricals natively, H2O explores a broad model space automatically, SuperLearner optimizes a weighted combination, and stacking learns nonlinear combinations. By combining predictions across all five, the ensemble captures patterns that no single algorithm would find alone.

Feature importance was assessed across all models, looking for features that consistently rank highly regardless of algorithm. A feature that matters to both CatBoost and GLMNet is more likely to represent a genuine predictive relationship than one that only appears important in a single tree-based model. This cross-model consistency check helps separate real predictors from algorithmic artifacts.

Key Decisions

Key Takeaway

When the goal is both prediction and explanation, preprocessing quality determines whether your feature importance rankings are trustworthy. Correlated features, inconsistent missing value handling, and rare categories all create artifacts that look like signal in individual models. A rigorous preprocessing pipeline combined with cross-model importance consistency checks separates genuine predictors from noise — giving stakeholders actionable insights rather than algorithmic artifacts.