Travel Time Prediction — Ward-Level Regression

The Problem

The variable we needed to predict — the rate of female-headed households below a wage threshold per ward in South Africa — is a socioeconomic indicator that depends on factors ranging from geographic accessibility to infrastructure proximity. This isn't a standard tabular problem where features and targets come from the same source. The target is a census-derived statistic measured at the ward level, and the features come from entirely separate data systems: road networks, traffic sensors, transit routes, and weather stations.

The fundamental modeling challenge is that the unit of prediction (a geographic ward) doesn't directly correspond to any single data source. Traffic sensor readings describe road segments. Transit data describes station locations. Weather data describes point measurements. Somehow, all of these need to be fused into a ward-level representation that captures the accessibility and infrastructure context that drives socioeconomic outcomes.

Beyond the data fusion challenge, the feature space after joining all sources was enormous relative to the number of wards. With hundreds of potential features and relatively few observations, dimensionality reduction wasn't optional — it was essential for any model to generalize. The question was how to reduce dimensions without losing the signal buried in correlated feature groups.

My Approach

I treated this as a multi-source spatial feature fusion problem. Each data source — road networks, VDS traffic sensors, transit routes, weather stations — required its own spatial join strategy to link it to ward-level observations. Road network attributes were aggregated by road segments within each ward. Traffic sensor data (speed, jam factor, vehicle counts) was summarized at the ward level using proximity-weighted averages. Transit route information was captured through station distance thresholds that classified wards as urban or banlieu.

The key spatial engineering insight was that raw proximity isn't enough — you need to capture the variation in accessibility within a ward. I computed both the mean and standard deviation of GPS trace distances within each ward, encoding geographic spread as a feature. A compact urban ward with uniform access to transit looks fundamentally different from a sprawling peri-urban ward where some residents are close to infrastructure and others are isolated, even if the average distance is similar.

For dimensionality reduction, I used ClustOfVar — hierarchical variable clustering with stability analysis — rather than standard PCA. The advantage of variable clustering is that it groups correlated features while preserving interpretable structure. Instead of abstract principal components, you get clusters of related variables where you can select a representative or take the cluster centroid. This produces a reduced feature set that a domain expert can still understand and validate.

The final model was a three-model ensemble of XGBoost, CatBoost, and LightGBM, each trained independently on the reduced feature set. A BEST_CONFIG.R file consolidated the optimal hyperparameters found across all experiments, making the pipeline fully reproducible. The ensemble averaging smoothed predictions across model families without adding overfitting risk from a meta-learner on limited data.

Key Decisions

Key Takeaway

This project reinforced that when your target is a geographic aggregate, the hardest work is building a feature representation that captures the spatial context driving the outcome. Model selection is secondary to feature fusion quality. Getting the spatial joins right — matching each data source's geometry to the prediction unit in a way that preserves meaningful signal — is where the real predictive power comes from in geospatial regression problems.