Malawi Flood Extent Prediction

The Problem

The fundamental insight was that you can't use event-specific data from one flood to predict a different flood. The training data came from a 2015 flood event. The test data was Cyclone Idai in 2019 — a different storm with a different path, different rainfall distribution, and larger geographic extent. Any model trained on 2015 rainfall patterns would learn the wrong thing. The 2015 rain fell in specific places at specific intensities. The 2019 rain fell elsewhere at different intensities. Event-specific features don't transfer.

What does transfer between flood events is the landscape itself. A depression that pooled water in 2015 will pool water again in 2019, regardless of where the rain falls. A steep hillside that drained rapidly in 2015 will drain rapidly in 2019. Soil that saturates quickly saturates quickly every time. The physical characteristics of the terrain are invariant across events — they determine flood susceptibility regardless of which storm triggers the flooding.

The target — fraction of each 1km² grid square that floods — is zero-inflated. Roughly 70% of grid squares experienced no flooding at all. A regression model minimizing RMSE on this distribution gets pulled toward predicting near-zero everywhere, which is technically a decent average prediction but useless for identifying at-risk areas. The modeling approach needed to handle this structural characteristic of the target.

My Approach

I built the entire feature set around physical landscape invariants — characteristics of the terrain that determine flood susceptibility regardless of which specific storm occurs. From the raw elevation raster, I computed six terrain derivatives: slope, aspect, Terrain Position Index (TPI — is this cell a depression or a ridge?), Terrain Ruggedness Index (TRI — how rough is the local surface?), flow direction (which of 8 cardinal directions does water drain toward?), and hillshade. These features describe the shape of the land in ways directly relevant to water accumulation.

The crucial distinction is between elevation and its derivatives. Low elevation alone doesn't predict flooding — a low-elevation flat plain pools water, but a low-elevation hillside drains rapidly. Slope, TPI, and flow direction capture this difference. They describe not just where the land is low, but where water naturally collects versus flows through. This is the physical reasoning that makes terrain derivatives more predictive than elevation itself.

For soil data, I joined FAO/ISRIC soil maps to each grid square, extracting drainage class, parent material, erosion type, and texture. These soil properties directly control the rate at which rainfall infiltrates versus pools on the surface. Sandy soils drain; clay soils saturate and create surface runoff. Combined with terrain shape, soil infiltration rates complete the physical picture of flood susceptibility.

To handle the zero-inflated target, I created auxiliary binary classifications from the training data (flooded vs. not, and severity levels) for stratified splitting and understanding where the regression error originates. All predictions were clipped to [0, 1] since gradient boosting can predict outside valid ranges for bounded targets. The final submission used an inverse-RMSE weighted ensemble of LightGBM, XGBoost, CatBoost, and H2O AutoML — weighting models by their individual accuracy on validation data.

Key Decisions

Key Takeaway

The core lesson is about generalization across events rather than across rows. When training on one event to predict another, the only features that transfer are invariants — properties of the system that don't change between events. For flood prediction, that means landscape geometry and soil properties. Everything event-specific (rainfall patterns, timing, intensity) is noise that creates an illusion of signal during training but disappears at prediction time.