Feature Engineering · Missing Values · Tabular ML · Tier B

isnull() is a feature. Sometimes the best one.

Missingness is often not a data quality problem — it is a behavioral signal. In 3 of my last 5 projects, the null flag ranked above the imputed variable in feature importance.

Feature Engineering Tabular ML Python Telecom · Credit · Healthcare

View on LinkedIn →

TL;DR: In tabular ML, the absence of data is itself data. Before filling NaNs, ask: does this absence mean something operationally? If yes, encode it explicitly. The model cannot learn a signal you erased before it saw it.

Visual Summary

A missing value can mean something — slide 2

Does this absence mean something? — slide 5

The Core Idea

I treat null indicators as first-class features. Not cleanup — feature engineering.

df['feature_X_is_null'] = df['feature_X'].isnull().astype(int)

This one-liner has been more predictive than the imputed value itself in 3 of my last 5 projects.

Why Missing Values ARE Information

Telecom churn model

recharge_amount = NaN doesn't mean "unknown" — it means "this customer DIDN'T recharge." That's a churn signal.

Credit scoring model

income = NaN doesn't mean "data entry error" — it might mean "applicant refused to declare." That's a risk signal.

Medical dataset

blood_test_X = NaN doesn't mean "missing" — it means "doctor didn't order this test." The absence IS clinical information.

The Correct Pattern

The order of operations matters:

Step 1: Create missingness indicators
Step 2: Create row-level null_count meta-feature
Step 3: Create interactions around missingness (if needed)
Step 4: THEN impute values

# Preserve the missingness signal first
df["income_is_null"] = df["income"].isnull().astype(int)

# Capture the overall missingness pattern
df["null_count"] = df[cols_with_nulls].isnull().sum(axis=1)

# Then impute
df["income"] = df["income"].fillna(df["income"].median())

Common mistake: Impute first, then engineer features later. That destroys the missingness signal permanently.

Real Project Evidence

The null flag ranked above the imputed variable in feature importance
The absence of the action mattered more than its amount
The missingness pattern itself segmented behavior

Key Takeaway

Design insight: In tabular ML, the absence of data is itself data. Before filling NaNs, ask: does this absence mean something operationally? If yes, encode it explicitly. The model cannot learn a signal you erased before it saw it.

Polars Lazy Execution: More Than a Faster pandas → Cyclical Encoding: Stop One-Hot Encoding Months →

Don't Ask Your Model to Learn What a Formula Already Knows →Categorical to Behavioral Signal →Encoding Is a Modeling Decision →Categorical Encoding Cheat Sheet →SHAP Waterfall Local Explanation →Focal Loss for Imbalanced Classification →Multimodal Late Fusion →AI Can Review Feature Code. It Cannot Approve It →

← PreviousAdversarial Validation Next →Decomposition into Forecasting Strategy

FAQ

What is the key takeaway from "isnull() Is a Feature"?

In tabular ML, the absence of data is itself data. Before filling NaNs, ask: does this absence mean something operationally? If yes, encode it explicitly. The model cannot learn a signal you erased before it saw it.

Who wrote this and what is it about?

This was written by Mahmoud Trigui, Senior Data Scientist. Missing values are not always data quality issues. In tabular ML, missingness can carry the strongest predictive signal. Learn how to preserve and encode null indicators before imputation for better feature engineering.