Encoding is a modeling decision, not a preprocessing checkbox
Every encoding imposes an assumption. Better tabular modeling starts with representing variables in ways that reflect their actual structure.
The Problem
One thing I still see oversimplified in tabular ML is categorical encoding. Too often, the question becomes: "Which encoding is best?" But that is usually the wrong question.
A better question is: What structure does this variable actually have, and which encoding respects that?
The Approach
day_of_week → Helmert Sequential
Each day vs. average of all previous days
RUType → Polynomial Ordered
Hardware generations, spacing non-uniform
region_group → Backward difference
Adjacent categories more similar than distant ones
tech_grade → M-estimator
Noisy grouping, needs Bayesian smoothing toward global mean
site_type → WoE
Binary split indicator, preserves log-odds direction
city_band → Target encoding
High cardinality, direct mapping to conditional mean
The Assumptions
Every encoding imposes an assumption:
- One-hot assumes no ordering
- Ordinal imposes rank
- WoE imposes directional contrast
- Shrinkage encoders assume partial pooling toward a global signal
Key Takeaway
Design insight: Encoding is not just a preprocessing step. It is part of how the model is allowed to interpret the variable. Better tabular modeling often starts with representing variables in ways that reflect their actual structure.