10 years of DS thinking,
one post at a time

A churn model can pass every technical check after launch and still fail quietly — a new self-service flow changed what a key feature meant while the pipeline stayed healthy and the dashboard stayed green.

Result: A successful deployment is not a running pipeline. It is a system with an owned feedback loop: feature-shift checks, delayed outcomes by segment, a named model owner, and a fallback rule.

Random K-fold lets training observe periods that occur after validation, producing overly optimistic scores, misleading comparisons, and false confidence before deployment.

Result: Use time-aware splits — expanding, sliding, or blocked holdout with a gap — and validate on the future. If a forecasting score looks too good, inspect the split first.

With thousands of categories, encoding is a modelling decision. One-hot explodes into a wide sparse matrix, naive target encoding leaks, and embeddings only help when categories repeat enough to generalise.

Result: Fit encoders inside training folds, define an unseen-category fallback, and keep only the representation that generalises against a strong baseline.

Better forecasting tools speed up the workflow, but they don't replace the fundamentals: realistic validation, domain knowledge, feature design, and honest comparison.

Result: The baseline should make complexity earn its place. If a simple, well-validated baseline beats a complex approach on realistic future periods, keep it simple.

Most organisations do not really want "document AI" — they want document in, reliable business action out. I use four patterns — direct extraction, classify-and-route, context-augmented extraction, and human review — and the model matters less than the architecture around it.

Result: Start with the simplest pattern that supports a trustworthy decision, then add routing, external context, or human review only when document variability and business risk justify the extra complexity.

A lot of preprocessing bugs do not come from sophisticated modelling mistakes. They come from ordinary transformation logic being repeated manually in different places — one notebook imputes, another encodes, another scales, and a slightly different version reaches validation or production.

Result: Defining preprocessing once with a scikit-learn Pipeline and ColumnTransformer gives one transformation graph, reused at training and inference. It is not magic leakage prevention — it is a guardrail against inconsistency, so sound validation design is easier to apply consistently.

In a multi-series forecasting project, not every series behaves the same way at the same moment. Some accelerate, some flatten, some have almost no signal, and some are too noisy for a more complex model to help. Sending every series through one strategy hides those differences.

Result: A lightweight routing layer measures recent trajectory at the forecast origin, classifies a current regime, and routes each series to candidate strategies suited to it. Improving the routing logic created more lift than another round of model tuning — model selection can be conditional on current series behaviour.

SMOTE is one option for class imbalance, not the default. I start with an evaluation plan instead: a strong baseline on the original distribution, class weighting, a tuned decision threshold, and a check on whether the problem is labels or framing — before any synthetic data is created.

Result: A strong baseline and realistic evaluation come first. Resample training folds only, never validation or test, and test SMOTE only when minority examples are sparse and local interpolation is plausible.

AI can produce a first-pass EDA in minutes — distributions, missing-value patterns, target balance, outliers, correlations, and time trends. That breadth is real reconnaissance, but reconnaissance is not analysis: a generic histogram can be correct and completely irrelevant.

Result: I generate a broad first pass with AI, delete most of it, keep the few patterns worth investigating, and build custom, hypothesis-driven analysis around them. AI provides breadth; analytical judgment creates depth — the goal is fewer charts with stronger questions behind them.

Target encoding is powerful, and it is easy to evaluate incorrectly. If a category's average target is computed on the full dataset, a validation row's encoded value has already been influenced by labels that will not exist at scoring time — its own label, and its held-out neighbours'. Offline evaluation then looks better than it has any right to.

Result: CatBoost's ordered target statistics control that specific risk during training: each row's category statistic comes only from rows earlier in a random permutation, never from its own label. The encoding is not the problem — information scope is. Realistic splits, prediction-time feature availability and careful external aggregations still decide whether the number is real.

A point forecast says what the model expects; it does not say how much risk the plan is taking. Conformal prediction wraps any forecasting model in a prediction interval built from errors on held-out calibration data, so a 90% interval targets 90% long-run coverage instead of an assumed error distribution.

Result: I report a point forecast with a calibrated interval — 1,200 units with a 980 to 1,420 range — and state coverage as a long-run marginal property, not a promise for any single forecast. Basic split conformal gives every prediction the same width, so peak versus off-peak demand needs conformalized quantile regression or locally scaled scores. Coverage gets backtested by horizon, segment, and future period before capacity or inventory plans depend on it.

Most production failures in ML and LLM systems are not model failures. They are boundary failures: an API accepts an impossible value, a config typo silently changes behaviour, an LLM returns structured output missing a field the application reads unchecked, one stage sends data the next stage cannot interpret.

Result: I use Pydantic at the four boundaries where an assumption crosses components — API input, LLM output, pipeline configuration, and stage-to-stage handoffs. Defining the contract and validating on entry moves failures earlier and closer to their cause, and makes assumptions reviewable instead of invisible.

A time-aware split removes the obvious leak — future rows inside training — but does nothing about a training row whose label reaches into the validation window. A forward 10-day return dated just before the boundary is realised across the very period the model is being judged on, and the metrics come out quietly optimistic.

Result: I diagnose the label instead of applying a ritual gap. Purging overlapping training rows and adding an embargo fixed the datasets whose labels spanned a window. For point-in-time labels with causal features the plain split was already correct, and an arbitrary gap only discarded training data the model needed.

Before serious model code, an AI assistant earns its keep as a reviewer, not an author: challenge the problem framing, inventory the feature categories, design the validation environment, and use it as a second set of eyes on leakage risk — before the pipeline exists.

Result: The assistant proposes and I verify. Availability, timing, data lineage, and business meaning stay human, but the five assumptions are written down and challengeable on day one instead of surfacing three weeks later. Suggested is not verified, and that is the whole discipline.

DLinear reaches competitive time series accuracy with almost no architecture: decompose the series into a trend and a residual component, project each one linearly into the forecast horizon, then add the two forecasts together. No attention blocks, no transformer stack.

Result: More architecture does not automatically create more forecast value. Start with a statistical benchmark, add a decomposed linear baseline, and keep the deeper model only when a fair point-in-time backtest shows a clear, stable advantage that justifies its latency, tuning, and maintenance cost.

In Document AI, an invoice, a contract, an identity document, and a free-form letter are different problems. A production system does not just extract fields - it classifies the document, routes it to the right extraction pipeline, then extracts.

Result: The routing layer decides which schema, model tier, and validation rules apply - and whether uncertainty is high enough to send the document to a human review queue.

In restaurant recommendation, delivery distance was a physical boundary, not a feature: a restaurant outside the delivery area was unavailable, not less preferred.

Result: The system was structured in two stages - define the feasible set of eligible vendors first, then rank options inside that set. Constraints belong in the architecture; preferences belong in the ranking model.

In noisy short-text NLP, the model can lose before training starts. Spelling mistakes, abbreviations, slang, and partial sentences fragment the vocabulary for sparse methods such as TF-IDF.

Result: Preprocessing became a vocabulary-design decision: normalise what is safely equivalent, review unknown tokens, and preserve terms that carry domain meaning instead of applying blind dictionary correction.

Do not treat a categorical variable as one flat label when it actually contains several behavioural mechanisms. In a tabular classification project, job_type looked simple - but an occupation category may partly reflect how income is generated, how regular it is, its seasonality, and access to formal financial services.

Result: I translated the field into domain-informed representations - income source type, income regularity, and financial-access context - and validated them, since a raw category is often not the business concept itself.

Some time series should be routed before forecasting - not learned by the forecasting model. In a weekly forecasting pipeline, some entities had long runs of zero activity, and the question was not whether the model could learn the pattern but whether it was still an active forecasting problem.

Result: When an entity met a carefully defined inactivity rule, I routed it outside the active forecasting path to a monitored deterministic zero forecast, and kept active or uncertain entities in the normal forecasting workflow.

A configuration that wins once is not necessarily a good configuration. In a forecasting-search workflow, some candidate configurations look excellent in one run - until the seed changes and the result changes too. That is a problem when the search process contains randomness: stochastic fitting, random sampling, evolutionary or genetic search, random feature or lag exploration, or resampled validation.

Result: Evaluate promising configurations across multiple seeds, then select based on a robust aggregate such as the median result. The flashy single-run winner can disappear, but configurations that stay strong across repeated runs become easier to trust.

One interesting use of LLMs in tabular machine learning is not prediction — it is semantic feature generation. Ask the LLM to assign a controlled behavioural archetype, and treat the label as a candidate feature.

Result: The LLM output is not automatically valuable. Evaluate it against the baseline — out-of-sample lift, stability, prediction-time availability, privacy and cost — watch for leakage in the generation process, and keep the feature only if it earns its place.

Hierarchical clustering gives useful nested structure but its memory and computational requirements grow quickly with the population. K-means scales easily but alone may not give the hierarchical interpretation the final segmentation needs.

Result: Use K-means with many centres to compress a large customer base into representative centroids, then run hierarchical clustering on those centroids to explore higher-level structure — and validate segment sizes, within-segment variation, stability, and centroid weights before trusting the result.

Before clustering telecom customers, the feature space must be usable — recharge tails, call-duration extremes, sparse and zero-heavy variables, and missing values that mean "no activity." Standardisation aligns units but not shapes.

Result: Decide missing-unknown versus missing-zero, review extremes, transform where justified, then standardise only after semantics are clear. In distance-based unsupervised learning, feature shape is part of the model definition.

A clustering algorithm can produce groups that are mathematically distinct — but business teams cannot act on Cluster 1, 2, 3. They need to know who the group is, how it behaves, and what treatment fits.

Result: The practical method is an iterative loop: represent, cluster, profile, rethink, rebuild — checking recharge patterns, voice/data mix, network behaviour and stability until each segment is actionable. Iteration is not extra work; it is the segmentation method.

An AI assistant generated a sensible validation suite — and everything passed while the data was still wrong. The missing rule was not technical; it was semantic: a customer was churned, yet a later activation existed without an approved reactivation transition.

Result: AI is right for the scaffold — repetitive syntax, standard null/type/range checks, suite structure. The highest-value quality rules need business context only humans can supply. AI can generate checks; humans define quality.

A discovered cluster is analytically interesting, but not operational until future customers can be assigned to it. The next step after unsupervised segmentation is to translate the pattern into explainable assignment logic.

Result: Feature profiling and interpretable conditions give teams a definition they can review, apply at scale, monitor over time and challenge when behaviour changes. Discovery identifies a pattern; operationalisation makes it usable.

The hardest segmentation decision was not the clustering method — it was deciding what one observation means. One row per subscriber gives stable customer profiles; one row per subscriber-month captures behavioural states over time.

Result: Before asking which method, how many segments, or which distance metric, ask this first: do we need stable customer types or changing customer states? The observation unit decides what the clustering can learn.

No single source tells the whole story. In a Multi-SIM analytics project, the hardest question was what combination of signals could distinguish low activity from behaviour distributed across more than one SIM or operator — a usage table alone was ambiguous.

Result: Feature engineering often means assembling partial evidence: combine usage behaviour, communication-network context, handset and SIM signals, and coarse mobility patterns into one behavioural view — partial signals become stronger when interpreted together, carefully and under governance.

Some ML projects begin before a target exists. In a telecom Multi-SIM project there was no column saying "this subscriber uses more than one SIM" — so the work started with a behavioural hypothesis, not a supervised workflow.

Result: Behavioural hypothesis → feature design across data domains → unsupervised discovery → profiling and validation → operational assignment. Clustering surfaces a pattern worth investigating, not confirmed truth; the output is an explainable, governable segment — the target is sometimes a discovery process, not a column.

One of the best uses of AI in data science is not generation — it is review. Asked an AI reviewer to check a feature pipeline for temporal leakage, target leakage, train/test contamination, and features unavailable at prediction time — it flagged three patterns worth investigating.

Result: AI is a fast second reviewer, not an approver. It flags suspicious patterns and suggests safer alternatives; you still verify timestamps, validation-fold design, data contracts, and business meaning. The highest-leverage prompt is not "Write this for me" but "Challenge this before it goes live."

Production recommenders are not one model making one prediction: candidate generation first, re-ranking second. If retrieval misses a relevant item, no re-ranker can recover it — candidate generation sets the quality ceiling of the whole system.

Result: Use fast retrieval for broad recall (collaborative filtering, co-visitation, embeddings, business rules), then score only the shortlist with a richer model — learning-to-rank, cross-encoder, or an LLM re-ranker where semantics matter. Evaluate recall@K, ranking quality, latency, cost, diversity, and business constraints separately.

Time-series foundation models produce a credible forecast with minimal custom setup — that raises the standard for any pipeline that adds complexity. But forecasting performance often depends on information outside the target history: promotions, pricing, holidays, weather, and operational events.

Result: Start with a simple statistical benchmark, add a zero-shot foundation-model benchmark, then build an engineered model with valid external signals. Compare all three with rolling-origin backtesting on accuracy by horizon and segment, latency, cost, interpretability, and operational complexity — and ship the best trade-off.

A zero-shot forecasting foundation model returns a forecast with prediction intervals from a single historical series — no manually engineered lags, no hyperparameter search, no training loop. It does not automatically become the best production solution, but it changes what a strong baseline can look like.

Result: Establish a simple statistical baseline, add a zero-shot benchmark, build the engineered pipeline, and compare every candidate fairly with rolling-origin backtesting — then ship the approach with the best trade-off between accuracy, cost, latency, explainability, and operational complexity.

pandas executes every operation immediately; Polars lazy mode describes the transformation first and executes only when you call collect(). Lazy execution separates transformation intent from execution strategy — building repeatable data and feature-engineering pipelines around the plan matters more than raw speed.

Result: Describe what transformation you need and let the engine build the execution plan. Benchmark on your own data; pandas stays practical for quick exploration.

One-hot for months tells the model that January and December are unrelated — but they are neighbours. Encode truly cyclical features — month, hour, weekday — as sin and cos on a unit circle instead.

Result: December and January become adjacent in feature space. Always use it for neural networks and linear models; for tree models, test both and pick the better CV.

If daily data is noisy, a daily forecast may not be the best starting point. Aggregating daily demand to a stable weekly level, forecasting there, then distributing the forecast back to daily reduces the noise the model has to chase.

Result: Forecast at the resolution of the signal, not the data. Daily forecasts must sum back to the weekly total — and backtest aggregation against the direct forecast on the same validation windows.

In many ML teams, the real bottleneck is not building the first model. It is maintaining models that are already understood. When the task, data contract, and risk boundary are well understood, much of that work can be automated safely.

Result: Automate repeatable maintenance with clear quality gates. Free data scientists for work that still requires human judgment — framing problems, investigating change, and choosing the right approach.

A SHAP summary plot shows which features tend to matter across a dataset. But the business question is often local: why did the model flag this customer?

Global SHAP (mean |SHAP|) helps answer what tends to matter overall. Local SHAP (waterfall) helps answer what drove this specific prediction.

Result: Interpretability becomes much more valuable when it shifts from feature ranking to decision explanation.

Every categorical encoding makes an assumption about the structure of a variable. A practical guide to choosing between one-hot, ordinal, target, WoE, and contrast coding based on what your data actually looks like.

Result: Ten minutes of encoding analysis per variable can save days of debugging a model that memorizes instead of generalizes.

Hierarchical clustering can reveal useful, nested structure in customer data — but applied directly to millions of customers, it becomes computationally expensive very quickly. K-Means scales much better, but using it as the final answer too early flattens meaningful structure.

A practical two-stage pattern: K-Means compresses the population into representative centroids, then hierarchical clustering explores structure across those centroids.

Result: An algorithm does not always have to be the final model. It can prepare a feasible representation for the next layer.

After years of using both in competitions and production, there is no universal winner. On a well-prepared tabular dataset, the gap between them is often smaller than the gap created by better features, a better validation split, or removing leakage.

Start with "what is expensive, risky, or awkward in this pipeline?" not "which library is best?" The data representation usually decides the project.

Result: LightGBM for large feature-heavy datasets, XGBoost for explicit regularization and custom objectives. Spend most of the time on features, labels, and leakage checks.

In 2018 I ranked 1st in an IBM Watson AI Lab competition building an NLP classification workflow. Today, a first prototype for a similar problem starts much faster: a system prompt, a structured output schema, one LLM call.

The interface changed — the engineering discipline did not. Clear instructions, useful examples, and well-defined contracts still decide quality, exactly as they did in 2018.

Result: Same engineering principles, expressed through better tools. A vague task definition still creates unreliable outputs.

Your model gets AUC 0.92 in cross-validation and drops to 0.71 in production. Sometimes the model is not the problem — train and production data simply do not look like the same world.

Adversarial validation trains a classifier to tell training data apart from production data. A high AUC means the two worlds have drifted apart — and that is detectable before you deploy.

Result: 10 lines of code that save you from deploying a model that is already outdated.

Missingness is often not a data quality problem — it is a behavioral signal. In telecom, missing recharge means no recharge happened. In credit, missing income may mean refusal to disclose. Those are not "unknown" — they carry information.

The order of operations matters: create missingness indicators first, then null_count meta-features, then impute. Imputing first destroys the signal permanently.

Result: In 3 of my last 5 projects, the null flag ranked above the imputed variable in feature importance. The model cannot learn a signal you erased before it saw it.

In a Nairobi ambulance deployment competition, the model predicted crash probability per road segment. But the real decision was: where should 6 ambulances be positioned right now?

The important part was the translation layer between prediction and action: predict risk → select highest-risk segments → cluster into geographic groups → place ambulances at centroids.

Result: In real-world ML, prediction quality and decision quality are related, but they are not the same thing.

Some categorical variables are informative but too coarse in raw form. Job_type mixes several underlying dimensions: how income is generated, how often it arrives, and how savings are managed.

I decomposed it into multiple behavioral features: income source, savings pattern, income frequency. Each derived feature captures a different financial mechanism.

Result: The variable is no longer a flat category — it becomes a behavioral representation layer separating latent mechanisms.

In a multi-entity time series problem, I spent more time on routing than on model tuning. The series were too heterogeneous: some decelerating, some accelerating, some near-zero, some ambiguous.

Before fitting anything, I built a classification layer: compute elasticity score, classify each entity into a trajectory regime, route to a different forecasting strategy per regime.

Result: When you have many entities to forecast, don't just build a better model. Build a better routing system.

I encoded six categorical variables differently based on what they represented: Helmert Sequential for day_of_week, Polynomial Ordered for hardware generations, WoE for binary splits, Target encoding for high cardinality.

Every encoding imposes an assumption. Better tabular modeling often starts with representing variables in ways that reflect their actual structure.

Result: Encoding is not preprocessing — it is part of how the model is allowed to interpret the variable.

Started from precomputed communication communities (SAS CLA results). A communication community is not necessarily a household — it can contain extended family, multiple households, and close friends.

Combined two signals: ranked communication intensity (top contacts weighted by position) and shared dominant evening cell-tower usage. Same community + top-ranked contact + shared evening tower = household tie.

Result: In applied graph analytics, the first cluster is often not the real unit you care about.

Most data science asks "what increases Y?" — leading to correlation, regression, SHAP. But some questions are different: what must already be true for high Y to even be possible?

Used Necessary Condition Analysis (NCA) to identify variables that act as minimum requirements, not average drivers. Below a threshold, the outcome simply does not appear.

Result: Some variables are not drivers — they are constraints. "Invest in X to boost" ≠ "guarantee X above threshold before anything else matters."

In a startup survival project (~50K funding rounds), the label "did not raise again" breaks if the company hasn't been observable for the full 36-month horizon. A startup founded in 2019, observed at 2021, looks like failure but simply hasn't had enough time.

Made two design decisions: eligibility window (only train on 36+ months observable history) and asymmetric holdout (mirror deployment prevalence, not balanced 50/50).

Result: Label design is often more important than algorithm choice. Once temporal contamination enters training, the model learns recency instead of risk.

In a telecom family-detection project, I needed to distinguish household-level ties from broader social communities. Added shared dominant cell-tower usage — "which cell tower shows up most in your evening CDR?" People spend ~8 hours a night in the same place.

Infrastructure data became behavioral evidence. The richest signals often hide in operational data, not in declared information.

Result: Cell tower dominance acts as a strong proxy for home location — operational traces as features.

Built an XGBoost model to predict hourly telecom data usage. Standard confidence intervals treated all predictions the same way — but peak hours had much larger errors. The business was making decisions based on false confidence.

Wrapped the model with conformal prediction: a distribution-free uncertainty quantification method. The interval width adapts to the data — wider in peak hours, narrower in off-peak.

Result: Point predictions without uncertainty are incomplete. The interval is the signal that tells you when to trust the forecast.

Focal loss reduces the learning effort spent on examples a classifier already predicts correctly, so rare, hard-to-predict cases actually drive model improvement. One term — (1 − p_t)^γ — controls how sharply the loss focuses on difficult examples.

Result: Focal loss is one lever among several for class imbalance — the right answer is the one that improves your actual validation metric, not the one that sounds most sophisticated.

In multi-series forecasting, the same mechanics repeat: lags, rolling transforms, calendar features, time ordering, multi-step forecasts. MLforecast lets you define that recipe once and apply it consistently across every series.

Result: Good tooling moves effort away from repeated feature plumbing and toward the forecasting decisions that actually matter.

Production data is rarely one clean modality. A customer record can combine tabular behaviour, free-text tickets, and documents or images — and the real challenge is representing each one well enough to combine it safely.

I start with separate branches: keep structured data structured, turn text into measurable signals, turn documents and images into structured outputs, then align every signal to the correct entity and prediction timestamp before fusing.

Result: Represent each modality well, validate each branch, then combine only what earns its place — start simple and add modalities with evidence.

Two weaker models can sometimes outperform one stronger model — not through magic, but because their mistakes are complementary.

Focus on error diversity: generate candidates from different families, feature views, training variations, or assumptions; verify in held-out (out-of-fold) predictions before learning blend weights.

Result: Error diversity, not model count, drives the gain. Evaluate blends on held-out predictions and accept only if the gain justifies operational cost.