Same Practitioner, Better Tools: ML Workflow Then vs Now
My competition workflow looks very different from a few years ago. The fundamentals do not. Better tools increase leverage — but fundamentals decide whether that leverage is useful.
TL;DR: Better tools increase leverage; fundamentals decide whether that leverage is useful. A modern ML workflow — reusable feature pipelines, automated hyperparameter search, experiment tracking, better diagnostics, and AI assistance for boilerplate and review questions — makes it possible to run many more experiments than before. But it does not remove the judgement that actually determines the result. You can run more bad experiments faster, poor validation can still select the wrong model, and generated code still cannot tell you whether a feature exists at prediction time or whether a business rule makes sense. The fundamentals have not changed: understand the problem before modelling it, build a reliable validation design, check for leakage, improve features with domain context, and add model complexity only when it earns its place. Leverage without fundamentals just produces more bad experiments at a larger scale; fundamentals are what turn that leverage into a decision you can defend.
The Problem
Earlier on, my workflow was more manual: feature engineering written from scratch, cross-validation loops built by hand, a small number of model families, parameter changes tested one by one, experiment tracking in spreadsheets, and simple averaging for ensembles.
That does not mean the tools suddenly do the thinking for you. If anything, better tools make weak reasoning easier to expose. You can run more experiments, but you can also run more bad experiments faster. You can test more models, but poor validation can still select the wrong one. You can use AI to generate code, but it still cannot decide whether a feature exists at prediction time or whether a business rule makes sense.
The Approach
Today the workflow has more leverage, but the way I work through it has not changed much. The stack evolved; the sequence of decisions stayed the same.
Then: manual feature engineering and hand-built CV loops
Features written from scratch, cross-validation loops assembled by hand, a small number of model families, parameter changes tested one at a time, and experiment tracking kept in spreadsheets.
Now: reusable pipelines and automated search
Reusable feature pipelines, stronger baseline tools, automated hyperparameter search, real experiment tracking, easier model comparison, better diagnostics, and AI assistance for boilerplate, review questions, and documentation.
Understand the problem before modelling it
The prediction target, the decision it drives, and the data that will genuinely exist at prediction time all come first.
Build a reliable validation design
Split the data the way the real deployment splits it, and make sure the validation setup answers the business question instead of a convenient proxy.
Check for leakage
Automated pipelines make it easier to leak a target by accident. Leakage checks are not optional, and they are harder to eyeball once the feature code is abstracted away.
Improve features with domain context
Domain knowledge still decides whether a feature carries signal, and no amount of automated search substitutes for a feature that means something.
Add model complexity only when it earns its place
More search does not mean more value. The extra complexity has to beat a strong baseline on held-out evidence before it stays.
Outcome
The stack evolved. My workflow became faster, more reproducible, and easier to inspect. But the core job is still the same: turn imperfect data into a decision you can defend.
The measurable change is not a better leaderboard score on its own. It is that experiments are cheaper to run, cheaper to reproduce, and cheaper to audit — so the questions I spend my attention on move back to the parts that actually matter.
Key Takeaway
Design insight: Better tools increase leverage; fundamentals decide whether that leverage is useful. A modern ML workflow — reusable feature pipelines, automated hyperparameter search, experiment tracking, better diagnostics, and AI assistance for boilerplate and review questions — makes it possible to run many more experiments than before. But it does not remove the judgement that actually determines the result. You can run more bad experiments faster, poor validation can still select the wrong model, and generated code still cannot tell you whether a feature exists at prediction time or whether a business rule makes sense. The fundamentals have not changed: understand the problem before modelling it, build a reliable validation design, check for leakage, improve features with domain context, and add model complexity only when it earns its place. Leverage without fundamentals just produces more bad experiments at a larger scale; fundamentals are what turn that leverage into a decision you can defend.