10K Clean Rows Can Beat 1M Noisy Rows: Data Quality First
The dataset had 1.2 million rows. After cleaning, deduplicating and filtering unreliable labels, only around 10,000 rows were usable — and the smaller dataset won.
TL;DR: Row count is not signal. A dataset of 1.2 million rows shrank to roughly 10,000 usable rows after cleaning, deduplicating and filtering unreliable labels — and the smaller dataset won. When someone says “we need more data,” the visible request often hides a deeper problem: labels built on weak heuristics, duplicate records inflating the apparent sample size, the same feature meaning different things across sources, missing values following an undocumented business process, or a training sample that does not resemble production. More rows do not fix those issues, and they can make them harder to see: a model trained on noisy labels learns noisy rules, a model trained on duplicates looks more confident than it should, and a model trained on inconsistent definitions can perform well in validation and disappoint in production. The order of operations that works is audit label quality, remove exact and near duplicates, understand missingness and impossible values, check that the split matches the real deployment setting, improve features with domain context, and only then ask whether more volume is worth the cost. The biggest modelling gain is sometimes not a new algorithm — it is deleting the data that teaches the model the wrong thing.
The Problem
That project changed how I react when someone says “we need more data.” Sometimes we do. But often, “more data” is just the visible request, and the real problem is lower down in the dataset itself.
Labels based on weak heuristics. Duplicate records inflating the apparent sample size. The same feature meaning different things across sources. Missing values that follow a business process nobody documented. A training sample that does not resemble production. More rows do not fix those issues — they can make them harder to see.
A model trained on noisy labels learns noisy rules. A model trained on duplicates looks more confident than it should. A model trained on inconsistent definitions can perform well in validation — and then disappoint in production.
The Approach
My order of operations is now simple, and it puts data work before volume questions rather than after them.
Audit label quality
Find out how the target was produced, which heuristics produced it, and where the rules that generated it fail.
Remove exact and near duplicates
Duplicates do not add information, they add confidence. Removing them changes what the model believes about its own confidence.
Understand missingness and impossible values
Missing values usually follow an undocumented business process. Once that process is understood, missingness becomes information instead of noise.
Check whether the split matches the real deployment setting
A random split on a dataset that does not resemble production will happily validate a model that fails in production.
Improve features with domain context
The same column across two sources is not the same feature. Domain context is what turns a raw column into a usable one.
Only then ask whether more volume is worth the cost
After the audit, the volume question becomes a real question with a real cost attached, instead of a reflex.
Outcome
This is not an argument against large datasets. It is an argument against treating row count as a proxy for signal. In my experience, the best ML work often starts with an uncomfortable question: which rows should not be in this dataset at all?
The biggest modelling gain is sometimes not a new algorithm. It is deleting the data that teaches the model the wrong thing.
Key Takeaway
Design insight: Row count is not signal. A dataset of 1.2 million rows shrank to roughly 10,000 usable rows after cleaning, deduplicating and filtering unreliable labels — and the smaller dataset won. When someone says “we need more data,” the visible request often hides a deeper problem: labels built on weak heuristics, duplicate records inflating the apparent sample size, the same feature meaning different things across sources, missing values following an undocumented business process, or a training sample that does not resemble production. More rows do not fix those issues, and they can make them harder to see: a model trained on noisy labels learns noisy rules, a model trained on duplicates looks more confident than it should, and a model trained on inconsistent definitions can perform well in validation and disappoint in production. The order of operations that works is audit label quality, remove exact and near duplicates, understand missingness and impossible values, check that the split matches the real deployment setting, improve features with domain context, and only then ask whether more volume is worth the cost. The biggest modelling gain is sometimes not a new algorithm — it is deleting the data that teaches the model the wrong thing.