When a SQL Query Beats a Model: Start With a Rule-Based Baseline
The simplest acceptable solution should get the first chance to solve the problem. A well-designed SQL baseline sets a real benchmark for complexity — and a model should only earn its operating cost through incremental decision value.
TL;DR: Rules also need ownership, versioning, and monitoring — but they are often easier to inspect, explain, and change. The point is not that rules always win; it is that models should not start with complexity by default. They earn it through incremental decision value. Start simple, add complexity only when it pays, and monitor both rules and models as conditions change.
The Problem
Teams often reach for a model before asking whether a simpler solution could solve the decision. But a model does not only add predictive power — it adds an operating system: feature-pipeline maintenance, training and evaluation workflows, versioning, monitoring and retraining, explanation and governance, and failure and fallback paths. Without an honest baseline, you cannot tell whether that complexity is buying anything worth keeping.
The Approach
I write the strongest simple, rule-based baseline first — often in SQL — and treat it as a serious benchmark rather than a formality.
Build the strongest rule baseline
For a churn-style use case, a rule can combine declining recharge activity, falling usage over a recent window, a customer's behaviour relative to their segment, and a threshold that triggers a retention workflow. It is transparent, testable, and changeable — and a well-designed SQL baseline can be surprisingly useful.
Account for the cost of a model
A model may improve the relevant held-out metric, but it also introduces feature-pipeline maintenance, training and evaluation workflows, model versioning, monitoring and retraining decisions, explanation and governance requirements, and failure and fallback paths.
Weigh incremental decision value
Sometimes the additional lift is worth it: for a high-value decision, a modest improvement may create substantial value. For a low-risk workflow, a transparent rule may be the better operational choice — even if a model scores slightly higher offline.
My decision process: (1) define the decision and its cost of error; (2) build the strongest simple rule-based baseline; (3) measure it on realistic held-out data; (4) test whether a model produces meaningful incremental value; (5) compare that value with the cost of operating the model.
Visual Summary
Outcome
Comparing the rule and the model on the same realistic held-out period — on relevant metric, decision value, reliability, and operating cost — turns a style debate into a decision. When the rule was sufficient, we shipped and monitored it. When the model added meaningful value, we were able to justify the system investment with evidence rather than intuition.
Key Takeaway
Design insight: Rules also need ownership, versioning, and monitoring — but they are often easier to inspect, explain, and change. The point is not that rules always win; it is that models should not start with complexity by default. They earn it through incremental decision value. Start simple, add complexity only when it pays, and monitor both rules and models as conditions change.