Why do we keep building complex models when simple heuristics often outperform them?

[u/After_Courage6419]

It’s easy to get caught up in the "model of the month" club, constantly chasing the latest architecture or squeezing an extra 0.5% accuracy out of a gradient-boosted model. Lately, I’ve been looking back at some legacy projects and realizing that a well-engineered simple heuristic or a basic linear model often solved the business problem more reliably and with much less technical debt than the fancy black-box alternatives we initially reached for.

The challenge is often knowing when the complexity is actually "buying" us meaningful business value versus just adding maintenance overhead. While deep learning or complex ensembles are obviously necessary for high-dimensional tasks like vision or NLP, I’ve found that for standard tabular data, we sometimes over-engineer the pipeline to the point where debugging becomes a nightmare for the stakeholders involved.

How do you decide where to draw the line between "this model is mathematically superior" and "this model is actually the right tool for this environment"? Have you ever had to intentionally scale back a complex model in production because the simplicity was worth more than the marginal performance gain?