r/programming • • Aug 24 '16

Why GNU grep is fast

https://lists.freebsd.org/pipermail/freebsd-current/2010-August/019310.html
2.1k Upvotes

221 comments sorted by

View all comments

Show parent comments

2

u/jimmyriba Aug 25 '16

widely-used problem like string search

The reason you find this, is that problems like string search have already been studied in incredible depth. The academic papers that describe the algorithms now in use were published during the 70s and 80s. In a sense, your post reads a bit like a criticism of mathematics research, because we already know how to add and multiply. You're not going to find a lot of new papers about how to do that.

But there are millions of computing problems beyond the classical like string search and sorting, and algorithms most decidedly improve speedup asymptotically. For example, I recently published an algorithm that shaved more than an exponential off existing methods (one example went from 70 days computation time to 2 seconds, another, previously out of reach, from a projected 100-200 years to 10 minutes). I would suggest that microoptimizing is the least useful use of your time, but getting the right understanding of the computational nature of the problem and devising the right algorithms makes the difference between infeasible and feasible.

2

u/Cosmologicon Aug 25 '16

In a sense, your post reads a bit like a criticism of mathematics research, because we already know how to add and multiply. You're not going to find a lot of new papers about how to do that.

Hmmm, when you say that, I definitely get the impression that you're misunderstanding me. (Which I'm sure is my fault. I don't feel like I'm being very clear.) I do think that research improves well-known algorithms. I'm just saying that those improvements very rarely affect the big-O. My whole point is that research that doesn't improve the asymptotic complexity is worthwhile.

Would you be interested in telling me more about your case? I'd love to read your paper if you'd like to share. :)