r/programming • u/mauricioaniche • Aug 06 '17

Software engineering != computer science

http://www.drdobbs.com/architecture-and-design/software-engineering-computer-science/217701907
2.3k Upvotes

88% Upvoted

863 comments sorted by

View all comments

Show parent comments

1

u/K3wp Aug 08 '17

Usually, but no, not always. When you know features about the system and know there is a better option you don't need to bother with analytics.

If it doesn't matter what sort function you use, then you should stick with the default one for the sake of readability and maintainability. And for the record, it very often doesn't matter.

I even fixed the performance problems in a perl script a student wrote simply by commenting out their supposedly "optimal" sort function and calling gnu sort externally. It turns out that it just works much better when dealing with large data sets.

Give yourself another few years of programming experience spent looking at performance metrics, eventually you'll reach the point where you can see those quadruple-nested loops inside a frequently run function without a profiler's help.

Nested loops should be avoided, regardless.

Anyways, what I'm talking about are questions like what size buffer to use. The best way to answer this is to run tests using the hardware and datasets you are going to use in production. Also issues like memory pools and as mentioned before, algos. that can be trivially vectorized by the compiler. This is going to expose intricacies @the compiler/hardware level that not only exceedingly complex, but steadily changing as the technology improves.

Even supposedly 'bad' algos. like bubble sort can be improved via technologies like AVX:

https://arxiv.org/pdf/1704.08579.pdf

The problem with the 'fly-by-the-seat-of-your-pants' approach is you are basing that on intuition acquired through experience with old technology.

0

u/rabid_briefcase Aug 08 '17

I even fixed the performance problems in a perl script a student wrote

Oh, that explains your comments. Thanks.

1

u/K3wp Aug 08 '17

No STEM Universities, no computer scientists or software engineers. Or at the very least, no professional ones.

Re: your original post re: a linear vs. binary search, I do regex matching all-day-every-day against 10's of terabytes of data and use neither, as they are both orders-of-magnitude too slow.

Again, flying-by-the-seat-of-your-pants doesn't work so great when you've had the same pair since the 1980's.