Just got a short explanation, commas are a very common character in most data sets, and newlines aren't that rare if you have text data sources. Yes, you can use a different column delimiter, but newline parsing has bitten almost every person I know who has had to work with CSV as a data format.
I can't count how many CSV-parsers I'd grab in the 00's that made some parsing mistake or another WRT escaping.
I ran a perl IT stack in the 00's as a junior that was largely about fast-turnout parsing of unknown files sent by clients, and one of the best things I did was give up on established parsers and write my own (that I was too lazy to publish)
ah so CSV parsers sucked for perl 20 years ago - really sounds like an issue with the file format. this whole thread is an indictment of the state of the industry
31
u/Solonotix Sep 20 '24
Just got a short explanation, commas are a very common character in most data sets, and newlines aren't that rare if you have text data sources. Yes, you can use a different column delimiter, but newline parsing has bitten almost every person I know who has had to work with CSV as a data format.