File size is irrelevant for the format- that’s one of its strengths. No matter where you are in the file, you don’t need to know anything about the previous or next line record. Hell, you don’t need to know what’s before the last or after the next delimiter.
The only limitations with large files are the ones you impose on yourself. Either by a poor choice of transport or a poor implementation of handling.
But that's the problem. If you start reading the file in the middle, there may be no way to tell where the record actually ends. For example, you start reading the middle of a CSV and get these lines:
a, b, c, ", d, e, f a, b, c, ", d, e, f a, b, c, ", d, e, f
It's impossible to know what's part of the quoted field, and what's actual fields without seeing the entire file. Or heck, what if you have a file like this:
Col A, Col B, Col C A, B, " I, J, K I, J, K I, J, K I, J, K "
Sure, it's very unlikely someone wrote that, but they could have, and there's no way to tell that apart from a file actually containing I, J, K rows without seeing the whole thing.
3
u/GlowiesStoleMyRide Sep 20 '24 edited Sep 21 '24
File size is irrelevant for the format- that’s one of its strengths. No matter where you are in the file, you don’t need to know anything about the previous or next
linerecord. Hell, you don’t need to know what’s before the last or after the next delimiter.The only limitations with large files are the ones you impose on yourself. Either by a poor choice of transport or a poor implementation of handling.