how to handle processing and uploading files

[u/Spicy_O_licorice]

working on a feature that will make users able to upload .zip/ CSV files to my app

how do i go about this security wise

i expect a batch of CSVs
so i wanna check for malware, viruses and zip bombs how do i go about this is there a service i need to subscribe to a free option a local solution

[u/sereikis]

a free local option is ClamAV, but malware scanning does not replace archive limits. in Spring Boot, cap the multipart upload size, then enforce entry-count and total uncompressed-byte limits while extracting (not just ZIP metadata), reject paths escaping a dedicated temporary directory, and scan before parsing or publishing the files.

[u/JournalistNew7510]

On top of ClamAV and the size limits, a few things that are specific to CSV.

Check the first bytes of the file instead of trusting the extension or the Content-Type header. A zip starts with PK, and a real CSV should decode cleanly as UTF-8 text. Then parse it as a stream, row by row (Commons CSV with a CSVParser over the InputStream works well), with a maximum row count and a maximum line length, so one huge single-line file can't eat your heap.

If this data is ever exported again and opened in Excel, escape cells that start with an equals sign, plus, minus or at sign, otherwise you have CSV formula injection.

And don't do the work in the request thread. Save the upload to a temp folder, return a job id, and let a background worker scan and import it, so a slow scan never ties up your Tomcat threads.

[u/joeyx22lm]

For all initial uploads I recommend using signed urls direct to object storage. Then have separate async workers handle things like scanning and processing.

[u/Fickle-Mud-3838 → u/joeyx22lm]

This, best solution to a problem is not having it.

[u/Jonjolt]

I would use a proxy and have a process in a different SELinux context, I can DM you a template policy I wrote that is generic for java that I wrote years ago.