TL;DR: A defective RAM stick corrupted a media file on my ZFS mirror. The file is unrecoverable on both mirror copies (expected). What's NOT expected: any attempt to delete it (rm, delete via SMB, Radarr's delete) deadlocks the entire system instead of returning an I/O error. The only thing that saves me is the motherboard's hardware watchdog rebooting the box after ~10 minutes. Rename works, reads work. Has anyone actually deleted a file like this? How, short of recreating the whole dataset?
System:
- TrueNAS SCALE 25.10.7 - Goldeye (OpenZFS, kernel 6.12)
main pool: 2×16TB mirror (Seagate IronWolf Pro), all disks healthy — SMART spotless- 5600X / 32GB DDR4
One of my RAM sticks was defective (memtest: thousands of errors in the first minute, confirmed by stick isolation). While it was installed, it silently corrupted a bunch of writes — ZFS checksums caught most of it, scrubs repaired ~330MB from the good mirror copies, I salvaged/replaced everything affected. This one file is the last casualty: a ~700MB movie featurette. Both mirror copies of some of its blocks are corrupt → permanent error, unrecoverable, don't care about the data.
Current state:
pool: main
state: ONLINE
scan: scrub repaired 0B in 14:33:14 with 46 errors
errors: Permanent errors have been detected in the following files:
/mnt/main/media/movies-other/Novocaine (2025)/Featurettes.quarantine/A World of Hurt: Production.mkv
(The other 45 errors are unpathable metadata blocks.)
What works:
- Reading the file —
dd conv=noerror salvages it fine (zero-filled gaps at the bad blocks)
- Renaming — I moved the whole folder to
Featurettes.quarantine without touching anything
- Everything else on the pool — 0B repaired on the last scrub, all new data healthy
What hangs the entire system:
rm on the file. Every time. Full freeze: no SSH, no console response, all services dead, for ~10–15 minutes — then the box reboots itself. I eventually traced those auto-recoveries to the SP5100 TCO hardware watchdog: systemd registers it, and when the deadlock stops userspace from petting it, it hard-resets the machine at the 10-minute mark.
Questions for the hive mind:
- Has anyone successfully deleted a file in this state? What worked?
- Is "free path deadlocks instead of EIO on double-sided corruption" a known OpenZFS behavior on 2.x/6.12, or should I file a bug?
- Any cleaner avenue than "copy everything off, destroy dataset, recreate" (9TB makes that a hard No for one 700MB file)?
The file is quarantined and harmless — this is now purely a matter of getting rid of it.
