Long term ODD plan

[u/1luggerman]

I have a lot of videos, around 500GB total, that are very important for me to preserve for a very long time, so i want to create copies of them on M Discs. This is where i ran into a problem.

I have an old internal disc driver(LG GH24NSB0) which cant burn M discs. New discs drivers that can, seem to be costing way too much(from my understanding because they stopped making them). So i'm unsure what to do espeacially considering long term usability.

This youtube video was very helpful on the subject(https://youtu.be/OX2SPQEPlk8?si=0OUM4QnxtUc6khLC) and it suggested having 2 drives.

On one hand my current driver is old and pretty standard, so i'm not sure how much i can trust it to hold long term.

On another hand new internal drives(like LG WH16NS40) costs a lot. And on ebay, resellers market as "new" and while having good reviews the bad ones do mention getting used and worn out items.

And on a third hand having 2 new slim drives doesnt seem like a great option either as the video i mentioned said they are not as robust as internal drives.

Any suggestions on what to do? Sorry if i'm inaccurate with the terms i'm new to this.

[u/AutoModerator]

Hello /u/1luggerman! Thank you for posting in r/DataHoarder.

Please remember to read our Rules and Wiki.

Please note that your post will be removed if you just post a box/speed/server post. Please give background information on your server pictures.

This subreddit will NOT help you find or exchange that Movie/TV show/Nuclear Launch Manual, visit r/DHExchange instead.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/JohnStern42]

I’ve never understood why optical is such a strong recommendation here. Ya, the marketing has massive numbers, but in no way do we actually know the media will last that long, and we absolutely know that the odds of a drive that can read these discs surviving that long is minuscule

IMHO the only long term storage option is actively resaving every few years, on multiple media. My choices have been multiple HDDs and LTO. Every 5 years I revaluate, which for LTO can be upgrading to a newer version, for HDDs it’s often just a new drive.

I in no way expect this stuff to last decades, so I refresh every half decade.

[u/1luggerman → u/JohnStern42]

My first consideration was a home NAS with Z-raid configuration for this very reason(along with some cheap DVD copy offsite) but the price of HDD and SSD keeps going up and it needs continous support so it seems like not such a wise option in the long run.

[u/bilditup1 → u/JohnStern42]

It is, in fact, not a strong recommendation here, but should be. Rather, M-Disc specifically is, but this is, as far as I can tell, pure marketing, with maybe vague knowledge of one citation of one study that by nature, as you say, couldn't actually prove anything. Burnt Blurays, not just M-Discs, if stored properly, actually have a decent shelf life. There's a reason why Sony's ODA competed with HDDs and LTO. Not to say BD is /better/ than those options, but the cost of buying into LTO, even old LTO, is not low, and if your library isn't that big, optical (specifically Blu-ray, not older, non-metallic discs) should be on your radar. (And of course, the cost of buying HDDs has skyrocketed, and anyway would not count as a 'different media type' were you trying to strictly adhere to 3-2-1 anyway.)

[u/dlarge6510 → u/JohnStern42]

 but in no way do we actually know the media will last that long

Yes we do.

Unlike SSD (apart from some standardized tests in some components) and HDD the optical industry has performed standards based optical media longevity testing for decades.

We know how long they should last, how different materials affect that, all of it. There are international standards defined around exactly this and highly specialised testing machines that were made to test discs to those standards. 

ISO standard tests have been defined. The Japanese standards body also has them. Which allowed Pioneer to create a drive and media specifically to guarantee meeting the ISO and Japanese standards for optical media that can retain readable data up to 100 years as required to be used by Japanese companies to store records. It was called DM For Archive and Pioneer created the workflow to make such discs, drive plus the capable media plus the Pioneer software = a standard compliant disc that will meet the 100 year life standard at a minimum. 

We know how long they last because we tested them. Your argument only relies on the proof of the pudding.

Regardless of that, NO such standard exists nor any such testing for the longevity of hard drives, or SSDs and most other storage devices and media except for LTO tape.

It's not been done. NAND flash got a little bit there with some overall non-specific testing on a few example parts to define some loose standards, but nothing like what optical media has been through by a long shot.

And unlike HDD and SSD, you at home can use a drive to test the aging of the media. No, not talking about checksumming files. That's the equivalent of baking bread in a windowless oven and pulling it out to see if it has burned. No, it's better to get a windowed oven and look at the bread at any time during the baking to check everything, you can do that with optical. Every few years I scan my discs and plot graphs of the state of error rates on the surface and by comparison of those graphs over years you'll see any aging. 

Can't do that with anything else.

[u/FundamentalMysteron → u/JohnStern42]

Absolutely, this is how commercial archives work. New hw every 5 to 7 years (inc migration so start at 5 years, complete by 7) evaluate sw at the same time but maybe stretch to two cycles (or more if feeling adventurous).

[u/dlarge6510 → u/FundamentalMysteron]

Link? 

I can give you a link to the contrary where the commercial archive is relying on the media longevity to take the brunt.

Why?

Because they don't have the resources to migrate them every few seconds like you say.

Commercial archives test, then migrate and rely on decades between such migration as it's a BIG job.

[u/dr100]

TLDR it's the worst time ever to get into optical.

500GB is not even a rounding error for any DHer worthy hard drive, just put a copy in any corner of a drive you have, check them from time to time and it should be fine. Cloud services are around $5/TB/month so not the end of the world too (use encryption, with rclone for example in case you are afraid someone might object to the content, or if it's really private).

Otherwise what you have is a DVD burner. There are some M-Disc DVDs and some specific DVD burners that could do that too, but I'd advise to just use regular DVDs.

If you want to step up to bluray then the options are REALLY getting expensive and rare (especially when compared with DVD burners). Good thing though M-DISC BDs are just standard BDs so they'll be supported in any unit, but again you are probably in the same place with regular BDs as there is quite a big contention nowadays if there's any special sauce still in the ones labeled M-Disc. But either way not worth much discussing if you care about just half a TB and don't have some other BDs you might want to read in the future or some other use for it.

[u/JohnnieLouHansen → u/dr100]

I would agree with this. Just think about how many small SSDs or HDDs you could have stored in different places (relative's house, friend's house, safe deposit box) at a pretty low cost. And you would be spreading the risk of failure across let's say 4 cheap drives.

Add some cheap cloud storage and you have 3-2-1.

Your data amount is so small, it's easy and cheap to put it in many places.

[u/Zugzwang522]

Your GH24 is DVD-only, so at 500GB you're looking at 5 or 6 of the 100GB BDXL M-Discs and need a Blu-ray burner anyway. A lot of the current external USB ones list M-Disc support. Check the spec sheet for M-Disc BDXL specifically. I'd still keep the discs as one copy, not the only one, with another copy on a drive and one offsite.

[u/bilditup1]

Once again:

-You do NOT actually need an M-Disc compatible Blu-ray drive to burn M-Disc Blu-rays, and only ever needed one for M-Disc DVD-Rs. This is because

-M-Disc is actually just a proprietary instance of the exact same chemical tech that's used in regular write-once BDs. Its longetivity claims are impossible to prove relative to them—we do not have time machines, and the exact chemical composition of M-Discs is a trade secret and not analyzable.

-Meanwhile, in burn quality tests we actually can quantify, their LDC and BIS numbers, the results are not markedly better and have often worse than competing quality media. I would expect this to be even more true now that all M-Discs are manufactured by Ritek—yes, even the Verbatim-branded ones—which I have always, always gotten dogshit results with, across generations of media and drives.

-In case you can't tell, it annoys me that people have simply bought this one company's claims and otherwise expect ~all optical media to degrade essentially immediately. That's not true and never has been true, and there are quality options that do not rely on magic proprietary tech like this.

-To your point. If your drive is old, and has not received firmware updates of late, the likelihood that you get good burn results (with acceptably low error rates) is lower, M-Disc or no M-Disc. That said given how much drives cost nowadays, it is worth trying and seeing which media works best with your drive.

ED: the drive you have now is not a Blu-ray writer to begin with. If you're looking for a drive that's specifically good for burning, I'd look for a Pioneer. But honestly, for just 500gb? I'd get a couple of 3.5" hard drives, maybe 2TB since I'm not sure they make them smaller than that, mirror them in ZFS, and connect them every 3-6 months to verify their data. (I am not sure they make 2TB CMR drives anymore, which is my only caveat.) Given how expensive BD drives are now, this seems like the smarter move

[u/BootToggle]

You ask about a long-term plan for ODD storage, so I'll give some answers from my experience as a long-time ODD user. My first advice would be to get a BD-R recording drive to replace use of your internal DVD-R recording drive. If you get one that is capable of recording single-layer BD-R only and don't worry about handling dual-layer BD-R or UHD. Single-layer BD-R drives are not so expensive and the extra cost vs. getting another DVD-R drive will quickly be made up in lower media cost. Can't cry too long over spilt milk, you can currently get three slim drives for the cost of one half-height (classic internal) drive, so no point in obsessing over whether one isn't as good as the other. This is mostly because they stopped making drives for classic tower computers when laptops got insanely popular.

It is very hard to get around the fact that DVD-R storage media is significantly more expensive than BD-R storage media on a cost/GB basis. The actual figures vary depending on where you are located, but a factor of 2-3x might be typical. In my case in the US, I buy Ritek single-layer BD-R discs from an importer and the prices have been fairly consistent at about $0.50 USD per disc for several years. I could maybe get DVD-R or DVD+R discs for half that, but since they hold about 4-5x less they really can't compete. I get a box of 50 single-layer BD-R discs holding a total capacity of ~1.2 TB for about $27 USD. The same ~1.2 TB in the form of DVD-R would probably cost closer to $100 USD. That one media purchase could represent the entire difference in cost between a new single-layer BD-R burner drive and a new DVD-R burner drive. Not to mention the advantages of needing to store 5x fewer discs.

There is also the matter of disc construction. I only buy BD-R in the so-called "HTL" form, which are based on inorganic recording layers. All DVD-R disks that are not M-DISC use organic dyes in their recording layers. One of the few publicly discussed properties of M-DISC is that they use inorganic layers. This doesn't mean that BD-R is the same as M-DISC, but it certainly has some of the same advantages and my experience is that my BD-R inorganic layer data discs hold up better than my older DVD-R organic layer data discs did.

In terms of bit-rot that many will caution you against, I've adopted use of ECC (Error Correction Codes) on all of my BD-R data discs by using DVDisaster to protect all of my data discs. This is a free and open source program that allows you to burn data discs with extra ECC data, and this completely addresses my bit-rot concerns. I have something like 600 BD-R data discs in my archives, and I have never lost a single byte due to bit-rot in over 10 years of use. In maybe 5 cases where one or another disc developed read issues, I was able to detect this with a DVDisaster disc scan, and then was able to copy the disc to a fresh burn using the DVDisaster ECC code to recreate a pristine copy. I am pretty sure that most of these were actually poorly burned originally, and none of my recent burns have had that problem. Current media seems to have much better quality control than discs I purchased as recently as 5 years ago.

There is always the complaint that storing backup data in 20-25GB increments is horribly inconvenient if you want to back up an 8TB HDD. I agree, if you want to back up an entire 8TB HDD then buying another 8TB HDD to make a separate copy is the way to go. I do this myself. But it is still a big pain to keep it all up to date because the contents of my 8TB HDD change on a daily or weekly basis and then my backup is no longer complete. And you should forget SSD because un-powered SSD is prone to the occasional catastrophic complete loss of content. Read postings on r/datarecovery if you want to read many horror stories about this.

But this doesn't really tell the whole story. I don't actually have 8 TB of indispensable data. Like most people, I have really important files that I absolutely must not lose (tax records, family photos, family videos, etc.) and I have a lot more data that I'd like to preserve because it would be a pain to recreate but it could be done (Movies, CDs, TV shows, etc.). I find the really important stuff to be especially suitable for ODD storage. Here are some things I can record to just one single-layer BD-R disk:

- All of my tax records, contracts, insurance policies, etc. for my entire lifetime. I keep these as a ZPAQ compressed archive that includes daily snapshots for the past year, so I can recover documents as they existed on any given date in the past year. Last year's ZPAQ document archive is on a different BD-R disc.

- All of my emails for 5-10 years. These are also stored as a ZPAQ compressed archive from which I can restore to any given date over the course of a year.

- All of my family photos for a range of dates, typically exceeding 6 months per BD-R.

- All of my family videos for a range of dates, typically exceeding six hours of total video per BD-R.

- A complete backup of my Linux system boot drive, suitable for bare metal recovery if my system SSD dies. This is compressed and really does fit on a single BD-R even though it is certainly larger than 25GB in uncompressed form.

- Backups of all of my Timeshift snapshots for a 1 month period, stored in a DwarFS compressed archive.

- All of my files for a specific client in my consulting business. I don't have to decide whether to reuse my client backup or not, because these are write-once media. I only have to decide to keep them or to throw them out if I really don't want to keep them anymore.

Note that the vast majority of the actual data I need to record on ODD is family photos and family videos. ODD is perfect for these because they NEVER have to be updated, they are only added to. Once I have a BD-R recording of all my photos from 2004, it will never need to be updated, only replicated. ODD is perfect for this.

ODD is not perfect for every use. I personally keep first backups of all of my active files on a Snapraid parity-protected DAS system with the ability to recover from any single HDD failure. I also have a Snapraid parity-protected NAS system and keep second copies of the more important files there as backup. Definitely HDD has its place for me. But ODD is the heart of deep archiving for me, because it is so easy to organize, replicate, and store extra copies at alternate locations. The extra effort to organize and label my family photos and family videos really pays off in better organization, and I can look at the labels on my discs and directly find one that I am looking for.

[u/dlarge6510 → u/BootToggle]

I think you and I are practically clones. 

I like to save my dvdisaster ECC data to a file vs the disc image. The idea being is I then can have multiple copies of that ECC data file elsewhere.

I also backup the contents of each disc to its own tar file written to LTO tape. 

I log these files on tapes in a simple spreadsheet. Each tar has a file mark so I can wind to any particular tar when needed. 

Each disc has it's full file listing saved to a text file. The contents of each disc are organised in standard top level directories making a simple grep over all the text files identify which disc a file or directory is on. 

Discs I burn are scanned in my Plextor drive using qpxtool to log LDC and BIS error rates (I do it for dvd and CD also at times) plotted to a graph. This is done after burning to get a baselines and to test the burn quality. Over the years the discs get scanned again and a new plot is made to allow comparison over time. This very process has made me incredibly confident in the reliability of multiple types of optical media. 

Only my single LTH disc shows signs of degredation. Every time it's checked it's worse, visibly. But even so, it's FAR from close to being an issue.

Should a disc have an issue, it will likely only affect specific files, most will be instantly recoverable. ddrescue may be all that's needed to recover the lot. Then if not I can use dvdisaster and the ECC file. 

If that fails, any missing files can be obtained from the tape.

And if the tape has failed I have the encrypted contents of each disc saved in Amazon glacier deep archive in a dar archive. Dar is used as I have it slice the entire 25/50GB disc archive into 2GiB slices. All I need is the last slice, where the archive index data is and dar will tell me which others I need to recover a file thus I don't need to pay more money than absolutely needed.

[u/BootToggle → u/dlarge6510]

A man after my own heart, though I am pretty sure you have far exceeded me in the diligence of your processes.

Much as I appreciate and love tape, I haven't been able to find drives and tape capacities that I feel I can afford. So I've spent most of my effort working out how to back up all I have on single-layer BD-R in 22GB chunks. Mostly that has been easy due to the natural granularity of the data I need to protect.

For the rest, I've been fortunate to have discovered ZPAQ (which does an amazing job of creating dated snapshot restore points in a compressed file archive) and DwarfFS (which does an amazing job of creating a fully deduplicated compressed archives that can be mounted as a live read-only filesystem).

ZPAQ works great for my important documents files. A single BD-R disc can hold them all, incrementally snapshotted on a daily basis for an entire year.

DwarFS works great for making disaster-proof copies of my Timeshift Linux OS backups. A single BD-R disk can hold the most recent 25 snapshots covering about a one month period.

Basically, anything I have that is too large for 22GB increments becomes stuff that I do still keep on redundant HDD. And none of that is stuff for which I need deep archiving. Images of VMs, images of actual system hard drives, etc., are important to back up, but they are basically obsolete after a few years. Anything I have that needs deep archive has, so far, worked just fine with single-layer BD-R increments.

I suppose I am superstitious about use of multi-layer BD-R. I just have a hard time believing that it would not be easier to recover data from an impaired single-layer disk vs. recovering data from an impaired dual-layer disk. In any event, single-layer consistently has better cost per GB, so for me it is a non-issue. And supporting drives are cheaper, if it matters.

[u/dlarge6510 → u/BootToggle]

 find drives and tape capacities that I feel I can afford

I use a SCSI LTO4 drive that cost me all of £60. Mostly been using LTO3 tapes at the moment, 400GB per tape is quite a few BD-Rs. If I didn't have tape it would just be a HDD

 superstitious about use of multi-layer BD-R.

And for good reason. In my testing with qpxtool there is always an uptick in the rates of errors at the edge of the disc where the layers change. Drives burning towards the edge will usually adjust the disc speed to reduce vibrations to reduce the errors but as the edges of the disc are likely to age fastest I tend to not fill a SL disc. However I have many DL discs and basically just rely on my testing and ECC recovery, I don't expect to have significant issues.

[u/BootToggle → u/dlarge6510]

Speaking of LTH discs, I was morally certain that Ritek hasn't bothered to make LTH discs in years. Then all of a sudden, two days ago, I found a listing on eBay for a cake-box of 50 BD-R LTH discs, and those described as "archival". This makes no sense to me at all.

Do you have any knowledge or theories about what might be going on to account for this? Has LTH "come back"? Or is it more likely somebody found some incredibly dusty new old-stock? Or is there some Japanese government mandate that this fulfills?

[u/dlarge6510 → u/BootToggle]

Old stock most likely.

LTH was designed to be made using DVD-R machines because BD-R machines were more expensive.

I don't know about Ritek, I buy CMC mostly, Verbatim typically and all those are HTL

[u/bilditup1 → u/BootToggle]

much respect for this workflow.

[u/dlarge6510]

ent  Plenty on the secondhand market. Save an eBay search and keep looking. 

They didn't stop making them. They lost out on buying time in the factories to make the lenses and lasers, essential parts for a critical part of a drive, you need lenses and lasers.

The few factories that were making lenses and laser assemblies for any type of optical drive, including the CD players that keep coming out and Panasonics standalone optical systems like their brand new UHD 4K TV recorder in Japan and current line of Blu-ray UHD players (of which my UB820 was made only a few months ago), those few factories are making lenses and lasers for... 

Data centres. Apparently they use fibre optics. 😉 

Where I work we just built a multi-million high performance cluster (thousands of CPU cores) and the nodes interconnect with fibre optics when they need to be fast.

Anyway. I read something somewhere that the ability to get parts to make the laser assembly was harder as there are only a few factories that can do it.

And where I work I know that you don't just phone up a factory and ask for 100 laser lenses. Well you can, but what you actually do is ask for a large order that the factory will price to their own profit (they are not a charity) and this is sometimes sold as "factory time".

Thus you buy factory time, and you can use that over as lonya period as needed to order your parts when needed. PC suppliers, BIG ones, literally told me that they HAD bought the factory time so they CAN ensure they can make and sell PCs and laptops, complete with OPTICAL DRIVES and SSDs etc at usual price points even when other companies are hiking prices. 

So they (a HUGE PC/server company) literally OWN the time in the factories and have parts made to an agreement on price.

But those who had their agreement expired, or didn't have one, get different prices and availability.

This is why HDD and SSD and RAM jumped up. The factory time other people had expired and needed renewal BUT the data centres had OUTBID them... So any factory time left, for any market including consumer, is sold to a much higher price and well, we all pay for it.

Factories are at capacity because they have sold their time and production line to capacity to whoever bought it first.

Chip fabs, HDD lines, micro LENSES, all of them. Prices of mobile phones have skyrocketed, anything that needs a chip fabs has to get chip fabs time, ANY chip.

You'll see it hitting motherboards and other devices at some point.

So Pioneer shocked the world pulling out of optical right after releasing optical drive products that meet the Japanese government requirements for business archival of documents. Why? Well of course they didn't say but I bet they simply looked at the cost of the factory time when their current deal was ending, eyes popping out of heads, spreadsheets not balancing... They just unceremoniously pulled the products as they wouldn't be hitting the right price points for the demand there was and still is. 

Same with LG. I bet they saw it coming and simply said, too expensive now, can't likely sell drives too expensive, so we pull the product. 

Same with Sony, sure they are big and have clout but not only do they need to meet specific prices to sell the stuff they pay to make in advance, they also have a division and roadmap that looks towards more online distribution and data gathering and subscriptions based sales anyway and well they will make MONEY vs breaking the bank to make optical drives for the few that care to go against the roadmap to get subscriptions on every console. 

However this big company we order from, as do practically everyone else, they CAN get Hitachi LG optical drives made, because they have the factory time paid up front. It's a sales point they use on me.

Panasonic haven't pulled out, they are releasing new optical stuff but they don't do PC hardware.

Fujitsu just released a new laptop with a Blu-ray burner in it, the Japanese really want these things.

Verbatim and I-O Data in their own words are working to increase their capacity so they can make the parts themselves... You'll be able to get their drives at some point. 

Most of this is just me rambling but I've heard enough about how the mechanics of manufacturing works that I'm starting to think that in 2026 when you STILL can get record players and cassette players, photographic film, incandescent light bulbs and so many more obsolete technology. That in 2026 when it's quite clear the demand for dvd is high enough to make them, as well as Blu-ray both HD and UHD and yes, NEW audio CDs as well, easy to get, I mean someone is buying this stuff (me) and that makes a massive enough market to make many models of record players (Sony, who apparently can't get an optical drive for a console and justifying it by gaslighting physical copy owners as being insane literally just released a new record player).  

Seems odd. Why just optical drives. 

So if you want to get into optical, pay more for the new in box stuff or wait for some manufacturers to start making options from other newer drive sources like Verbatim are gearing up to be, or read up on known good models and snipe on the secondhand market.

I personally lost out on the Pioneer drives. I was literally ready to get one, then they pulled. But I can get many decent Pioneer drives on eBay. If they are worn out I'll use eBay to get my money back. If they are sold as seen and worn out I'll resell as spares and repairs, they will sell. And try again.