What to do for long term storage of data? (on a DVD+R now)

Today at work, I backed up a PC hard drive that contains old scanned college
transcripts. The data on the hard drive does not exist anywhere else, and
they have been running the machine for YEARS-no backup. Crazy, just ****ing
crazy.

So, I booted to a Ghost floppy and made an image of the hard drive
(actually, 2 drives and 2 images). Then I burned them to DVD+R, I made 2
copies on DVD and kept the images on my PC. Gave a copy to the head lady in
that dept.

It occurred to me, that DVD+R isn't the best way to store data for the long
term. I mean, they will never want to lose this data-EVER. What advice
might you offer? Does anybody offer a service of making a real pressed DVD
from a +R disc? That would be cool.

THANKS!
--Dan

In article ,
dg wrote:
Today at work, I backed up a PC hard drive that contains old scanned college
transcripts. The data on the hard drive does not exist anywhere else, and
they have been running the machine for YEARS-no backup. Crazy, just ****ing
crazy.

So, I booted to a Ghost floppy and made an image of the hard drive
(actually, 2 drives and 2 images). Then I burned them to DVD+R, I made 2
copies on DVD and kept the images on my PC. Gave a copy to the head lady in
that dept.

It occurred to me, that DVD+R isn't the best way to store data for the long
term. I mean, they will never want to lose this data-EVER. What advice
might you offer? Does anybody offer a service of making a real pressed DVD
from a +R disc? That would be cool.

THANKS!
--Dan




Here's what the US National Institute of Standards has to say about
DVD/CD media lifetime:

http://www.itl.nist.gov/div895/carefordisc/

--

a d y k e s @ p a n i x . c o m

Don't blame me. I voted for Gore.

dg wrote:

Today at work, I backed up a PC hard drive that contains old scanned
college
transcripts. The data on the hard drive does not exist anywhere else, and
they have been running the machine for YEARS-no backup. Crazy, just
****ing crazy.

So, I booted to a Ghost floppy and made an image of the hard drive
(actually, 2 drives and 2 images). Then I burned them to DVD+R, I made 2
copies on DVD and kept the images on my PC. Gave a copy to the head lady
in that dept.

It occurred to me, that DVD+R isn't the best way to store data for the
long
term. I mean, they will never want to lose this data-EVER. What advice
might you offer? Does anybody offer a service of making a real pressed
DVD
from a +R disc? That would be cool.

It can be done--there's probably some outfit near you that does it. The
trouble is that the setup charge is far more than you want to pay.

Magneto-optical is generally considered to be pretty good for long-term
storage. So is DLT believe it or not.

THANKS!
--Dan

--
--John
Reply to jclarke at ae tee tee global dot net
(was jclarke at eye bee em dot net)

Previously J. Clarke wrote:
dg wrote:

Today at work, I backed up a PC hard drive that contains old scanned
college
transcripts. The data on the hard drive does not exist anywhere else, and
they have been running the machine for YEARS-no backup. Crazy, just
****ing crazy.

So, I booted to a Ghost floppy and made an image of the hard drive
(actually, 2 drives and 2 images). Then I burned them to DVD+R, I made 2
copies on DVD and kept the images on my PC. Gave a copy to the head lady
in that dept.

It occurred to me, that DVD+R isn't the best way to store data for the
long
term. I mean, they will never want to lose this data-EVER. What advice
might you offer? Does anybody offer a service of making a real pressed
DVD
from a +R disc? That would be cool.

It can be done--there's probably some outfit near you that does it. The
trouble is that the setup charge is far more than you want to pay.

Magneto-optical is generally considered to be pretty good for long-term
storage. So is DLT believe it or not.

For usual end user budget, get a 3.5" MOD (E.g. Fujitsu makes them).
The media lifetimes are in the "we don't know" class with possibly
80 years. Every disk manufacturer garantees 30 years, most 50
years. And the drive manufacturers are commited to maintain backward
compatibility with older media. Current genreation reads and writes
(!) alll 5 media generations from todays 2.3GB media down to the first
128MB media.

Also put a FAT filesystem on them, since that is likely readable for
a long time, because it is so simple. And it is readable on almost
any OS today.

For image format the consent seems to be that TIFF is the way to
go for long-term storage, because it is structured and well-documented.
However GIF should also be fine. JPG is dangerous, they mess with
the algorithms from time to time.

Arno
--
For email address: lastname AT tik DOT ee DOT ethz DOT ch
GnuPG: ID:1E25338F FP:0C30 5782 9D93 F785 E79C 0296 797F 6B50 1E25 338F
"The more corrupt the state, the more numerous the laws" - Tacitus

wrote:

On 15 Jan 2005 07:37:15 GMT, Arno Wagner wrote:

Previously J. Clarke wrote:
dg wrote:

Today at work, I backed up a PC hard drive that contains old scanned
college
transcripts. The data on the hard drive does not exist anywhere else,
and
they have been running the machine for YEARS-no backup. Crazy, just
****ing crazy.

So, I booted to a Ghost floppy and made an image of the hard drive
(actually, 2 drives and 2 images). Then I burned them to DVD+R, I made
2
copies on DVD and kept the images on my PC. Gave a copy to the head
lady in that dept.

It occurred to me, that DVD+R isn't the best way to store data for the
long
term. I mean, they will never want to lose this data-EVER. What
advice
might you offer? Does anybody offer a service of making a real pressed
DVD
from a +R disc? That would be cool.

It can be done--there's probably some outfit near you that does it. The
trouble is that the setup charge is far more than you want to pay.

Magneto-optical is generally considered to be pretty good for long-term
storage. So is DLT believe it or not.

For usual end user budget, get a 3.5" MOD (E.g. Fujitsu makes them).
The media lifetimes are in the "we don't know" class with possibly
80 years. Every disk manufacturer garantees 30 years, most 50
years. And the drive manufacturers are commited to maintain backward
compatibility with older media. Current genreation reads and writes
(!) alll 5 media generations from todays 2.3GB media down to the first
128MB media.

Also put a FAT filesystem on them, since that is likely readable for
a long time, because it is so simple. And it is readable on almost
any OS today.

For image format the consent seems to be that TIFF is the way to
go for long-term storage, because it is structured and well-documented.
However GIF should also be fine. JPG is dangerous, they mess with
the algorithms from time to time.

Arno



DVD-RAM is the now recommended system to use, and its used for Medical
record backup..

100,000 read writes and 100 year life..

Nothing else to touch it..

Believe that "100 year life" when it has been in use 100 years. Those
estimates are a based on accelerated aging studies that may or may not
catch all causes of degradation. You might want to look up the sad history
of archival inkjets--Epson was selling a printer that based on what they
believed to be exhaustive accelerated aging studies produced output that
would last 30 years or so with no color changes. That printer had been on
the market for about a month when the reports started coming in of radical
color shifts--turns out that there was some fairly common combination of
atmospheric constituents that they had not accounted for.

There is a mistaken belief that DVD-RAM is somehow fundamentally different
from CD-RW and DVD+/-RW. In fact they all use pretty much the same
chemistry and that same 100 year life is _claimed_ for all of them, so why
should we believe that DVD-RAM is in the real world going to last any
longer than DVD-RW or CD-RW? The rewriteability comes not from any
fundamental difference but from different formatting and an elaborate
sparing scheme that allows a _lot_ of bad sectors to be mapped out to good
ones.

It's more important that a pthalocyanine or other long-lived dye layer be
used than that any particular format be used.

--
--John
Reply to jclarke at ae tee tee global dot net
(was jclarke at eye bee em dot net)

In article ,
wrote:
On 15 Jan 2005 07:37:15 GMT, Arno Wagner wrote:

Previously J. Clarke wrote:
dg wrote:

Today at work, I backed up a PC hard drive that contains old scanned
college
transcripts. The data on the hard drive does not exist anywhere else, and
they have been running the machine for YEARS-no backup. Crazy, just
****ing crazy.

So, I booted to a Ghost floppy and made an image of the hard drive
(actually, 2 drives and 2 images). Then I burned them to DVD+R, I made 2
copies on DVD and kept the images on my PC. Gave a copy to the head lady
in that dept.

It occurred to me, that DVD+R isn't the best way to store data for the
long
term. I mean, they will never want to lose this data-EVER. What advice
might you offer? Does anybody offer a service of making a real pressed
DVD
from a +R disc? That would be cool.

It can be done--there's probably some outfit near you that does it. The
trouble is that the setup charge is far more than you want to pay.

Magneto-optical is generally considered to be pretty good for long-term
storage. So is DLT believe it or not.

For usual end user budget, get a 3.5" MOD (E.g. Fujitsu makes them).
The media lifetimes are in the "we don't know" class with possibly
80 years. Every disk manufacturer garantees 30 years, most 50
years. And the drive manufacturers are commited to maintain backward
compatibility with older media. Current genreation reads and writes
(!) alll 5 media generations from todays 2.3GB media down to the first
128MB media.

Also put a FAT filesystem on them, since that is likely readable for
a long time, because it is so simple. And it is readable on almost
any OS today.

For image format the consent seems to be that TIFF is the way to
go for long-term storage, because it is structured and well-documented.
However GIF should also be fine. JPG is dangerous, they mess with
the algorithms from time to time.

Arno



DVD-RAM is the now recommended system to use, and its used for Medical record
backup..

100,000 read writes and 100 year life..

Nothing else to touch it..





Got a citation for that ? I'm not pulling your chain, I need a
citation for that figure if I'm going to recommend it to someone else.
Here's what the National Inst of Standards has to say about CD/DVD
media lifetime.

http://www.itl.nist.gov/div895/carefordisc/

IMO, it's easy to fixate on any single aspect of long-term data
retention. The media, the application, and the reader/drive, and the
computer architecture+OS all have to survive in some reasonable form.

For images and text, it's sort of easy since they are operating system
independent. Busines data is a PITA, becasue the version of the
application that reads the data needs to be recovered, along with the
data. Either that or you just produce flat-file ASCII reports that
are machine-indepedndent.

IMO, common image formats will be understood, forever. So will
documents encoded in HTML, and flat ASCII text files. Nobody's going
to make a change to JPG that breaks the trillions of images that
exist, already.

As for the media and reader, you have to be prepared to copy your data
to new media every few years, this solves the obsolete reader problem,
and keeps you within the expected lifetime for any particular media
technology.

sh*t happens. you need to keep a couple generations of media, and a
couple copies in each generation.

Virtual machines and software emulators are becomming the salvation of
people that archive computer applications needed to read old data
that's in propriatery formats. There are companies that specialize in
owning every old media reader ever made and they can recover your
files for you, buy inm some cases you need to run an application to
make sense of the data.

--

a d y k e s @ p a n i x . c o m

Don't blame me. I voted for Gore.

Previously wrote:
On 15 Jan 2005 07:37:15 GMT, Arno Wagner wrote:

Previously J. Clarke wrote:
dg wrote:

[...]


DVD-RAM is the now recommended system to use

Recommended by whom?

, and its used for Medical record backup..

100,000 read writes and 100 year life..

Nothing else to touch it..

So-so. Not as good as MOD by a fair margin. 30.000.000 writes
are a bit better than 100.000. The 100 years is claimed by no
serious vendor, as nobody knows at this time.

BRW there is no read-cycle limity in most optical storage media.

Still definitely the second-best choice.

Arno





--
For email address: lastname AT tik DOT ee DOT ethz DOT ch
GnuPG: ID:1E25338F FP:0C30 5782 9D93 F785 E79C 0296 797F 6B50 1E25 338F
"The more corrupt the state, the more numerous the laws" - Tacitus

Previously Al Dykes wrote:
In article ,
wrote:
On 15 Jan 2005 07:37:15 GMT, Arno Wagner wrote:

Previously J. Clarke wrote:
dg wrote:
[...]

DVD-RAM is the now recommended system to use, and its used for Medical record
backup..

100,000 read writes and 100 year life..

Nothing else to touch it..





Got a citation for that ? I'm not pulling your chain, I need a
citation for that figure if I'm going to recommend it to someone else.

I got a non-citation on MOD (which is now 20 year old technology and
well understood): An Engineer with Phillips told me they now estimate
80-100 year life after decades of accelerated ageing, but that the
models are shaky for that long a time. They are confident to
state publicly that the data on a MOD will keep at least 50 years.

One more comment about DVD-RAM: MOD does mandatory verify of all
data written and reallocated if the read-back is not perfect.
DVD-RAM uses a disk technology similar to MOD, but has two problems
which decrease reliability significantly in comparison:

- For DVD-RAM with cartridge, the drive does not do automatic
verifies. You can only hope your software driver does. However
a software verify is significantly inferior to one done by the
drive, since the drice can e.g. read at reduced amplifier settings
on the verify and do other things to be more thorough than a
normal read.

- DVD-RAM without cartridge has mandatory verify by the drive,
but you don't have the cartridge. That means less protection
from physical damage and light.

My personal opinion is that DVD-RAM is somewhere halfway between
MOD and DVD-RW. Good, but not very good. Note that unlike
the other DVD writable formats DVD-RAM uses phase-change
and not dyes to store the data. It is similar to MOD in that
regard.

Arno
--
For email address: lastname AT tik DOT ee DOT ethz DOT ch
GnuPG: ID:1E25338F FP:0C30 5782 9D93 F785 E79C 0296 797F 6B50 1E25 338F
"The more corrupt the state, the more numerous the laws" - Tacitus

Arno Wagner wrote:

Previously Al Dykes wrote:
In article ,
wrote:
On 15 Jan 2005 07:37:15 GMT, Arno Wagner wrote:

Previously J. Clarke wrote:
dg wrote:
[...]

DVD-RAM is the now recommended system to use, and its used for Medical
record backup..

100,000 read writes and 100 year life..

Nothing else to touch it..





Got a citation for that ? I'm not pulling your chain, I need a
citation for that figure if I'm going to recommend it to someone else.

I got a non-citation on MOD (which is now 20 year old technology and
well understood): An Engineer with Phillips told me they now estimate
80-100 year life after decades of accelerated ageing, but that the
models are shaky for that long a time. They are confident to
state publicly that the data on a MOD will keep at least 50 years.

One more comment about DVD-RAM: MOD does mandatory verify of all
data written and reallocated if the read-back is not perfect.
DVD-RAM uses a disk technology similar to MOD,

Common misconception. DVD-RAM uses disk technology similar to DVD-RW.
What's different is the formatting and encoding.

but has two problems
which decrease reliability significantly in comparison:

- For DVD-RAM with cartridge, the drive does not do automatic
verifies. You can only hope your software driver does. However
a software verify is significantly inferior to one done by the
drive, since the drice can e.g. read at reduced amplifier settings
on the verify and do other things to be more thorough than a
normal read.

- DVD-RAM without cartridge has mandatory verify by the drive,
but you don't have the cartridge. That means less protection
from physical damage and light.

My personal opinion is that DVD-RAM is somewhere halfway between
MOD and DVD-RW. Good, but not very good. Note that unlike
the other DVD writable formats DVD-RAM uses phase-change
and not dyes to store the data.

CD-RW, DVD+/-RW, and DVD-RAM all use basically the same phase change
technology, changing a substance from an amorphous to a crystalline state
and back--what varies is the detailed composition of the phase-change layer
and the formatting and encoding of the disk. It's the write-once media
that use dyes.

It is similar to MOD in that
regard.

No. MOD does not use a phase change, it changes the orientation of a
magnetic field in a material heated above the Curie point--the spots with
the reversed field have a different polarization from the others, which is
what allows reading. There is no phase change.

There is a very good discussion of the chemistry of optical and
magneto-optical recording at http://www.hkbu.edu.hk/~chem/Discworld Web
version.ppt. Note that this is a PowerPoint presentation that can also be
displayed using OpenOffice Impress.

Arno

--
--John
Reply to jclarke at ae tee tee global dot net
(was jclarke at eye bee em dot net)

Previously J. Clarke wrote:
Arno Wagner wrote:

Previously Al Dykes wrote:
In article ,
wrote:
On 15 Jan 2005 07:37:15 GMT, Arno Wagner wrote:

Previously J. Clarke wrote:
dg wrote:
[...]

DVD-RAM is the now recommended system to use, and its used for Medical
record backup..

100,000 read writes and 100 year life..

Nothing else to touch it..





Got a citation for that ? I'm not pulling your chain, I need a
citation for that figure if I'm going to recommend it to someone else.

I got a non-citation on MOD (which is now 20 year old technology and
well understood): An Engineer with Phillips told me they now estimate
80-100 year life after decades of accelerated ageing, but that the
models are shaky for that long a time. They are confident to
state publicly that the data on a MOD will keep at least 50 years.

One more comment about DVD-RAM: MOD does mandatory verify of all
data written and reallocated if the read-back is not perfect.
DVD-RAM uses a disk technology similar to MOD,

Common misconception. DVD-RAM uses disk technology similar to DVD-RW.
What's different is the formatting and encoding.

Are you sure? Hmmm. Not good.

but has two problems
which decrease reliability significantly in comparison:

- For DVD-RAM with cartridge, the drive does not do automatic
verifies. You can only hope your software driver does. However
a software verify is significantly inferior to one done by the
drive, since the drice can e.g. read at reduced amplifier settings
on the verify and do other things to be more thorough than a
normal read.

- DVD-RAM without cartridge has mandatory verify by the drive,
but you don't have the cartridge. That means less protection
from physical damage and light.

My personal opinion is that DVD-RAM is somewhere halfway between
MOD and DVD-RW. Good, but not very good. Note that unlike
the other DVD writable formats DVD-RAM uses phase-change
and not dyes to store the data.

CD-RW, DVD+/-RW, and DVD-RAM all use basically the same phase change
technology, changing a substance from an amorphous to a crystalline state
and back--what varies is the detailed composition of the phase-change layer
and the formatting and encoding of the disk. It's the write-once media
that use dyes.

It is similar to MOD in that
regard.

No. MOD does not use a phase change, it changes the orientation of a
magnetic field in a material heated above the Curie point--the spots with
the reversed field have a different polarization from the others, which is
what allows reading. There is no phase change.

There is a very good discussion of the chemistry of optical and
magneto-optical recording at http://www.hkbu.edu.hk/~chem/Discworld Web
version.ppt. Note that this is a PowerPoint presentation that can also be
displayed using OpenOffice Impress.

O.K., my opinion of DVD-RAM just dropped significanlty. I will stay
with MOD.

Thank's for the info and references!

Arno
--
For email address: lastname AT tik DOT ee DOT ethz DOT ch
GnuPG: ID:1E25338F FP:0C30 5782 9D93 F785 E79C 0296 797F 6B50 1E25 338F
"The more corrupt the state, the more numerous the laws" - Tacitus