MyISAM engine: worst case scenario in case of crash (mysql, O/S, hardware, whatever)

Jerry Stuckle wrote:
toby wrote:
Jerry Stuckle wrote:


Journals are written synchronously, ...

And RAID-1 and RAID-10 are fault tolerant mirrored systems. You would
have to lose multiple disks and adapters at exactly the same time to
loose the journal.


As long as there is a single point of failure (software or firmware bug
for instance)...


They will also handle hardware failures. I have never heard of any loss
of data due to hardware failures on RAID-1 or RAID-10. Can you point to
even one instance?

There are several examples of such hardware failures in the links
cited, but I'll crosspost this to comp.arch.storage - I'll eat my hat
if no-one there has seen a RAID data loss.



...
I don't know of anyone who has "a story" about these systems where data
was lost on RAID-1 or RAID-10.


It hasn't happened to me either, but it has happened to many others.


Specifics? Using RAID-1 or RAID-10?


These systems duplicate everything. They have multiple controllers.
Separate cables. Even separate power supplies in the most critical
cases. Even a power failure just powers down the device (and take the
system down).

Also, ZFS doesn't protect against a bad disk, for instance. All it does
is guarantee the data was written properly.


It does considerably better than RAID-1 here, in several ways - by
verifying writes; verifying reads; by healing immediately a data error
is found; and by (optionally) making scrubbing passes to reduce the
possibility of undetected loss (this also works for conventional RAID
of course, subject to error detection limitations).


And how does it recover from a disk crash? Or what happens if the data
goes bad after being written and read back?

You use the redundancy to repair it. RAID-1 does not do this.


Additionally, it depends on the software correctly detecting and
signaling a data error.

Which RAID-1 cannot do at all.



A failing controller can
easily overwrite the data at some later time. RAID-1 and RAID-10 could
still have that happen, but what are the chances of two separate
controllers having exactly the same failure at the same time?


The difference is that ZFS will see the problem (checksum) and
automatically salvage the data from the good side, while RAID-1 will
not discover the damage (only reads from one side of the mirror).
Obviously checksumming is the critical difference; RAID-1 is entirely
dependent on the drive correctly signalling errors (correctable or
not); it cannot independently verify data integrity and remains
vulnerable to latent data loss.


If it's a single sector. But if the entire disk crashes - i.e. an
electronics failure?

That's right, it cannot bring a dead disk back to life...


But all data is mirrored. And part of the drive's job is to signal
errors. One which doesn't do that correctly isn't much good, is it/

You're right that RAID-1 is built on the assumption that drives
perfectly report errors. ZFS isn't.

As Richard Elling writes, "We don't have to rely on a parity protected
SCSI bus, or a bug-free disk firmware (I've got the scars) to ensure
that what is on persistent storage is what we get in memory. ... by
distrusting everything in the storage data path we will build in the
reliability and redundancy into the file system."



I have in the past been involved in some very critical databases. They
all use various RAID devices. And the most critical use RAID-1 or RAID-10.


We can do even better these days.

Related links of interest:
http://blogs.sun.com/bonwick/
http://blogs.sun.com/relling/entry/zfs_from_a_ras_point
https://www.gelato.unsw.edu.au/archi...er/003008.html
http://www.lockss.org/locksswiki/fil...urosys2006.pdf [A Fresh
Look at the Reliability of Long-term Digital Storage, 2006]
http://www.ecsl.cs.sunysb.edu/tr/rpe19.pdf [Challenges of Long-Term
Digital Archiving: A Survey, 2006]
http://www.cs.wisc.edu/~vijayan/vijayan-thesis.pdf [IRON File Systems,
2006]
http://www.tcs.hut.fi/~hhk/phd/phd_Hannu_H_Kari.pdf [Latent Sector
Faults and Reliability of Disk Arrays, 1997]


So? I don't see anything in any of these articles which affects this
discussion. We're not talking about long term digital storage, for
instance.

I think that's quite relevant to many "business critical" database
systems. Databases are even evolving in response to changing
*regulatory* requirements: MySQL's ARCHIVE engine, for instance...

I'm just curious. How many critical database systems have you actually
been involved with? I've lost count. ...
These systems are critical to their business. ...

None of this is relevant to what I'm trying to convey, which is simply:
What ZFS does beyond RAID.

Why are you taking the position that they are equivalent? There are
innumerable failure modes that RAID(-1) cannot handle, which ZFS does.


BTW - NONE of them use zfs - because these are mainframe systems, not
Linux. But they all use the mainframe versions of RAID-1 or RAID-10.

I still claim - along with Sun - that you can, using more modern
software, improve on the integrity and availability guarantees of
RAID-1. This applies equally to the small systems I specify (say, a
small mirrored disk server storing POS account data) as to their
humongous storage arrays.


In any case - this is way off topic for this newsgroup. The original
question was "Can I prevent the loss of a significant portion of my data
in the case of a MySQL, OS or hardware failure, when using MyISAM?".

The answer is no.

--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.

==================