no answers on "Recovery on a RAID6/Gentoo/mdadm? Active drive is marked as spare."?

Hi again.

I still kept my old HDDs in hope someone could help me with the
problem I was writing about. Is there no hope for the lost data?

Following a tip in another forum, I tried a mdadm --create --assume-
clean on the copies of the HDDs, it warned me that the Disks are in a
RAID and then created a clean empty RAID on it, so I deleted the
copies - I still have the original HDDs though.

Following is a copy of my original posting.

many thanks
Anja

-----------------------------------------------------------------------------------------------
I am new to this group but I am desperate to find some hope or maybe
even an answer to a problem I have now for 3 days. I used to have a
700 GB RAID6 on a Gentoo Server (made with 7 HDDs) and sine it was a
RAID6, I figured I can put some data on it that is quite valuable like
my personal files folder. Guess this was stupid and I will probably
rather use a mirrored drive and to daily backups with backular or
rsync in the future, but this will not help me right now:
Some days ago, I lost one of the HDDs, which was more or less ok,
since I still had 6/5 so I postponed the purchase of a new HDD until
payday. The bad thing however happened and a second drive went offline
at the beginning of last week, probably due to the summer heat.

As I noticed this, I turned off the Server and bought a new HDD, put
it in and rebooted. I added it to the raid and it started recovering.

Now things started to go wrong: The PC crashed (lockup) at about 2% of
the recovery and I had to turn it off. Upon reboot, Only 3 drives
(plus the new one showed up in /proc/mdstat and the RAID would not
start. However I checked for the other drives of the array and they
where there (ls /dev/hd*) and also had a valid superblock (mdadm -E /
dev/hd*). The superblocks told me however, that 5 drives where in
"active sync", 2 where missing.

Now I probably made a big mistake and did (after trying mdadm --
assemble) a "mdadm --add /dev/md0 /dev/hd*" on one of the drives that
where for some reason not in the RAID. It was added as "spare" to the
RAID, not as a regular drive (so the Superblocks now told me there
where 5 drives in "active sync", one "spare" and two "missing"). I am
not sure if I did use --re-add or just --add, in any case it was bad.
Shortly after this, I recognized the problem with the missing drives
as a matter of faulty ATA-Controllers, since depending on which
controller a drive was plugged in, it was recognized by /proc/mdstat
or not upon reboot. So I got a really big SATA drive, plugged it in
and did subsequently a "dd if=/dev/hd*1 of=/mnt/satadrive/hd*1" to
make backups of the partitions of the RAID drives. I used only one at
a time on a working controller (and dumped the faulty controllers).
After a day of dd'ing, I did "losetup /dev/loop* /mnt/hd*1" on all the
5 partition mirrors and tried to assemble a RAID from it (mdadm --
assemble /dev/md0 /dev/loop1 /dev/loop2/ ...). It said "assemble from
4 drives and 1 spare" and did not start of course, since I need 5
drives.

I was getting desperate and tried a tool called mddump to show me the
superblocks of the partitions. As mdadm said, 4 drives where rather ok
(showing up as "active sync" in mdadm -E however) and one was maked as
spare. I made backups of the SBs and tried to modify the one of the
spare drive. There where 5 drives listed as ok, but only 4 drives
existed, one was just not there, so I put that drives device number
(and major/minor number) on the device claiming to be a spare and
tried to assemble the RAID again. This time, I got an "IO-Error" and
of course no RAID.

So I put the backups of the superblocks back and started to think
about ways to get the lost Data back from a couple of DVDs, CDs or
Memorysticks, where little bits of the data may still have had copies
(about 1-2% of the data). Recovery on a RAID6/Gentoo/mdadm? Active
drive is marked as spare.

In article .com,
wrote:
Hi again.

I still kept my old HDDs in hope someone could help me with the
problem I was writing about. Is there no hope for the lost data?



You could try to reassemble the original array from the working
drives or from your large sata drive partitions via madm --level=faulty.
Frankly, though, this looks like a case of "restore from tape".


Or maybe try

echo "clean" /sys/block/md0/md/array_state

If /proc/mdstat looks ok you could try and mount /dev/md0.

I don't know if this will work on raid6 or in an array that's
actually damaged and not just confused, and I wouldn't try it on your
only copy of the real data...