"Repair" hard disk with bad sectors by reading and rewriting samedata?

I have a 6TB drive that shows 8 bad sectors. Would it be possible to
repair it without harm to the data using

dd if=/dev/daX of=/dev/daX (and perhaps with bs settinng as well) ??

Perce

"Percival P. Cassidy" wrote:

I have a 6TB drive that shows 8 bad sectors. Would it be possible to
repair it without harm to the data using

dd if=/dev/daX of=/dev/daX (and perhaps with bs settinng as well) ??

dd lets you copy an entire partition atop of itself? Seems dangerous.
If there is an error during the operation, well, that's why dd is also
known as disk destroyer.

Presumably you don't know in file(s) encompass the bad sectors. Or do
any files use those sectors, and instead the bad sectors are in free
space?

Why not use fsck (if the volume can be unmounted, like "sudo unmount
/dev/daX", and then "sudo fsck /dev/daX)? If the bad sectors are in
free space, go for it. If the bad sectors are used in files, do a
logical (file) backup beforehand, or use dd to save a disk image (in
another partition or to an .img file).

I haven't used [U|Li]nux for many years. Wait until a *NIX guru shows
up, but what you propose to write a partition atop of itself looks
dangerous, and I would think dd would error on that attempt.

Since dd is just a copy tool, I don't see that it will attempt to
recover iffy sectors. The copy may be devoid of any files with bad
sectors because, after all, they couldn't be read without error.

"dd conv=noerror" will continue after an error, but it skips bad blocks.
Adding the sync option just adds zeros instead of skipping the bad
blocks. gddrescue also keeps going after an error, but it attempts
recovery on a partial yield from a bad block while trying to read each
sector in a block. However, in either case, if a block is completely
unreadable then no data gets read from it into the copy or image.

Is this such an old HDD that it doesn't include its own firmware to
remap bad sectors using S.M.A.R.T.? If you read the SMART data from the
HDD, and there is still a non-zero Current Pending Sector Count (raw
count), that count doesn't go down after a reboot? The pending count
should go down (to do the remapping using the HDD's firmware) while the
Reallocated Sectors Count goes up.

https://www.disktuna.com/reallocated...h-bad-sectors/
See "Pending Sectors" section.

Note that more remapped sectors means slower read and write speeds. The
remapping (reading a remapped sector and then having to read the
alternate sector used for remapping) takes time. If the Current Pending
Sector Count does not go down, there are no more reserve sectors on that
hard disk, so the bad sectors cannot get remapped. If that count
doesn't go down, you need to look at replacing the hard disk, and soon.
If the count goes down, the HDD's firmware did its job of remapping the
bad sectors. With an ever-increasing pending count, you need to clone
to a partition on a new or different HDD, and get all partitions off the
old HDD onto the new one.

There are many free tools to read SMART data from a drive. There might
be some good free disk health monitor tools, but to get decent testing
and recovery features often means having to pay. For example, there's a
free version of HD (Hard Disk) Sentinel. I use that but under Windows,
and I eventually went to a paid version. They have a full-feature trial
version usable for 30 days. It's been too long since I used their free
version to remember if it was effective to get bad sectors remapped (or
prods the HDD to do it). Try the free version, and it that is
insufficient then try their trial version. Just be aware that some
"refresh" operations on a disk can be destructive (make sure your files
are saved elsewhere).

A simple copy operation, like you proposed, does not get a bad sector
remapped. It just tries to copy what it can to the destination, but the
bad sector remains bad at the source.

While disk recovery tools are geared towards recovering data from bad
sectors and then prodding the hardware to remap the bad sectors, maybe
you can use "smartctl -t offline /dev/daX" to tell the disk's firmware
to do an offline surface scan to get the bad sectors remapped. However,
I don't know that attempts any data recovery from the bad sectors.

Percival P. Cassidy wrote:
I have a 6TB drive that shows 8 bad sectors. Would it be possible to
repair it without harm to the data using

dd if=/dev/daX of=/dev/daX (and perhaps with bs settinng as well) ??

Perce

This depends a lot on how you've concluded you have 8 bad sectors.

ATA (either IDE or SATA drives) have automatic sparing.

There are a certain number of spare blocks "within reach" of a
given block. Say for example, there were 32 spare sectors per track,
then if one sector was unrecoverable, the sector could be replaced
by a good sector. The replacement might happen during a write, so
fresh data goes into the sector. If any reads were attempted, the
read-tries occur for 15 seconds, so 1800 tries happen in the hope
the data can be read. Sectors which took a lot of tries to read,
are put in a queue for processing on the next write to the
area. The blocks that are remapped, pointers are kept in a table
in the cache RAM, for quick access during normal operation.

The SMART has

Reallocated Raw Data = 0

Current Pending Raw Data = 0

There was a theory, that the sectors of poor quality (needed the 1800
read-tries), those show up as logged in Current Pending. And as the
drive whittles them down and resolves them, they cause the Reallocated
to be incremented. However, my experience with Seagate at least,
is the Current Pending indicator only starts to increment, when the
drive is in serious trouble and is throwing CRC errors. Whether each
Current Pending block is a CRC error block, who knows. I don't have
enough sick drives, to be making finer interpretations than that.
All I can tell you is, Seagate behavior, does not match the Internet
description of how it works.

It's because there is Automatic Sparing, that the drives can be "repaired"
in a sense, by applying an external stimulus. Writing the drive
from end to end, allows an opportunity for evaluation of the various
tables of badness. The operation "harvests" blocks that needed
attention, and the result is, the SMART table then is a better
reflection of current conditions.

In the old days, with SCSI drives, sparing was also reset-able by the
operator. There were "factory defects" and "grown defects". A person
could manage the defect lists if they wanted. I tried doing that
once, by removing the grown defects, and scanning the drive, and
the grown defects, to a man, all came back. Which proved they
really were defects. Even if your current 6TB drive were a SCSI
drive (SAS maybe), and you messed around like that, the status
of the media is unlikely to change.

The Reallocated also have weird growth characteristics. I
correlate growth there with room humidity. The drive
has a breather hole, and while there is a labyrinth to
prevent moisture entry, if the AC is off in the facility
for a month or so, whatever moisture problem is evident
there, it could get into the drive. The Reallocated would
be seen to increase, when you rewrite the drive from
end-to-end, and new suspect sectors are processed and
replaced with spares. I have drives here, where the
Reallocated have (more-or-less) stopped growing, so
if you see a "growth rate", if you remove the
stimulus (get the AC working again), the humidity
drops and the drive might not get any worse.

A Helium-filled drive, doesn't have that exposure.
The Helium could (and will) leak out. The drive is
guaranteed to have Helium for five years, before the
adhesive in the cover lets it all escape. The adhesive
eeam is "wide", in an attempt to provide as much of
a barrier as possible. And it was considered that
the adhesive method would be better than a weld. The Helium
drive may have two covers, a Helium seal on an inside cover,
and a cover plate for mechanical protection (so if
Willy squeezes the drive in his fat fingers, it
doesn't immediately destroy the seal). But at
least with a Helium drive, there's no way for
water infiltration to be an issue with the media.

Summary: Go ahead and write it, expecting Reallocated to grow
according to the number of suspect sectors already present.
The write operation is like a "harvest". If a sector has
thrown a CRC error in the past, it's still going to do that,
as it got that way because the disk area has run out of
spares to fix it.

Verify the semantics of "dd", by using a VM, writing
a random pattern on a small data drive in the VM first
with dd. Then, do a "copy" pass as proposed. Then
read out the drive and compute a checksum and see if
the "before" and "after" checksum on the "copy" operation
are the same.

As far as I know, "dd" does read before write, but I don't
know if there are any corner conditions to be concerned
about or not. If the drive will not sustain an unadorned
dd operation attempting to read it from end to end,
then you don't want to try your copy pass. The copy pass
would be reserved for "good times", where to the best
of your knowledge, every write attempted by dd, would succeed.

The gddrescue has more recovery options, such that its
copy semantics are less likely to cause grief. It maintains
a logfile with data on what sectors kicked up a fuss.
Repeated runs can be used to "resolve problems" and
reduce the outstanding work shown in the logfile. The
package name in Linux is "gddrescue", but the executable
at runtime is "ddrescue". Block size on transfers, is variable,
and the program can gear down to "doing single sectors" if it wants.
Regular dd is more likely to do fixed values of "bs=" size.

Practice on a VM drive *first*, before unleashing these
lengthy lengthy operations on a 6TB drive. All that a VM
drive can test, is that the basic operation works OK, not
that all the corner cases are handled. A gddrescue operation
on a virtual drive, will have an empty logfile. Or at least
the symbols indicating no work remains to be done. It won't
be a particularly human-readable format, because the notation
also seeks to compress the size of the logfile and not make
it huge-beyond-words.

Paul

On 2020-08-14 02:56, Paul wrote:
Percival P. Cassidy wrote:
I have a 6TB drive that shows 8 bad sectors. Would it be possible to
repair it without harm to the data using

dd if=/dev/daX of=/dev/daX (and perhaps with bs settinng as well) ??

Perce

This depends a lot on how you've concluded you have 8 bad sectors.

ATA (either IDE or SATA drives) have automatic sparing.

There are a certain number of spare blocks "within reach" of a
given block. Say for example, there were 32 spare sectors per track,
then if one sector was unrecoverable, the sector could be replaced
by a good sector. The replacement might happen during a write, so
fresh data goes into the sector. If any reads were attempted, the
read-tries occur for 15 seconds, so 1800 tries happen in the hope
the data can be read. Sectors which took a lot of tries to read,
are put in a queue for processing on the next write to the
area. The blocks that are remapped, pointers are kept in a table
in the cache RAM, for quick access during normal operation.

The SMART has

Â*Â* ReallocatedÂ*Â*Â*Â*Â*Â* Raw Data = 0

Â*Â* Current PendingÂ*Â* Raw Data = 0

There was a theory, that the sectors of poor quality (needed the 1800
read-tries), those show up as logged in Current Pending. And as the
drive whittles them down and resolves them, they cause the Reallocated
to be incremented. However, my experience with Seagate at least,
is the Current Pending indicator only starts to increment, when the
drive is in serious trouble and is throwing CRC errors. Whether each
Current Pending block is a CRC error block, who knows. I don't have
enough sick drives, to be making finer interpretations than that.
All I can tell you is, Seagate behavior, does not match the Internet
description of how it works.

It's because there is Automatic Sparing, that the drives can be "repaired"
in a sense, by applying an external stimulus. Writing the drive
from end to end, allows an opportunity for evaluation of the various
tables of badness. The operation "harvests" blocks that needed
attention, and the result is, the SMART table then is a better
reflection of current conditions.

In the old days, with SCSI drives, sparing was also reset-able by the
operator. There were "factory defects" and "grown defects". A person
could manage the defect lists if they wanted. I tried doing that
once, by removing the grown defects, and scanning the drive, and
the grown defects, to a man, all came back. Which proved they
really were defects. Even if your current 6TB drive were a SCSI
drive (SAS maybe), and you messed around like that, the status
of the media is unlikely to change.

The Reallocated also have weird growth characteristics. I
correlate growth there with room humidity. The drive
has a breather hole, and while there is a labyrinth to
prevent moisture entry, if the AC is off in the facility
for a month or so, whatever moisture problem is evident
there, it could get into the drive. The Reallocated would
be seen to increase, when you rewrite the drive from
end-to-end, and new suspect sectors are processed and
replaced with spares. I have drives here, where the
Reallocated have (more-or-less) stopped growing, so
if you see a "growth rate", if you remove the
stimulus (get the AC working again), the humidity
drops and the drive might not get any worse.

A Helium-filled drive, doesn't have that exposure.
The Helium could (and will) leak out. The drive is
guaranteed to have Helium for five years, before the
adhesive in the cover lets it all escape. The adhesive
eeam is "wide", in an attempt to provide as much of
a barrier as possible. And it was considered that
the adhesive method would be better than a weld. The Helium
drive may have two covers, a Helium seal on an inside cover,
and a cover plate for mechanical protection (so if
Willy squeezes the drive in his fat fingers, it
doesn't immediately destroy the seal). But at
least with a Helium drive, there's no way for
water infiltration to be an issue with the media.

Summary: Go ahead and write it, expecting Reallocated to grow
Â*Â*Â*Â*Â*Â*Â*Â* according to the number of suspect sectors already present.
Â*Â*Â*Â*Â*Â*Â*Â* The write operation is like a "harvest". If a sector has
Â*Â*Â*Â*Â*Â*Â*Â* thrown a CRC error in the past, it's still going to do that,
Â*Â*Â*Â*Â*Â*Â*Â* as it got that way because the disk area has run out of
Â*Â*Â*Â*Â*Â*Â*Â* spares to fix it.

Â*Â*Â*Â*Â*Â*Â*Â* Verify the semantics of "dd", by using a VM, writing
Â*Â*Â*Â*Â*Â*Â*Â* a random pattern on a small data drive in the VM first
Â*Â*Â*Â*Â*Â*Â*Â* with dd. Then, do a "copy" pass as proposed. Then
Â*Â*Â*Â*Â*Â*Â*Â* read out the drive and compute a checksum and see if
Â*Â*Â*Â*Â*Â*Â*Â* the "before" and "after" checksum on the "copy" operation
Â*Â*Â*Â*Â*Â*Â*Â* are the same.

Â*Â*Â*Â*Â*Â*Â*Â* As far as I know, "dd" does read before write, but I don't
Â*Â*Â*Â*Â*Â*Â*Â* know if there are any corner conditions to be concerned
Â*Â*Â*Â*Â*Â*Â*Â* about or not. If the drive will not sustain an unadorned
Â*Â*Â*Â*Â*Â*Â*Â* dd operation attempting to read it from end to end,
Â*Â*Â*Â*Â*Â*Â*Â* then you don't want to try your copy pass. The copy pass
Â*Â*Â*Â*Â*Â*Â*Â* would be reserved for "good times", where to the best
Â*Â*Â*Â*Â*Â*Â*Â* of your knowledge, every write attempted by dd, would succeed.

Â*Â*Â*Â*Â*Â*Â*Â* The gddrescue has more recovery options, such that its
Â*Â*Â*Â*Â*Â*Â*Â* copy semantics are less likely to cause grief. It maintains
Â*Â*Â*Â*Â*Â*Â*Â* a logfile with data on what sectors kicked up a fuss.
Â*Â*Â*Â*Â*Â*Â*Â* Repeated runs can be used to "resolve problems" and
Â*Â*Â*Â*Â*Â*Â*Â* reduce the outstanding work shown in the logfile. The
Â*Â*Â*Â*Â*Â*Â*Â* package name in Linux is "gddrescue", but the executable
Â*Â*Â*Â*Â*Â*Â*Â* at runtime is "ddrescue". Block size on transfers, is variable,
Â*Â*Â*Â*Â*Â*Â*Â* and the program can gear down to "doing single sectors" if it
wants.
Â*Â*Â*Â*Â*Â*Â*Â* Regular dd is more likely to do fixed values of "bs=" size.

Â*Â*Â*Â*Â*Â*Â*Â* Practice on a VM drive *first*, before unleashing these
Â*Â*Â*Â*Â*Â*Â*Â* lengthy lengthy operations on a 6TB drive. All that a VM
Â*Â*Â*Â*Â*Â*Â*Â* drive can test, is that the basic operation works OK, not
Â*Â*Â*Â*Â*Â*Â*Â* that all the corner cases are handled. A gddrescue operation
Â*Â*Â*Â*Â*Â*Â*Â* on a virtual drive, will have an empty logfile. Or at least
Â*Â*Â*Â*Â*Â*Â*Â* the symbols indicating no work remains to be done. It won't
Â*Â*Â*Â*Â*Â*Â*Â* be a particularly human-readable format, because the notation
Â*Â*Â*Â*Â*Â*Â*Â* also seeks to compress the size of the logfile and not make
Â*Â*Â*Â*Â*Â*Â*Â* it huge-beyond-words.

Â*Â* Paul


Mostly over my head but thank you very much all the same!

On 8/13/20 3:18 PM, VanguardLH wrote:
"Percival P. Cassidy" wrote:

I have a 6TB drive that shows 8 bad sectors. Would it be possible to
repair it without harm to the data using

dd if=/dev/daX of=/dev/daX (and perhaps with bs settinng as well) ??

dd lets you copy an entire partition atop of itself? Seems dangerous.
If there is an error during the operation, well, that's why dd is also
known as disk destroyer.

I don't know for sure that dd does allow that. I was just wondering.

Presumably you don't know in file(s) encompass the bad sectors. Or do
any files use those sectors, and instead the bad sectors are in free
space?

I don't know -- haven't tried. But if I knew, I guess I could simply
make a copy of the file with a different name and it would end up on a
different part of the drive, and then I could try writing zeroes to the
bad sectors.

Why not use fsck (if the volume can be unmounted, like "sudo unmount
/dev/daX", and then "sudo fsck /dev/daX)? If the bad sectors are in
free space, go for it. If the bad sectors are used in files, do a
logical (file) backup beforehand, or use dd to save a disk image (in
another partition or to an .img file).

I haven't used [U|Li]nux for many years. Wait until a *NIX guru shows
up, but what you propose to write a partition atop of itself looks
dangerous, and I would think dd would error on that attempt.

Since dd is just a copy tool, I don't see that it will attempt to
recover iffy sectors. The copy may be devoid of any files with bad
sectors because, after all, they couldn't be read without error.

"dd conv=noerror" will continue after an error, but it skips bad blocks.
Adding the sync option just adds zeros instead of skipping the bad
blocks. gddrescue also keeps going after an error, but it attempts
recovery on a partial yield from a bad block while trying to read each
sector in a block. However, in either case, if a block is completely
unreadable then no data gets read from it into the copy or image.

Is this such an old HDD that it doesn't include its own firmware to
remap bad sectors using S.M.A.R.T.? If you read the SMART data from the
HDD, and there is still a non-zero Current Pending Sector Count (raw
count), that count doesn't go down after a reboot? The pending count
should go down (to do the remapping using the HDD's firmware) while the
Reallocated Sectors Count goes up.

It's a comparatively recent (although now out of warranty) Seagate SATA
drive.

Current Pending Sector Count and Uncorrectable Sector Count are both 8
sectors, but Reported Uncorrectable Sectors and Reallocated Sector Count
are both 0 sectors. Haven't changed after several reboots and power
off/on cycles.


https://www.disktuna.com/reallocated...h-bad-sectors/
See "Pending Sectors" section.

Note that more remapped sectors means slower read and write speeds. The
remapping (reading a remapped sector and then having to read the
alternate sector used for remapping) takes time. If the Current Pending
Sector Count does not go down, there are no more reserve sectors on that
hard disk, so the bad sectors cannot get remapped. If that count
doesn't go down, you need to look at replacing the hard disk, and soon.
If the count goes down, the HDD's firmware did its job of remapping the
bad sectors. With an ever-increasing pending count, you need to clone
to a partition on a new or different HDD, and get all partitions off the
old HDD onto the new one.

or There are many free tools to read SMART data from a drive. There might
be some good free disk health monitor tools, but to get decent testing
and recovery features often means having to pay. For example, there's a
free version of HD (Hard Disk) Sentinel. I use that but under Windows,
and I eventually went to a paid version. They have a full-feature trial
version usable for 30 days. It's been too long since I used their free
version to remember if it was effective to get bad sectors remapped (or
prods the HDD to do it). Try the free version, and it that is
insufficient then try their trial version. Just be aware that some
"refresh" operations on a disk can be destructive (make sure your files
are saved elsewhere).

A simple copy operation, like you proposed, does not get a bad sector
remapped. It just tries to copy what it can to the destination, but the
bad sector remains bad at the source.

I have done dd if=/dev/zero on several drives. The Pending Sector Count
and Uncorrectable Sector Count both become 0 sectors, but Reported
Uncorrectable Sectors and Reallocated Sector Count both remain at 0 sectors.

I have HD Sentinel Pro on a couple of Windows machines,, but i don't
know what a Windows machine will make of an ext4-formatted drive.

While disk recovery tools are geared towards recovering data from bad
sectors and then prodding the hardware to remap the bad sectors, maybe
you can use "smartctl -t offline /dev/daX" to tell the disk's firmware
to do an offline surface scan to get the bad sectors remapped. However,
I don't know that attempts any data recovery from the bad sectors.

I've run smartctl with both short and long offline tests, but they do
not bring the bad sector count down. They also show "Completed without
error."

Perce

"Percival P. Cassidy" wrote:

I have HD Sentinel Pro on a couple of Windows machines,, but i don't
know what a Windows machine will make of an ext4-formatted drive.

Supported operating systems:
https://www.hdsentinel.com/compatibility_os.php

Linux is supported.

https://www.hdsentinel.com/eula.php

Apparently, for personal use, the trialware version doesn't expire. I
don't know what they mean by "may be extended". Maybe you have to renew
the registration, or the trialware version cripples itself to the
freeware version.

Alas, it does appear 1 license can only be used on 1 computer, so no
sharing of a license between Windows and Linux hosts.

https://www.hdsentinel.com/hard_disk_sentinel_linux.php

I don't see anything mentioned that their tests are included in the
Linux version, like to read bad sectors, if possible, and get the hard
disk to remap the bad sectors.

https://www.diskpart.com/articles/fo...dows-7201.html

Seems Windows can read/write ext4-formatted partitions, but it cannot
create them. The article mentions a 3rd party partition manager (AOMEI)
will do the ext4 formatting, and so will other partition managers
(https://www.easeus.com/partition-man...ndows-10.html).
Of course, you don't want to format the HDD. Looks like Windows will
handle ext4. Just clone the HDD before attempting migration or brain
surgery on it.

I did mention other Linux tools to massage the HDD to get rid of the bad
sectors (if there are remaining free sectors for remapping).

I've run smartctl with both short and long offline tests, but they do
not bring the bad sector count down. They also show "Completed without
error."

Then perhaps there are no free sectors available for remapping. That's
what happens when the Pending count doesn't decrease: no spare sector
where to copy the data to remap the bad sector. You could try copy any
files that have the bad sectors (but you'll probably, at a minimum, lose
the bits in the bad sectors) to create a new file (after making sure the
bad sectors were already inuse, so they don't get used for the new
file), but without spare remapping sectors those bad sectors won't go
away, and some later file will try to use them.

Seagate has their own SeaTools you might try. Make a bootable CD from
which to run their software. That way, the bad HDD is quiescent, and it
doesn't matter what OS is there. Any remapping is done at the hardware
level, not at the OS level in whatever file system the OS uses.

https://www.seagate.com/support/downloads/seatools/
Get the bootable version. Then you don't have to install it in any OS,
and you don't have to move the HDD between computers.

VanguardLH wrote:

Seagate has their own SeaTools you might try. Make a bootable CD from
which to run their software. That way, the bad HDD is quiescent, and it
doesn't matter what OS is there. Any remapping is done at the hardware
level, not at the OS level in whatever file system the OS uses.

https://www.seagate.com/support/downloads/seatools/
Get the bootable version. Then you don't have to install it in any OS,
and you don't have to move the HDD between computers.

I found:

https://www.hdsentinel.com/usbboot.php

I'm guessing it uses an image of their free DOS edition mentioned at:

https://www.hdsentinel.com/hard_disk_sentinel_dos.php

I still don't see in its description that it does any testing to prod
the HHD's firmware to remap the bad sectors. Seems to be just for
analysis. The Seatools might just be a diagnostic tools, too.

I'm surprised smartctl didn't get remapped the bad sectors/blocks. Bad
blocks is one of its functions. If there are no spare sectors reserved
for remapping, then there's no way to remap any more bad sectors.

There is no information in SMART or by reading an HDD's firmware
signature to know the size of the pdata table. The p-list table is the
remapping done at the factory. g-list (growth list) is used for further
remapping during the use of the drive. I've seen where there was a jump
in bad sectors that got remapped (pending count went up, and then fell
to zero), and it could be a couple hundred sectors; however, without
knowing the total size of the g-list table (to know how many spare
sectors are available), there is no way to know how much is left. The
drive makers don't publish the size of their g-list table. 200 bad
sectors is really bad if there are only 400 spare sectors total for
remapping, but not immediately serious if there are 20,000 spare
sectors. Bad sectors tend to grow radially on spinning magnetic media,
so if some show up then it becomes more likely that more will show up.
You cannot know severity of a remap event when the total or remaining
capacity isn't known.

But if the HDD's firmware doesn't remap the bad sectors, that could be
because there are no more spare sectors.

https://dtidatarecovery.com/how-to-f...-a-hard-drive/
"Since we know that all bad sectors are remapped to a pool of unused
sectors, and the size of this pool is substantial, the only way a bad
sector will show up on your system is if the pool has been completely
used. In other words, your remapping pool is so full that it cannot take
another bad sector and now the drive is throwing errors that it can¢t
read from a sector."

Start checking on prices for replacement HDDs of equal, or greater,
capacity to which you can clone the old HDD. If possible, see if you
can determine in which files are the bad sectors. The clone software
should keep a long, too, to let you know if there were read errors from
the old drive. If they are data files, you can recover those from your
backups. If they are program files, well, you could uninstall and
reinstall the programs to ensure all their bits are correct. If the bad
sectors are in system/OS files (which rarely get rewritten except by
updates, and then only some of the OS files get replaced), well, you'll
have to see if the cloned HDD is bootable, the OS behaves, and you
experience no problems, or reinstall the OS on the new HDD, reinstall
all the apps, and restore the data files to it. If your backups include
full-partition images, you can obviously not bother with the clone
operating and just restore the backup image to the new drive.

If the pending count doesn't go down, you have a drive that won't heal.

On 8/14/20 4:44 PM, VanguardLH wrote:
VanguardLH wrote:

Seagate has their own SeaTools you might try. Make a bootable CD from
which to run their software. That way, the bad HDD is quiescent, and it
doesn't matter what OS is there. Any remapping is done at the hardware
level, not at the OS level in whatever file system the OS uses.

https://www.seagate.com/support/downloads/seatools/
Get the bootable version. Then you don't have to install it in any OS,
and you don't have to move the HDD between computers.

I found:

https://www.hdsentinel.com/usbboot.php

I'm guessing it uses an image of their free DOS edition mentioned at:

https://www.hdsentinel.com/hard_disk_sentinel_dos.php

I still don't see in its description that it does any testing to prod
the HHD's firmware to remap the bad sectors. Seems to be just for
analysis. The Seatools might just be a diagnostic tools, too.

I'm surprised smartctl didn't get remapped the bad sectors/blocks. Bad
blocks is one of its functions. If there are no spare sectors reserved
for remapping, then there's no way to remap any more bad sectors.

There is no information in SMART or by reading an HDD's firmware
signature to know the size of the pdata table. The p-list table is the
remapping done at the factory. g-list (growth list) is used for further
remapping during the use of the drive. I've seen where there was a jump
in bad sectors that got remapped (pending count went up, and then fell
to zero), and it could be a couple hundred sectors; however, without
knowing the total size of the g-list table (to know how many spare
sectors are available), there is no way to know how much is left. The
drive makers don't publish the size of their g-list table. 200 bad
sectors is really bad if there are only 400 spare sectors total for
remapping, but not immediately serious if there are 20,000 spare
sectors. Bad sectors tend to grow radially on spinning magnetic media,
so if some show up then it becomes more likely that more will show up.
You cannot know severity of a remap event when the total or remaining
capacity isn't known.

But if the HDD's firmware doesn't remap the bad sectors, that could be
because there are no more spare sectors.

https://dtidatarecovery.com/how-to-f...-a-hard-drive/
"Since we know that all bad sectors are remapped to a pool of unused
sectors, and the size of this pool is substantial, the only way a bad
sector will show up on your system is if the pool has been completely
used. In other words, your remapping pool is so full that it cannot take
another bad sector and now the drive is throwing errors that it can¢t
read from a sector."

Start checking on prices for replacement HDDs of equal, or greater,
capacity to which you can clone the old HDD. If possible, see if you
can determine in which files are the bad sectors. The clone software
should keep a long, too, to let you know if there were read errors from
the old drive. If they are data files, you can recover those from your
backups. If they are program files, well, you could uninstall and
reinstall the programs to ensure all their bits are correct. If the bad
sectors are in system/OS files (which rarely get rewritten except by
updates, and then only some of the OS files get replaced), well, you'll
have to see if the cloned HDD is bootable, the OS behaves, and you
experience no problems, or reinstall the OS on the new HDD, reinstall
all the apps, and restore the data files to it. If your backups include
full-partition images, you can obviously not bother with the clone
operating and just restore the backup image to the new drive.

If the pending count doesn't go down, you have a drive that won't heal.


The Pending Count alwsys does go to zero after writing zeroes to the
whole drive.

Perce

"Percival P. Cassidy" wrote:

On 8/14/20 4:44 PM, VanguardLH wrote:
VanguardLH wrote:

Seagate has their own SeaTools you might try. Make a bootable CD from
which to run their software. That way, the bad HDD is quiescent, and it
doesn't matter what OS is there. Any remapping is done at the hardware
level, not at the OS level in whatever file system the OS uses.

https://www.seagate.com/support/downloads/seatools/
Get the bootable version. Then you don't have to install it in any OS,
and you don't have to move the HDD between computers.

I found:

https://www.hdsentinel.com/usbboot.php

I'm guessing it uses an image of their free DOS edition mentioned at:

https://www.hdsentinel.com/hard_disk_sentinel_dos.php

I still don't see in its description that it does any testing to prod
the HHD's firmware to remap the bad sectors. Seems to be just for
analysis. The Seatools might just be a diagnostic tools, too.

I'm surprised smartctl didn't get remapped the bad sectors/blocks. Bad
blocks is one of its functions. If there are no spare sectors reserved
for remapping, then there's no way to remap any more bad sectors.

There is no information in SMART or by reading an HDD's firmware
signature to know the size of the pdata table. The p-list table is the
remapping done at the factory. g-list (growth list) is used for further
remapping during the use of the drive. I've seen where there was a jump
in bad sectors that got remapped (pending count went up, and then fell
to zero), and it could be a couple hundred sectors; however, without
knowing the total size of the g-list table (to know how many spare
sectors are available), there is no way to know how much is left. The
drive makers don't publish the size of their g-list table. 200 bad
sectors is really bad if there are only 400 spare sectors total for
remapping, but not immediately serious if there are 20,000 spare
sectors. Bad sectors tend to grow radially on spinning magnetic media,
so if some show up then it becomes more likely that more will show up.
You cannot know severity of a remap event when the total or remaining
capacity isn't known.

But if the HDD's firmware doesn't remap the bad sectors, that could be
because there are no more spare sectors.

https://dtidatarecovery.com/how-to-f...-a-hard-drive/
"Since we know that all bad sectors are remapped to a pool of unused
sectors, and the size of this pool is substantial, the only way a bad
sector will show up on your system is if the pool has been completely
used. In other words, your remapping pool is so full that it cannot take
another bad sector and now the drive is throwing errors that it can¢t
read from a sector."

Start checking on prices for replacement HDDs of equal, or greater,
capacity to which you can clone the old HDD. If possible, see if you
can determine in which files are the bad sectors. The clone software
should keep a long, too, to let you know if there were read errors from
the old drive. If they are data files, you can recover those from your
backups. If they are program files, well, you could uninstall and
reinstall the programs to ensure all their bits are correct. If the bad
sectors are in system/OS files (which rarely get rewritten except by
updates, and then only some of the OS files get replaced), well, you'll
have to see if the cloned HDD is bootable, the OS behaves, and you
experience no problems, or reinstall the OS on the new HDD, reinstall
all the apps, and restore the data files to it. If your backups include
full-partition images, you can obviously not bother with the clone
operating and just restore the backup image to the new drive.

If the pending count doesn't go down, you have a drive that won't heal.

The Pending Count alwsys does go to zero after writing zeroes to the
whole drive.

A bad sector does not become a good sector because of what bit string
was written to it. That's like saying a crack plate atop of which is
chocolate cake becomes uncracked because the chocolate cake got removed
to put angel food cake atop the plate.

Just how are you determining there are bad sectors? More likely what
happened is that those bad sectors where not in use by files that were
getting rewritten. The write operation (1's or 0's) will generate the
sector error whereupon the drive's firmware will remap the bad sectors
to spare/reserve sectors. Something has to use the bad sector for the
drive to see there are errors using the bad sector and to remap it.

On 8/15/20 12:24 AM, VanguardLH wrote:
"Percival P. Cassidy" wrote:

On 8/14/20 4:44 PM, VanguardLH wrote:
VanguardLH wrote:

Seagate has their own SeaTools you might try. Make a bootable CD from
which to run their software. That way, the bad HDD is quiescent, and it
doesn't matter what OS is there. Any remapping is done at the hardware
level, not at the OS level in whatever file system the OS uses.

https://www.seagate.com/support/downloads/seatools/
Get the bootable version. Then you don't have to install it in any OS,
and you don't have to move the HDD between computers.

I found:

https://www.hdsentinel.com/usbboot.php

I'm guessing it uses an image of their free DOS edition mentioned at:

https://www.hdsentinel.com/hard_disk_sentinel_dos.php

I still don't see in its description that it does any testing to prod
the HHD's firmware to remap the bad sectors. Seems to be just for
analysis. The Seatools might just be a diagnostic tools, too.

I'm surprised smartctl didn't get remapped the bad sectors/blocks. Bad
blocks is one of its functions. If there are no spare sectors reserved
for remapping, then there's no way to remap any more bad sectors.

There is no information in SMART or by reading an HDD's firmware
signature to know the size of the pdata table. The p-list table is the
remapping done at the factory. g-list (growth list) is used for further
remapping during the use of the drive. I've seen where there was a jump
in bad sectors that got remapped (pending count went up, and then fell
to zero), and it could be a couple hundred sectors; however, without
knowing the total size of the g-list table (to know how many spare
sectors are available), there is no way to know how much is left. The
drive makers don't publish the size of their g-list table. 200 bad
sectors is really bad if there are only 400 spare sectors total for
remapping, but not immediately serious if there are 20,000 spare
sectors. Bad sectors tend to grow radially on spinning magnetic media,
so if some show up then it becomes more likely that more will show up.
You cannot know severity of a remap event when the total or remaining
capacity isn't known.

But if the HDD's firmware doesn't remap the bad sectors, that could be
because there are no more spare sectors.

https://dtidatarecovery.com/how-to-f...-a-hard-drive/
"Since we know that all bad sectors are remapped to a pool of unused
sectors, and the size of this pool is substantial, the only way a bad
sector will show up on your system is if the pool has been completely
used. In other words, your remapping pool is so full that it cannot take
another bad sector and now the drive is throwing errors that it can¢t
read from a sector."

Start checking on prices for replacement HDDs of equal, or greater,
capacity to which you can clone the old HDD. If possible, see if you
can determine in which files are the bad sectors. The clone software
should keep a long, too, to let you know if there were read errors from
the old drive. If they are data files, you can recover those from your
backups. If they are program files, well, you could uninstall and
reinstall the programs to ensure all their bits are correct. If the bad
sectors are in system/OS files (which rarely get rewritten except by
updates, and then only some of the OS files get replaced), well, you'll
have to see if the cloned HDD is bootable, the OS behaves, and you
experience no problems, or reinstall the OS on the new HDD, reinstall
all the apps, and restore the data files to it. If your backups include
full-partition images, you can obviously not bother with the clone
operating and just restore the backup image to the new drive.

If the pending count doesn't go down, you have a drive that won't heal.

The Pending Count alwsys does go to zero after writing zeroes to the
whole drive.

A bad sector does not become a good sector because of what bit string
was written to it. That's like saying a crack plate atop of which is
chocolate cake becomes uncracked because the chocolate cake got removed
to put angel food cake atop the plate.

Just how are you determining there are bad sectors? More likely what
happened is that those bad sectors where not in use by files that were
getting rewritten. The write operation (1's or 0's) will generate the
sector error whereupon the drive's firmware will remap the bad sectors
to spare/reserve sectors. Something has to use the bad sector for the
drive to see there are errors using the bad sector and to remap it.

I write zeroes to the whole drive, then run a long smartctl test,
followed my smartctl -x. Pending and Uncorrectable counts are then found
to be zero but there is no indication that any sectors have been remapped.

This is using smartctl in FreeNAS (based on FreeBSD).

Perce