ALWAYS close USB flash drive before removing?

Jerry Peters wrote:
In comp.sys.ibm.pc.hardware.storage Arno wrote:
Well, in a perfect world apps would also make sure that everything
they write to disk is consistent immediately after the write and
then close the file and reopen it for further writes. They
should also open files only for reading, unless they really need
to wrote. Unfortunately the worls is not perfect and some
people cannot abandon misbehaving apps.

The advice to always securely remove is in respect of that
reality. Yanking out can be made very safe, indicentially,
from a filesystem PoV by using journalling as for example
Linux ext3 does. This still leaves the app problem. In
addition, under Windows, even the filesystem level safety
is not present.

Arno

I'd worry about what happens with a poorly coded FTL in the drive if
power happens to be removed while a write is in progress. Could lead
to corrution or even complete failure of the drive.

Hmm. Good point. _Hardware_ that has issues with being
removed while writing is done is an entirely more serious
problem.

There was a very extensive series of emails about this on the Linux
kernel mailing list a few weeks ago, prompted by tests performed by
one of the developers. The summary was that even a jfs won't save you
if the FTL gets damaged.

Can you give me a link to the initial posting? I did not
find it on a quick search.

Arno

--
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email:
GnuPG: ID: 1E25338F FP: 0C30 5782 9D93 F785 E79C 0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

On Sun, 22 Nov 2009 13:44:40 -0800, Ant
wrote:

On 11/22/2009 1:30 PM PT, Arno typed:

The dismount is pretty important. Otherwise an application
could start writing between the "you may now remove..."
message and the actual removal.

Question: How does one unmount/dismount in Windows 98 SE since it
doesn't have those USB safety removal option in its system tray?


By physically removing it from the USB port. Just as with
later windows OS left at their defaults, you don't need to
unmount the volume. If an app then claims it can't write,
plug the flash drive back in, though it is typically pretty
clear whether that situation would exist, who really wants
apps writing to their flash drive that they weren't aware
of, and who doesn't realize that when they produce something
in an app they have to save the file at the end?

Ultimately, instead of training a user to use a removal app
if there is or were one, they can simply be instructed to
carry on as they always have, saving files while the drive
is plugged in and not removing it while the access indicator
LED is showing access.

Arno wrote:
Jerry Peters wrote:
In comp.sys.ibm.pc.hardware.storage Arno wrote:
Well, in a perfect world apps would also make sure that everything
they write to disk is consistent immediately after the write and
then close the file and reopen it for further writes. They
should also open files only for reading, unless they really need
to wrote. Unfortunately the worls is not perfect and some
people cannot abandon misbehaving apps.

The advice to always securely remove is in respect of that
reality. Yanking out can be made very safe, indicentially,
from a filesystem PoV by using journalling as for example
Linux ext3 does. This still leaves the app problem. In
addition, under Windows, even the filesystem level safety
is not present.

Arno

I'd worry about what happens with a poorly coded FTL in the drive if
power happens to be removed while a write is in progress. Could lead
to corrution or even complete failure of the drive.

Hmm. Good point. _Hardware_ that has issues with being
removed while writing is done is an entirely more serious
problem.

There was a very extensive series of emails about this on the Linux
kernel mailing list a few weeks ago, prompted by tests performed by
one of the developers. The summary was that even a jfs won't save you
if the FTL gets damaged.

Can you give me a link to the initial posting? I did not
find it on a quick search.

Arno


The original poster was Pavel Machek with a title like "ext3 unsafe on
flash device". IIRC LWN also had an article on the thread.

Pavel claims to have destroyed several devices by removing them with
writes in progress, in addition to corrupting filesystems.

The FS problem is that the underlying blocksize of the flash medium is
larger than the FS block size so that additional data can become corrupted
which the FS does not anticipate.

Jerry

Jerry Peters wrote:
Arno wrote:
Jerry Peters wrote:
In comp.sys.ibm.pc.hardware.storage Arno wrote:
Well, in a perfect world apps would also make sure that everything
they write to disk is consistent immediately after the write and
then close the file and reopen it for further writes. They
should also open files only for reading, unless they really need
to wrote. Unfortunately the worls is not perfect and some
people cannot abandon misbehaving apps.

The advice to always securely remove is in respect of that
reality. Yanking out can be made very safe, indicentially,
from a filesystem PoV by using journalling as for example
Linux ext3 does. This still leaves the app problem. In
addition, under Windows, even the filesystem level safety
is not present.

Arno

I'd worry about what happens with a poorly coded FTL in the drive if
power happens to be removed while a write is in progress. Could lead
to corrution or even complete failure of the drive.

Hmm. Good point. _Hardware_ that has issues with being
removed while writing is done is an entirely more serious
problem.

There was a very extensive series of emails about this on the Linux
kernel mailing list a few weeks ago, prompted by tests performed by
one of the developers. The summary was that even a jfs won't save you
if the FTL gets damaged.

Can you give me a link to the initial posting? I did not
find it on a quick search.

Arno


The original poster was Pavel Machek with a title like "ext3 unsafe on
flash device". IIRC LWN also had an article on the thread.

Found it, thanks. The first relevant message of the thread
seems to be this one he

http://lkml.indiana.edu/hypermail/li...1.0/01044.html

Pavel claims to have destroyed several devices by removing them with
writes in progress, in addition to corrupting filesystems.

I did not find the part about broken hardware as a result of
removal, but the serious corruption scenario were you get corruption
in data that was not even written (due to large sector saize up to a
MB on some Flash) is entirely plausible.

The FS problem is that the underlying blocksize of the flash medium
is larger than the FS block size so that additional data can become
corrupted which the FS does not anticipate.

Indeed. And this scenario makes the safe removal even
more important for Flash.

Seems my suspicions of Flash being unreliable in general
are more true than I thought.

Arnp
--
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email:
GnuPG: ID: 1E25338F FP: 0C30 5782 9D93 F785 E79C 0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

Arno wrote:
Jerry Peters wrote:
Arno wrote:
Jerry Peters wrote:
In comp.sys.ibm.pc.hardware.storage Arno wrote:
Well, in a perfect world apps would also make sure that everything
they write to disk is consistent immediately after the write and
then close the file and reopen it for further writes. They
should also open files only for reading, unless they really need
to wrote. Unfortunately the worls is not perfect and some
people cannot abandon misbehaving apps.

The advice to always securely remove is in respect of that
reality. Yanking out can be made very safe, indicentially,
from a filesystem PoV by using journalling as for example
Linux ext3 does. This still leaves the app problem. In
addition, under Windows, even the filesystem level safety
is not present.

Arno

I'd worry about what happens with a poorly coded FTL in the drive if
power happens to be removed while a write is in progress. Could lead
to corrution or even complete failure of the drive.

Hmm. Good point. _Hardware_ that has issues with being
removed while writing is done is an entirely more serious
problem.

There was a very extensive series of emails about this on the Linux
kernel mailing list a few weeks ago, prompted by tests performed by
one of the developers. The summary was that even a jfs won't save you
if the FTL gets damaged.

Can you give me a link to the initial posting? I did not
find it on a quick search.

Arno


The original poster was Pavel Machek with a title like "ext3 unsafe on
flash device". IIRC LWN also had an article on the thread.

Found it, thanks. The first relevant message of the thread
seems to be this one he

http://lkml.indiana.edu/hypermail/li...1.0/01044.html

Pavel claims to have destroyed several devices by removing them with
writes in progress, in addition to corrupting filesystems.

I did not find the part about broken hardware as a result of
removal, but the serious corruption scenario were you get corruption
in data that was not even written (due to large sector saize up to a
MB on some Flash) is entirely plausible.

The FS problem is that the underlying blocksize of the flash medium
is larger than the FS block size so that additional data can become
corrupted which the FS does not anticipate.

Indeed. And this scenario makes the safe removal even
more important for Flash.

Seems my suspicions of Flash being unreliable in general
are more true than I thought.

Arnp

Might have been another poster that mentioned destroying a flash
drive.
I think Ted Tso gave a short exposition on the FTL and problems if
the ordering of operations is not correct.

The whole thread made interesting reading. My recomendation is to
always use the OS provided mechanism to safely remove flash drives.
It's sort of like wearing seatbelts while driving, 99.99% of the time
they're superfulous, it's the other .01% that gets you.

Jerry