Very Slow deleting from USB thumb drive.

Hello group - I've just transfered nearly 1 gig of files from my thumb
drive to my PC hard drive via USB 2.0 port and the transfer was
pleasingly fast. However, now I try to delete the files on the thumb
drive and the process is painfully slow deletes the files. What
gives? Why is it so slow deleting when the transferring (copying) was
so fast?? Watching task manager during the deleting process showed
very little use of CPU. I have a new 2.2 gig dual core processor.

Can anyone shed some light?

thanks,
Robert

Previously wrote:
Hello group - I've just transfered nearly 1 gig of files from my thumb
drive to my PC hard drive via USB 2.0 port and the transfer was
pleasingly fast. However, now I try to delete the files on the thumb
drive and the process is painfully slow deletes the files. What
gives? Why is it so slow deleting when the transferring (copying) was
so fast?? Watching task manager during the deleting process showed
very little use of CPU. I have a new 2.2 gig dual core processor.

Can anyone shed some light?

Writing to flash is inherently several orders of magnitude slower
than reading from it. Historically writing took something like
50ms and reading something like 1us, i.e. 50'000 times faster.
These times are over, but a speed difference by a factor of 10
is still not uncommon. Every write to FLASH damages it. The problem
is that writing FLASH fast, while possible, damages the FLASH more
than a slow write, so writes are slowed down, expecially in cheaper
FLASH (or, better: "not extremely expensive FLASH).

Arno

(sorry for the one month delay)
On Thu, 28 Feb 2008 11:57:05 -0800 (PST),
wrote:

Hello group - I've just transfered nearly 1 gig of files from my thumb
drive to my PC hard drive via USB 2.0 port and the transfer was
pleasingly fast. However, now I try to delete the files on the thumb
drive and the process is painfully slow deletes the files. What
gives? Why is it so slow deleting when the transferring (copying) was
so fast?? Watching task manager during the deleting process showed
very little use of CPU. I have a new 2.2 gig dual core processor.

Can anyone shed some light?
How many files were involved?

While writing is now typically more than 1/2 the speed as reading
for large files, writing an operating system sector or block can
involve writing a hardware block. I am told that typical hardware
blocks are 1MB with today's flash memory drives.

If the flash memory key is mounted to facilitate fast removal,
each new file, matter how small, may require 1 write, 2 writes
if it doesn't fit in the MFT.

In addition, some programs, even backup programs that know how
much room is needed for an entire input file, may allocate space
in single clusters, which means that writing even a large file may
require multiple writes of large physical blocks for each 4096 byte
sector.

One way of speeding up things is to make a virtual disk on the output
device - it is more likely that the copy operation will be entirely
lost if there is a power failure during the copy, but it might
take only 50 seconds to copy 4GB to a virtual disk as compared to
about 250 times as long (almost 4 hours) to write directly to the
flash memory key.

The flash memory manufacturers know all of the above, but they
keep information such as physical blocksize "under wraps", so setting
up things to get the best performance for a particular flash memory
key is a matter of trial and error.

Since the manufacturers don't want to reveal anything it is hard
to determine which USB flash memory key is best for your application.

I have about 12 different models flash memory keys. Most have been
very slow when writing my favorite 2GB or so of data in 16000 or so
files. My experience is that each device is optimized for a
particular type of access but that the manufacturer won't reveal what
that access pattern is.

Since I haven't run extensive tests, the only comparison that
I'll make is that the Corsair Flash Voyager GT 8GB is several times
faster Corsair Flash Voyager 8GB when writing using the Windows
XP Copy operation.
than the Corsair


thanks,
Robert

On Sun, 06 Apr 2008 18:29:48 -0400, Mark F
wrote:

(sorry for the one month delay)
On Thu, 28 Feb 2008 11:57:05 -0800 (PST),
wrote:
Oops - I meant to give both the writing side and the deleting side of
the story, but I only gave the writing side.

Hello group - I've just transfered nearly 1 gig of files from my thumb
drive to my PC hard drive via USB 2.0 port and the transfer was
pleasingly fast. However, now I try to delete the files on the thumb
drive and the process is painfully slow deletes the files. What
gives? Why is it so slow deleting when the transferring (copying) was
so fast?? Watching task manager during the deleting process showed
very little use of CPU. I have a new 2.2 gig dual core processor.

Can anyone shed some light?
How many files were involved?
How slow is "painfully slow"? I found the write times to be
painfully slow, but, having figured out why they were so slow
I wasn't surprised at the long times needed for the deletes.
Basically, if your files only take a cluster or so on average
the delete will take about 1/2 the time as the copy. If you have a
few MB or larger files in the mix the write time might not seem
so bad, but the delete time, taking 20 or more times the delete
time on a hard disk, might seem "painfully slow".

While writing is now typically more than 1/2 the speed as reading
for large files, writing an operating system sector or block can
involve writing a hardware block. I am told that typical hardware
blocks are 1MB with today's flash memory drives.

If the flash memory key is mounted to facilitate fast removal,
each new file, matter how small, may require 1 write, 2 writes
if it doesn't fit in the MFT.

In addition, some programs, even backup programs that know how
much room is needed for an entire input file, may allocate space
in single clusters, which means that writing even a large file may
require multiple writes of large physical blocks for each 4096 byte
sector.

One way of speeding up things is to make a virtual disk on the output
device - it is more likely that the copy operation will be entirely
lost if there is a power failure during the copy, but it might
take only 50 seconds to copy 4GB to a virtual disk as compared to
about 250 times as long (almost 4 hours) to write directly to the
flash memory key.

The flash memory manufacturers know all of the above, but they
keep information such as physical blocksize "under wraps", so setting
up things to get the best performance for a particular flash memory
key is a matter of trial and error.

Since the manufacturers don't want to reveal anything it is hard
to determine which USB flash memory key is best for your application.

I have about 12 different models flash memory keys. Most have been
very slow when writing my favorite 2GB or so of data in 16000 or so
files. My experience is that each device is optimized for a
particular type of access but that the manufacturer won't reveal what
that access pattern is.

Since I haven't run extensive tests, the only comparison that
I'll make is that the Corsair Flash Voyager GT 8GB is several times
faster Corsair Flash Voyager 8GB when writing using the Windows
XP Copy operation.
than the Corsair


thanks,
Robert

On Apr 7, 12:14*am, Mark F wrote:
On Sun, 06 Apr 2008 18:29:48 -0400, Mark F
wrote:

(sorry for the one month delay)
On Thu, 28 Feb 2008 11:57:05 -0800 (PST),
wrote:

Oops - I meant to give both the writing side and the deleting side of
the story, but I only gave the writing side.

Hello group - I've just transfered nearly 1 gig of files from my thumb
drive to my PC hard drive via USB 2.0 port and the transfer was
pleasingly fast. *However, now I try to delete the files on the thumb
drive and the process is painfully slow deletes the files. *What
gives? *Why is it so slow deleting when the transferring (copying) was
so fast?? *Watching task manager during the deleting process showed
very little use of CPU. *I have a new 2.2 gig dual core processor.

Can anyone shed some light?
How many files were involved?

How slow is "painfully slow"? *I found the write times to be
painfully slow, but, having figured out why they were so slow
I wasn't surprised at the long times needed for the deletes.
Basically, if your files only take a cluster or so on average
the delete will take about 1/2 the time as the copy. *If you have a
few MB or larger files in the mix the write time might not seem
so bad, but the delete time, taking 20 or more times the delete
time on a hard disk, might seem "painfully slow".





While writing is now typically more than 1/2 the speed as reading
for large files, writing an operating system sector or block can
involve writing a hardware block. *I am told that typical hardware
blocks are 1MB with today's flash memory drives.

If the flash memory key is mounted to facilitate fast removal,
each new file, matter how small, may require 1 write, 2 writes
if it doesn't fit in the MFT.

In addition, some programs, even backup programs that know how
much room is needed for an entire input file, may allocate space
in single clusters, which means that writing even a large file may
require multiple writes of large physical blocks for each 4096 byte
sector.

One way of speeding up things is to make a virtual disk on the output
device - it is more likely that the copy operation will be entirely
lost if there is a power failure during the copy, but it might
take only 50 seconds to copy 4GB to a virtual disk as compared to
about 250 times as long (almost 4 hours) to write directly to the
flash memory key.

The flash memory manufacturers know all of the above, but they
keep information such as physical blocksize "under wraps", so setting
up things to get the best performance for a particular flash memory
key is a matter of trial and error.

Since the manufacturers don't want to reveal anything it is hard
to determine which USB flash memory key is best for your application.

I have about 12 different models flash memory keys. *Most have been
very slow when writing my favorite 2GB or so of data in 16000 or so
files. *My experience is that each device is optimized for a
particular type of access but that the manufacturer won't reveal what
that access pattern is. *

Since I haven't run extensive tests, the only comparison that
I'll make is that the Corsair Flash Voyager GT 8GB is several times
faster Corsair Flash Voyager 8GB when writing using the Windows
XP Copy operation.
than the Corsair

thanks,
Robert- Hide quoted text -

- Show quoted text -

You mention writing MFTs. This means that you are using NTFS.

I have not seen a 'customer' thumb drive that uses NTFS, all the ones
I see are FAT16 or FAT32.

Thumb drives do work OK as NTFS, but I expect it would need to be
converted first.

Is your thumb drive NTFS or FAT32?


Michael
www.cnwrecovery.com