View Single Post
  #14  
Old December 6th 18, 03:01 PM posted to alt.comp.hardware.pc-homebuilt
[email protected]
external usenet poster
 
Posts: 533
Default Copieing 60.000 items from Windows Live Mail extreme performance degradation

On Wednesday, December 5, 2018 at 7:53:49 PM UTC+1, Paul wrote:
wrote:
Disabling the mouse is done by clicking on the title bar of cmd.exe window and going to properties and disabling "quick edit mode".

Quick edit mode is insane.

Anyway I will try the following command:

robocopy w: r: /MIR /ETA

This is the last damn thing I am going to try and if I don't like it, this will be the end of it, though I did already noticed "lagging in sound playback" so this could indicate it's somewhat faster perhaps.

Here goes:

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.

C:\Users\Skybuckr:

R:\dir
Volume in drive R is TestTestTest
Volume Serial Number is D4F6-722F

Directory of R:\

File Not Found

R:\w:

W:\dir
Volume in drive W is Windows Live Mail
Volume Serial Number is 103F-69D4

Directory of W:\

05/12/2018 17:23 DIR Windows Live Mail
0 File(s) 0 bytes
1 Dir(s) 4,536,350,720 bytes free

W:\robocopy w: r: /MIR /ETA


It's a folder copying tool, and I was hoping you'd just
copy a folder, not a whole hard drive partition.

To copy the C: drive, requires at least one other special
parameter to avoid problems with Junction Points.

To measure drive performance, on an older OS...

Start : Run

perfmon.msc

In there, right click to add new performance counters.
Add the "Physical Disk" one, which has "Disk write bytes per second"
or similar. As well as "Disk read bytes per second".

The perfmon.msc display will show the transfer rate.

On Win8/Win10, the Task Manager performance display, and
the particular disk, you click on that and a transfer
graph is shown.

On Win7/Win8/Win10, there is Resource Monitor, a button
you click from the Performance screen of Task Manager.
it shows transfer rate versus "program", such as robocopy.

There are plenty of ways to monitor transfers, without
resorting to "/ETA".

Robocopy is capable of async transfer, where reads and writes
can overlap on two drives. It's unlike a conventional
copy where "read" then "write" happens, as the tool alternates
between drives for each phase. The extent to which overlap
happens on the async (non-blocking) transfer, has varied
a lot over the years. But with some luck, you might see
it overlapping the reads and writes (on two different hard
drives). If you transfer files between two partitions
on the same hard drive, the operation will be serialized
by the hardware itself (the need to move the heads back
and forth between partitions).

Paul


I tried this, it does work... it's a bit cumbersome to setup... especially the scale is somewhat weird.

I setup it up into "report mode" for the graph...

I will have to do more experimentation with seconds for measuring the average and such... it was set to 100.

This is a bit long... 10 or 15 secs might be more appriorate.

The robocopy copy speed fluctuates a lot between reading and writing.

Eventually it did go way down to just 1.7 megabytes/sec.

So this more or less does prove NTFS has a problem as far as I am concerned.

A new file system could be designed which stores many small files into 1 single file instead of many to reduce seeks on disk, and instead seek in memory.

Anyway the VCL media player wasn't even playing this time.

I am not convinced that robocopy is any faster, it may actually be slower though, though the console i/o test might have something to do with that though it didn't seem to be too bad.

Overall not much difference speed wise... maybe at the start though... but those were large files... maybe for larger files robocopy might be a bit faster, not sure about that.

It's interesting to see with what file it's busy though... though for long folders the text display becomes a bit short.

Also now estimated time left which is a big short coming in robocopy + perfmon

But thanks for your tips, they were a bit interesting ! =D

Especially the perfmon... it does show "total disk performance"... something which resource monitor struggles with somewhat... I think... not sure... there is total i/o... but I am not sure it's seperated into read vs write speeds for totals.

Bye,
Skybuck =D