A computer components & hardware forum. HardwareBanter

If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

Go Back   Home » HardwareBanter forum » General Hardware & Peripherals » Storage (alternative)
Site Map Home Register Authors List Search Today's Posts Mark Forums Read Web Partners

Why is a smaller folder taking longer to be backed up than a largerone?



 
 
Thread Tools Display Modes
  #11  
Old January 11th 18, 06:34 AM posted to alt.comp.os.windows-10,comp.sys.ibm.pc.hardware.storage
Paul[_28_]
external usenet poster
 
Posts: 1,467
Default Why is a smaller folder taking longer to be backed up than alarger one?

Yousuf Khan wrote:
On 1/10/2018 1:08 AM, Yousuf Khan wrote:
On 1/9/2018 2:15 AM, Paul wrote:
To establish baseline performance, you can use Robocopy to copy the
files to a RAMdisk. That should expose any source HDD performance
anomalies.

And in the "not a fair fight" category, you can also try 7ZIP
and use "Store Mode", which does not compress a file tree when
creating an archive on another storage device. That should
return a much better result than your Macrium result.


Hmm, those are interesting ideas.


Okay, so I took your advice, and ran a Robocopy to a RAMdisk to see how
fast that goes. It still took over 2.5 hours just to copy to RAMdisk!

Robocopy's own output showed:
Total Copied Skipped Mismatch FAILED Extras
Dirs : 21 20 1 0 0 0
Files : 652802 652802 0 0 0 0
Bytes : 1.580 g 1.580 g 0 0 0 0
Times : 2:38:31 2:35:58 0:00:00 0:02:32


Speed : 181312 Bytes/sec.
Speed : 10.374 MegaBytes/min.

Strange observations: The total space of the files is 1.6 GB, while the
total allocated space is 3.2 GB. This makes sense, in that NTFS blocks
are probably not being fully utilized from file to file. However, the
strange thing I ran into on the RAMdisk is that I constantly kept
running out of space, even though I allocated more than 3.2 GB
everytime. I started out allocating 3.5 GB, then 5 GB, and then finally
8 GB, and then it finally worked! Right now, my 8 GB RAMdisk is showing
Used space of 5.9 GB, while Free space is 2.1 GB. So if the files are
only taking up 3.2 GB, then 2.7 GB of space is being taken up by
something else? NTFS metadata maybe?

Then I tried an archiving operation from the RAMdisk back to the HDD,
using the zip-store method, and that took 51:49 minutes. I assume it
will take Macrium about as long to archive off of the RAMdisk as a zip
archiver would, so I could conceivably save a ton of time just by
copying the folder to a RAMdisk and then archiving off of the RAMdisk.
But then I'd need to keep an 8GB RAMdisk in memory for those few times
that I need to archive this folder?!?

Yousuf Khan


Is that folder compressed (compression is supported on partitions
with 4KB cluster size, not for other sizes) ? You can use fsutil
to get ntfsinfo for the partition, if needed.

You could try nfi.exe and focus in on the folder in question.
And see if there's anything strange about the files at the cluster
level. You might have to use web.archive.org for this stuff if
Microsoft has removed it.

http://support.microsoft.com/kb/253066

http://download.microsoft.com/downlo...us/oem3sr2.zip

(Try here instead...)
https://web.archive.org/web/20150329...us/oem3sr2.zip

nfi C: nfi_c.txt

The text file will show where the clusters
for the data are stored, amongst other things. The
one thing the tool could have done, but doesn't, is
annotate the filenames of hardlinked files. The info
simply isn't in there (as hardlinking back in the
Win2K era when the tool was written, would have been
a foreign concept).

If a file had no clusters, maybe it's stored inside
the $MFT entry ? Good for files somewhat smaller
than 1KB in size.

I've run out of ideas as to what is wrong, so I'd be
poking it with various tools to see what's up.

Paul
 




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Upgrade system Taking longer to loadup Windows XP Red Cloud Homebuilt PC's 4 March 1st 11 05:35 PM
Netbackup script to show totals of backed up data? Kyle Tucker Storage & Hardrives 3 April 7th 05 04:44 AM
read from backed up second hard drive after installing new primary master [email protected] Storage (alternative) 8 January 13th 05 06:34 AM
HELP: Backed Up Data Skint4sure Cdr 9 October 12th 04 11:56 PM
Help: Backed up data Skint4sure Homebuilt PC's 2 October 7th 04 11:26 PM


All times are GMT +1. The time now is 08:25 PM.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright ©2004-2024 HardwareBanter.
The comments are property of their posters.