If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below. |
|
|
Thread Tools | Display Modes |
|
#1
|
|||
|
|||
Why is this folder so slow?
I have a folder on one of my SSD drives that takes 8 to 10 hours to back
up. It is only about 1.4 GB, but it is allocated 2.4 GB of space altogether, and there are 580,000 files here. Indicates that per file it's using up a little bit over half of a cluster on average. File system is NTFS. Meanwhile, this same drive can backup the remainder of the drive in under 2 hours, and the remainder of the drive is 390 GB! Is NTFS this inefficient for small files like this? Yousuf Khan |
#2
|
|||
|
|||
Why is this folder so slow?
Yousuf Khan wrote:
I have a folder on one of my SSD drives that takes 8 to 10 hours to back up. It is only about 1.4 GB, but it is allocated 2.4 GB of space altogether, and there are 580,000 files here. Indicates that per file it's using up a little bit over half of a cluster on average. File system is NTFS. Meanwhile, this same drive can backup the remainder of the drive in under 2 hours, and the remainder of the drive is 390 GB! Is NTFS this inefficient for small files like this? Using WHAT backup software? Doing a file-based or image-based backup? Is it a direct access to the folder, or are you using a redirection, like a junction (reparse point)? Does that folder itself have any redirections which could run the backup program into a loop if it doesn't specifically ignore those? |
#3
|
|||
|
|||
Why is this folder so slow?
VanguardLH wrote in :
Yousuf Khan wrote: I have a folder on one of my SSD drives that takes 8 to 10 hours to back up. It is only about 1.4 GB, but it is allocated 2.4 GB of space altogether, and there are 580,000 files here. Indicates that per file it's using up a little bit over half of a cluster on average. File system is NTFS. Meanwhile, this same drive can backup the remainder of the drive in under 2 hours, and the remainder of the drive is 390 GB! Is NTFS this inefficient for small files like this? Using WHAT backup software? Doing a file-based or image-based backup? Is it a direct access to the folder, or are you using a redirection, like a junction (reparse point)? Does that folder itself have any redirections which could run the backup program into a loop if it doesn't specifically ignore those? That may be it. I remember reading about junctions causing havoc if they were in a backup scheme (I think in a folder/file backup). Amazingly, there was some helpful information (for me, at the time) on a Microsoft forum, about identifying junctions, found in paragraph two of darrenc1's answer. To the OP: https://answers.microsoft.com/en-us/...rum/windows_7- performance/what-is-a-reparse-point-can-anyone-reveal-the/17b9b457-6c8a- 4e83-a445-e603011a6b95 or https://tinyurl.com/y8hssmg6 |
#4
|
|||
|
|||
Why is this folder so slow?
On 4/26/2020 9:32 PM, VanguardLH wrote:
Using WHAT backup software? Doing a file-based or image-based backup? Macrium, file-based. Is it a direct access to the folder, or are you using a redirection, like a junction (reparse point)? Does that folder itself have any redirections which could run the backup program into a loop if it doesn't specifically ignore those? No, none of that. Straightforward unredirected. Yousuf Khan |
#5
|
|||
|
|||
Why is this folder so slow?
Yousuf Khan wrote:
On 4/26/2020 9:32 PM, VanguardLH wrote: Using WHAT backup software? Doing a file-based or image-based backup? Macrium, file-based. Is it a direct access to the folder, or are you using a redirection, like a junction (reparse point)? Does that folder itself have any redirections which could run the backup program into a loop if it doesn't specifically ignore those? No, none of that. Straightforward unredirected. What did you use to check if there were junctions defined within the folder? For example, you could use Nirsoft's NTFSLinksView tool to scan for junctions to list them. You can specify the start folder from where to search, like the folder with the 500K+ files, or search from the root folder of a drive (junctions cannot point to other drives). Alas, if you pick the problematic folder, a scan will only show any junctions in that folder, not those that point at that folder. You might want to scan from the root folder, and then check if that folder is under a junction. Windows has been using junctions for a long time, especially when Microsoft decides to change the name of the special folder, like changing "Documents and Settings", the old name, and "Documents", that both point to C:\Users. Could be your problematic folder is under a junction, like Documents. https://knowledgebase.macrium.com/pa...ageId=23397420 That gives some information. As I recall, Macrium is supposed to ignore symlinks and junctions when creating backups (to prevent looping). That is, it still records the reparse points, but it shouldn't follow them. Make damn sure that Macrium Reflect is *NOT* following reparse points (recording them is okay, but following them during a backup is usually not okay). Go into Reflect under its Other Tasks menu to select Edit Defaults. Under the Backup tab, and under the Reparse Points category, make sure "System - Do not follow" is selected. However, the default for User Reparse Points is to follow them, but I've seen users screw them up and generate circular links. See what happens when you set "User - Do not follow". Those are for the default settings used when you /create/ a backup job. For old saved job definitions, they may differ than the current global defaults. Also go into the backup job's definition and set the reparse follow options the same ("Do not follow" for both system and user defined reparse points). You could run a test by moving or copying the problematic folder to elsewhere that is guaranteed not to be under a junction (after first checking the folder itself has no junctions), like copying the folder to C:\problemfolder, and then having Reflect backup just that folder. Are the files in the problematic folder in use? If open for write, another process has to either wait for the file handle to close (get deleted) or times out. Although I also use Macrium Reflect, configuring it to run pre- and post-job commands is *very* clumsy. You have to create a Powershell, VBscript, or batch file and have Macrium run that as its scheduled task. Once you create the script template, you edit it to add your own commands before or after the backup job. The problem that I've run into is that Reflect will have the script run the backup job by calling Reflect as a service which has admin privileges, but doesn't load the command shell itself with admin privs in which the script runs, so commands you enter there that require admin privs won't run. There might be a way around that, but I gave up on Reflect's clumsy pre- and post-command workaround feature, plus you have to maintain the script instead of having an easily configurable command line to edit in the Reflect GUI when creating or editing a backup job. However, if you can get Reflect's script feature to work to emulate a pre- and post-job feature, you might look at running the SysInternals' handle.exe command to see which files might be in-use (have open file handles) before the backup job starts. Getting locked out from reading a file can be thwarted by using VSS (Volume Shadow Service). I'm pretty sure on image backups that Reflect defaults to using VSS. I don't see an option to not use VSS. However, under Other Tasks menu, Advanced tab, check if Reflect will "Automatically retry without VSS writers on failure". If there is a problem with VSS, Reflect will try to backup without VSS. Also check the VSS service will change into Running status. Go into Windows services (services.msc), scroll down to "Volume Shadow Copy" service. It should be set to Manual startup mode, and not Disabled. It runs when called. It does not stay running during the entire time that Windows is running. It is only needed when a shadow copy is needed to get at in-use or system-restricted files, and you are not backing up the entire time you have Windows loaded. If you go into Event Viewer, Application logs, and filter on event ID 8224, you'll see informational events for "The VSS service is shutting down due to idle timeout." I forget the idle interval, probably 15 minutes, but once started the VSS service will eventually stop after the last time it got called by a VSS requestor and after a VSS writer has completed its task. Those users that whine the VSS has idle-stopped don't understand this service is not meant to be always running (Automatic mode). It is manually called by a requestor, used for a while, and then it stops because it's not being used anymore. Been that way since Microsoft introduced VSS back in Windows XP to facilitate backing up of in-use and system files. https://docs.microsoft.com/en-us/win...w-copy-service Right click on that service and select Start, or select it and click the Start button. Did it change into Running status (for awhile)? Some programs install their own VSS writers. As I recall, Paragon supplied their own optional VSS writer you could select instead of using the Windows-provided one. Reflect uses the copy-on-write writer already provided by Windows. You can see a list of VSS writers by running in a command shell: vssadmin list writers Sorry, I haven't delved far enough into this to know which system VSS writer that Reflect will employ. Might be the ASR Writer as noted at https://docs.microsoft.com/en-us/win...ox-vss-writers. Not sure even Reflect cares, as it likely just issues some system API call to use VSS. VSS is only usable when NTFS is used as the file system. You didn't mention WHERE is the problematic folder. If it is a folder on an internal drive that uses NTFS, VSS can come into play (if the targeted files are locked). If the folder is on some external storage media, like a USB HDD or flash drive, could be that uses FAT32 or some other file system than NTFS, so VSS can't be used there. If VSS fails when called by Macrium Reflect, the backup job's log should note the error. See: https://knowledgebase.macrium.com/di...oft+VSS+errors |
#6
|
|||
|
|||
Why is this folder so slow?
On 4/27/2020 2:03 PM, VanguardLH wrote:
What did you use to check if there were junctions defined within the folder? For example, you could use Nirsoft's NTFSLinksView tool to scan for junctions to list them. You can specify the start folder from where to search, like the folder with the 500K+ files, or search from the root folder of a drive (junctions cannot point to other drives). Alas, if you pick the problematic folder, a scan will only show any junctions in that folder, not those that point at that folder. You might want to scan from the root folder, and then check if that folder is under a junction. Windows has been using junctions for a long time, especially when Microsoft decides to change the name of the special folder, like changing "Documents and Settings", the old name, and "Documents", that both point to C:\Users. Could be your problematic folder is under a junction, like Documents. I don't have to look for junctions, I know where they are. If there were junctions here, I would have put them in myself, otherwise they aren't there. Yousuf Khan |
#7
|
|||
|
|||
Why is this folder so slow?
Yousuf Khan wrote:
I have a folder on one of my SSD drives that takes 8 to 10 hours to back up. It is only about 1.4 GB, but it is allocated 2.4 GB of space altogether, and there are 580,000 files here. Indicates that per file it's using up a little bit over half of a cluster on average. File system is NTFS. Meanwhile, this same drive can backup the remainder of the drive in under 2 hours, and the remainder of the drive is 390 GB! Is NTFS this inefficient for small files like this? Yousuf Khan Have you tried to "defragment" the drive ? Normally, the "optimize" dialog will not offer defragmentation as an option in Windows 10. It's supposed to offer "TRIM" as the option for an SSD. However, there is a "Copy On Write" or COW issue with SSDs. Under the right circumstances, there will be a slowdown. Now, consider what you're doing. Your backup software uses VSS to make a shadow copy. It's possible some "COW activity" is happening during the backup. The Optimize dialog knows about this, and the Optimize dialog has some sort of metric it uses to decide what to do. While most of the time, it will only offer TRIM, I bet in your case, it's "going to have a COW" and defragment your drive. This should not be as thorough as a regular defragment, and the design of what's done, should have something to do with whatever the root cause of "having a COW" is. I've not seen this slow behavior here, so have no first hand experiences to offer on it. Note that over the years Windows 10 has existed, the behavior of the Optimize panel has been "as crazy as Cocoa Puffs". The software frequently could not properly tell an HDD from an SSD, and it would be damn hard to see any "subtle" behaviors, when this software has had so many bugs in the past. I've had a machine full of HDDs offer nothing but TRIM and the Optimize panel declared all my drives as SSD drives. Which is total bull**** and most annoying when you actually want the defrag to work. As far as I can remember, Optimize is working in 1909 OK now. It's been a hell of a bumpy ride though, over the years. See if you're offered a defrag option. Do Properties on the drive letter, and in the Tools tab you'll find the Optimize. Then retest your backup rate after the partition has been cleaned up. Paul |
#8
|
|||
|
|||
Why is this folder so slow?
On 4/26/2020 9:46 PM, Paul wrote:
Have you tried to "defragment" the drive ? No, considering it's an SSD. But as you pointed out later, the optimize option is available for both of my SSD's, but optimize recognizes them as SSD's, so the only optimization available is trimming, no defragging. Normally, the "optimize" dialog will not offer defragmentation as an option in Windows 10. It's supposed to offer "TRIM" as the option for an SSD. However, there is a "Copy On Write" or COW issue with SSDs. Under the right circumstances, there will be a slowdown. Yes, likely this is exactly that circumstance. Do you know what the symptoms of that circumstance are? Now, consider what you're doing. Your backup software uses VSS to make a shadow copy. It's possible some "COW activity" is happening during the backup. Yes, VSS is used by the software, which is Macrium Reflect 6 BTW. Reflect's logs show that it creates the VSS shadows immediately before beginning the backup. This backup runs after midnight, and there is little activity while any of the backups run. All of the backups run after midnight and they finish relatively quickly, except this one. The Optimize dialog knows about this, and the Optimize dialog has some sort of metric it uses to decide what to do. While most of the time, it will only offer TRIM, I bet in your case, it's "going to have a COW" and defragment your drive. This should not be as thorough as a regular defragment, and the design of what's done, should have something to do with whatever the root cause of "having a COW" is. VSS is used on all of the backup jobs. None of the others exhibit this behaviour. In fact, I've experienced this issue for nearly a decade now. The problem started on Windows XP, continued on into Windows 7, and continues to plague me in Windows 10. This particular folder has also been migrated around from HDD to SSD, to a 2nd SSD, etc. So it's not a problem that is specific to HDD's or SSD's, or to any particular version of Windows. I'll tell you what this folder is. It's actually my Thunderbird News folder (exactly what I'm using to ask this question here), which exists under the my User folder structure. The problem was discovered when I started doing daily backups of my User folder and discovered that the User folder was taking forever. After investigating it some, I figured out that the problem was this particular substructure under News. Once I excluded the News folder, backups finished 6 times faster! So I moved the backups of the News folder to their own job, and let the rest of the User folder get backed up separately. Before, you ask, I only backup the News folder once a week, but it's still a pain in the ass watching it take so long even once a week. Some other background. When this particular backup is happening, it's not the drives that are showing as busy, it's the CPU cores! 4 out of the 8 cores on my FX-8300 are fluctuating between 50% to 100% busy, while the other 4 are not that busy. Yousuf Khan |
#9
|
|||
|
|||
Why is this folder so slow?
Yousuf Khan wrote:
Some other background. When this particular backup is happening, it's not the drives that are showing as busy, it's the CPU cores! 4 out of the 8 cores on my FX-8300 are fluctuating between 50% to 100% busy, while the other 4 are not that busy. Yousuf Khan I seem to remember at some time in the past, you offered advice on putting an exception for an AV program, so it does not scan that particular directory (something in Thunderbird). If your CPU cores are railed, I'd be tracing down the PID of the offender. One way to do it on a Pro SKU of OS, is tasklist /svc # should not work on Home and that will tell you what is inside a SVCHOST. You can also do that with Process Explorer from Sysinternals, running concurrently with Task Manager, and flip over to Process Explorer to see what is in a busy PID in Task Manager. If you elevate Process Explorer using "Run as Administrator", it can even take a stack snapshot of a SVCHOST, and you can get additional information. For example, I have a SVCHOST with 15 things in it, and one is wuauserv. If a Windows Update scan is running, that SVCHOST lights up -- but then you have to guess that's the guilty service, as the rest of the services aren't normally a problem. When Macrium is running, CPU effort goes into two things: 1) Running a checksum to stamp the .mrimg when finished. This detects corruption later (like when restoring perhaps). 2) Compression. If the lightweight compressor is turned on, that will use a core. I don't think Macrium uses multi-core for its compressor. If you were seeing more than that, I'd be looking at MsMpEng as a culprit, as it could cause quite a penalty if every small file involved a scan by the Windows Defender. When I ran hashdeep64 in Windows 10, I think the calc ran 8x slower than normal, to give some idea what a penalty Windows Defender causes on reads. Paul |
#10
|
|||
|
|||
Why is this folder so slow?
On 4/27/2020 3:57 AM, Paul wrote:
I seem to remember at some time in the past, you offered advice on putting an exception for an AV program, so it does not scan that particular directory (something in Thunderbird). If your CPU cores are railed, I'd be tracing down the PID of the offender. One way to do it on a Pro SKU of OS, is Â*Â* tasklist /svcÂ*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â*Â* # should not work on Home Not even necessary, I can tell you right now which process is responsible, it's the Macrium Reflect binary. Also the System process which I assume the Reflect binary also makes heavy use of during this time. |
Thread Tools | |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
DH57dd/core i3 slow boot, slow wake up | cew | Intel | 2 | May 20th 12 12:52 AM |
p5n32-sli se deluxe and Vista x64 and 2gb (slow enough) then 4gb (horribly slow issues) | markm75 | Asus Motherboards | 1 | August 26th 07 01:51 PM |
scan to folder | Ricky | Printers | 5 | April 29th 05 02:35 PM |
ATI folder | Rob | Ati Videocards | 3 | January 25th 05 12:04 AM |
Cannot remove folder | sunksnook | Storage (alternative) | 0 | September 15th 03 02:00 AM |