Yousuf Khan wrote:

Okay, so I do daily Macrium file backups of my entire User folder, with
monthly fulls at the beginning of each month. All except one specific
subfolder which takes longer than all of the rest of the User folder
combined to backup. So I've excluded it, and I back it up in a separate
backup job which only runs twice a month, because I can't afford the
time to run that daily.

The User folder stats (minus excluded folder) are this:

Total Number of Files: 342862
Total Size: 913.78 GB
Backup Completed Successfully in 05:02:40

Now the excluded folder stats are this:
Total Number of Files: 651658
Total Size: 1.57 GB
Backup Completed Successfully in 11:47:09

As you can see, an approximately 1 TB folder is fully backed up in about
a mere 5 hours, whereas a puny 1.5 GB folder (almost 600x smaller!) can
only be fully backed up in about 12 hours? The only difference is that
there are about twice as many files in the smaller folder than the
larger folder. Both folders are on the exact same physical drive (HDD,
not SSD). File system is NTFS. What could be causing such a drastic
slowdown?

Disable your anti-virus and redo the backup to see if the AV was getting
in the way. Do you have more than one anti-virus or anti-malware
program installed? Despite their claim (like MalwareBytes saying their
compatible with Avast), that's not true. If you install multiple
anti-virus/malware programs, only one should be active at a time (i.e.,
its real-time scanner is running) while the others should be quiescent.
The others should only be used as on-demand or manual scanners. I've
seen where conflicting AVs would trigger each other to scan the same
file that one had opened for inspection, the other detects the file
access so it scanned the file, and it went back and forth between the
AVs while accumulating thousands of file I/O in the first couple of
minutes all the while slowing the system.

You might also want to disable any anti-exploit programs. Took awhile
to narrow down but I found MalwareBytes AntiExploit was causing disk
slowdowns and instability or improper behavior in some software.
Microsoft's EMET can do the same thing. Malwarebytes used to say MBAE
was compatible with EMET but that wasn't true, either.

You might also want to disable all startup program (use msconfig.exe or
SysInternals' AutoRuns), reboot, and retest the backup to see if it
takes a reasonable time. If so, you have a conflict with one of the
programs you load on starting Windows or when you login to your Windows
account.

In-use files will cause a hang waiting for them to stop being in-use.
Have you tried logging under the Administrator account and then backing
up the C:\Users\youraccount folder?

Is VSC (Volume Shadow Copy) Service aka VSS enabled in your Windows? Go
into the services manager (services.msc) and check "Volume Shadow Copy"
service is set to Manual. That means any program that issues a call to
use that service will start that service. It does not need to have
Automatic startup since it is only needed when a program wants to use
it. You don't want it set to Disabled. Under its Dependencies tab,
check that the services on which it depends are also set to Manual or
Automatic startup.

In a command shell with admin privileges, run:

vssadmin list providers

You should see Microsoft's provider (service) and may see others added
by backup software. I don't have a paid version of Macrium Reflect,
just the free version, so I don't know if the paid versions have Macrium
adding their own VSS provider. Then run:

vssadmin list writers

You're probably not employing all of them but check each has:

State: stable
Last error: No error

If the output is blank, there is a registry problem regarding
permissions preventing vssadmin from reading those registry entries, or
you have a major problem with VSS.

Macrium has its Other Tasks menu under which there is "View Windows VSS
Events". Check if that shows any errors. As I recall, each backup will
not only have a branch showing the backup process but a sub-branch for
VSS where you can look for errors, too.

You would expect that your own %userprofile% profile would have all
privileges assigned to your Windows account. I've run into problems
under the $userprofile\AppData% folder. Apps write under there and
might change permissions so only they can access their data. You can
use AccessEnum from SysInternals to scan a folder and its subfolders to
list. In its default setup, it will show Read, Write, and Deny
privileges on the selected parent (root) folder and all subfolders.
That would be a start to determine if your Windows account (under which
the Macrium job executes) has read permissions to everything under the
root folder you specify in the Macrium backup job. I just ran it and
several folders were listed as "???" for read privilege (and were in the
%temp% folder which is, yep, under the AppData folder) despite I was
logged under my Windows account which is the Administrators security
group. I even saw a couple subfolders with "Access denied" for read
privilege. It would list privileges for the folder (with read and write
privileges) but then sometimes follow with a wildcarded path\* which I
don't know what it is reporting for that syntax. If I right-click on
path and select Properties, that dialog appears. If I right-click on
path\*, an error dialog pops up saying that I don't have permissions.
The Help - Contents dialog is too terse and doesn't mention what
path\* means in the parlance of this tool.

You didn't identify the problematic subfolder. Knowing it might
indicate what might be locking it up. In fact, all you say is "User"
folder. That could include all Windows accounts (C:\Users) instead of
just yours (C:\Users\youraccount). You also did not state if you are
doing an image backup for a logical file backup. I only have the free
version which only saves images. Maybe the paid version also lets you
do file backups.

What level of compression did you select in the Macrium backup job(s)?
The highest compression level will take a LOT longer to complete with
little gain in backup file size.

A couple of options increase the backup job time: verify backup and
verify file system. However, I recommend using both despite the backup
jobs take longer.

What priority was set for the backup job(s)? A low priority means
anything else will get a bigger chunk of the CPU's time.

You did not mention if you are using the free or paid version of Macrium
Reflect. The paid version has additional backup options. One, you can
do incremental backups. You did not specify if your backups are the
same type: full, differential, incremental. Two, there is a backup type
(synthetic) that merges incrementals and differentials into their parent
full (base) backup image. That will take a lot longer to meld those
files together than keeping them separate. From the manual:

When purging Incremental backups, if the backup set only contains a
Full backup followed by Incremental backups, then this option causes
the Full backup to be 'rolled forward' to create a Synthetic Full
backup. This is also known as Incremental Forever.

If you have ever merged a full backup with subsequent differential or
incremental backups, you know those take a while. The synthetic full
backup (merged full + its incrementals) occurs based on your retention
schedule.

And just in case the files in the problematic folder happen to be on a
bad spot on the hard disk (and the only reason why I left intact your
inclusion to cross-post to comp.sys.ibm.pc.hardware.storage since
everything else is a software issue), have you ran "chkdsk c: /r" yet?