Yousuf Khan wrote:

On 4/26/2020 9:32 PM, VanguardLH wrote:
Using WHAT backup software? Doing a file-based or image-based backup?

Macrium, file-based.

Is it a direct access to the folder, or are you using a redirection,
like a junction (reparse point)? Does that folder itself have any
redirections which could run the backup program into a loop if it
doesn't specifically ignore those?

No, none of that. Straightforward unredirected.

What did you use to check if there were junctions defined within the
folder? For example, you could use Nirsoft's NTFSLinksView tool to scan
for junctions to list them. You can specify the start folder from where
to search, like the folder with the 500K+ files, or search from the root
folder of a drive (junctions cannot point to other drives). Alas, if
you pick the problematic folder, a scan will only show any junctions in
that folder, not those that point at that folder. You might want to
scan from the root folder, and then check if that folder is under a
junction. Windows has been using junctions for a long time, especially
when Microsoft decides to change the name of the special folder, like
changing "Documents and Settings", the old name, and "Documents", that
both point to C:\Users. Could be your problematic folder is under a
junction, like Documents.

https://knowledgebase.macrium.com/pa...ageId=23397420

That gives some information. As I recall, Macrium is supposed to ignore
symlinks and junctions when creating backups (to prevent looping). That
is, it still records the reparse points, but it shouldn't follow them.

Make damn sure that Macrium Reflect is *NOT* following reparse points
(recording them is okay, but following them during a backup is usually
not okay). Go into Reflect under its Other Tasks menu to select Edit
Defaults. Under the Backup tab, and under the Reparse Points category,
make sure "System - Do not follow" is selected. However, the default
for User Reparse Points is to follow them, but I've seen users screw
them up and generate circular links. See what happens when you set
"User - Do not follow". Those are for the default settings used when
you /create/ a backup job. For old saved job definitions, they may
differ than the current global defaults. Also go into the backup job's
definition and set the reparse follow options the same ("Do not follow"
for both system and user defined reparse points).

You could run a test by moving or copying the problematic folder to
elsewhere that is guaranteed not to be under a junction (after first
checking the folder itself has no junctions), like copying the folder to
C:\problemfolder, and then having Reflect backup just that folder.

Are the files in the problematic folder in use? If open for write,
another process has to either wait for the file handle to close (get
deleted) or times out. Although I also use Macrium Reflect, configuring
it to run pre- and post-job commands is *very* clumsy. You have to
create a Powershell, VBscript, or batch file and have Macrium run that
as its scheduled task. Once you create the script template, you edit it
to add your own commands before or after the backup job. The problem
that I've run into is that Reflect will have the script run the backup
job by calling Reflect as a service which has admin privileges, but
doesn't load the command shell itself with admin privs in which the
script runs, so commands you enter there that require admin privs won't
run. There might be a way around that, but I gave up on Reflect's
clumsy pre- and post-command workaround feature, plus you have to
maintain the script instead of having an easily configurable command
line to edit in the Reflect GUI when creating or editing a backup job.
However, if you can get Reflect's script feature to work to emulate a
pre- and post-job feature, you might look at running the SysInternals'
handle.exe command to see which files might be in-use (have open file
handles) before the backup job starts.

Getting locked out from reading a file can be thwarted by using VSS
(Volume Shadow Service). I'm pretty sure on image backups that Reflect
defaults to using VSS. I don't see an option to not use VSS. However,
under Other Tasks menu, Advanced tab, check if Reflect will
"Automatically retry without VSS writers on failure". If there is a
problem with VSS, Reflect will try to backup without VSS.

Also check the VSS service will change into Running status. Go into
Windows services (services.msc), scroll down to "Volume Shadow Copy"
service. It should be set to Manual startup mode, and not Disabled. It
runs when called. It does not stay running during the entire time that
Windows is running. It is only needed when a shadow copy is needed to
get at in-use or system-restricted files, and you are not backing up the
entire time you have Windows loaded. If you go into Event Viewer,
Application logs, and filter on event ID 8224, you'll see informational
events for "The VSS service is shutting down due to idle timeout."

I forget the idle interval, probably 15 minutes, but once started the
VSS service will eventually stop after the last time it got called by a
VSS requestor and after a VSS writer has completed its task. Those
users that whine the VSS has idle-stopped don't understand this service
is not meant to be always running (Automatic mode). It is manually
called by a requestor, used for a while, and then it stops because it's
not being used anymore. Been that way since Microsoft introduced VSS
back in Windows XP to facilitate backing up of in-use and system files.

https://docs.microsoft.com/en-us/win...w-copy-service

Right click on that service and select Start, or select it and click the
Start button. Did it change into Running status (for awhile)?

Some programs install their own VSS writers. As I recall, Paragon
supplied their own optional VSS writer you could select instead of using
the Windows-provided one. Reflect uses the copy-on-write writer already
provided by Windows. You can see a list of VSS writers by running in a
command shell:

vssadmin list writers

Sorry, I haven't delved far enough into this to know which system VSS
writer that Reflect will employ. Might be the ASR Writer as noted at
https://docs.microsoft.com/en-us/win...ox-vss-writers.
Not sure even Reflect cares, as it likely just issues some system API
call to use VSS.

VSS is only usable when NTFS is used as the file system. You didn't
mention WHERE is the problematic folder. If it is a folder on an
internal drive that uses NTFS, VSS can come into play (if the targeted
files are locked). If the folder is on some external storage media,
like a USB HDD or flash drive, could be that uses FAT32 or some other
file system than NTFS, so VSS can't be used there.

If VSS fails when called by Macrium Reflect, the backup job's log should
note the error. See:

https://knowledgebase.macrium.com/di...oft+VSS+errors