#42 RamDisk's Sync at Shutdown starting before Startup Copy (Hard disk to RamDisk) is finished

[Victor]

Today i've found that one of the sub-folders in my hard disk folder that i am using to Sync with RamDisk is missing.
I am using RamDisk with Synchronize at System Shutdown + Copy only files with Archive attribute + Delete data removed from RamDisk.
So i am wondering is it possible for Sync at Shutdown process to start before Startup process that copies files from Hard disk folder to RamDisk is finished ?
For example this can be possible when Windows just started and you immediately restart it (i did it few times right before my sub-folder gone), especially with a lot of files/folders to copy/sync between Hard disk and RamDisk.
Is there some kind of guard in Sync at Shutdown process that checks for Startup process still copying files from Hard disk to RamDisk ?
Because if Startup process is not finished copying files to RamDisk then Sync process can actually delete files/folders that is not yet copied to RamDisk.

[v77]

Well, yes, theoretically this could happen if you try to shutdown the system very early.
Of course, this is a major issue. It will be fixed in the next release.
Thanks for the report.

For now, the only workaround would be to check whether the RamDiskUI.exe process is still running before shutting down the system. This can be done in a script with a command such as
tasklist | find "RamDiskUI.exe" || shutdown
(the use of "shutdown" depends on whether you have administrative privileges or not)

[Victor]

Thanks for fast response and confirmation.
I briefly checked the source code of RamDiskUI.c and noticed that you are using xcopy/attrib command-line tools for copying files/setting attributes.
If you are going to use the presence of RamDiskUI.exe process as a guard for the Sync at Shutdown process, then make sure RamDiskUI.exe waits till those xcopy/attrib processes are completely finished.
Otherwise it is possible that RamDiskUI.exe will be closed because of Shutdown while those processes still doing their work.

[v77]

RamDiskUI waits for xcopy, not for attrib. It's not an issue because attrib does not involve a risk of data deletion.
The shutdown process can kill xcopy, and even RamDiskUI because, even if it acts as a service, it is not currently configured to delay the shutdown.
Don't worry, I will find something.

[Victor]

Another solution you can use:
Startup process, before starting to do its stuff (creating RamDisk, copying files, etc) can set some flag in registry that means "Job started but not yet finished".
When it completely finished its work it should change the flag's state to "Job finished".
This way even if Startup process will be prematurely terminated by Shutdown, Sync process can read this flag and act accordingly.

[v77]

Yes, I had this idea. :)
But if RamDiskUI is stopped, it will never set this flag to "finished".
Even if I prevent the shutdown like in the synchronization service, there is still the risk to have xcopy killed. It will not be simple...

[Victor]

But if RamDiskUI is stopped, it will never set this flag to "finished".

That's, actually, the whole point of using this flag.
That means early shutdown terminated your Startup process before it finished it's job and now Sync process can know about it.
Flag set to "NOT finished" means RamDisk folder's state is inconsistent, and you shouldn't Sync data with inconsistent state.
Eventually, on proper Windows startup without early shutdown, it will set this flag to "finished" anyway.

Even if I prevent the shutdown like in the synchronization service, there is still the risk to have xcopy killed

You don't need to prevent shutdown in your Startup process.
If shutdown kills your Startup process first, then flag will stay in "NOT finished" state anyway.
If shutdown kills xcopy/attrib process before your Startup process then you should check exit codes for those processes:

WaitForSingleObject(pi.hProcess, INFINITE);
DWORD exitCode;
GetExitCodeProcess(pi.hProcess, &exitCode);

DWORD exitCode for xcopy should be zero. It means successful completion. Non-zero usually means that something gone wrong.
It is not a standard, just general convention, application can specify any exit code it wants.
But for xcopy this is true, it's exit codes listed here.
Just tested it myself by manually killing xcopy and got exit code = 1.
So now if you got exitCode=0 then you can proceed further and when all work completed set registry flag to "finished".
I am not sure, though, what you should do if exitCode is non-zero.
Because it can be non-zero, actually, for some other reasons besides being prematurely killed.

[v77]

The exit code can be set to anything if the process is killed. So we cannot rely on this.
Maybe with the return value of WaitForSingleObject, but I find no documentation about that.

Anyway, the next release will not be right away, because I am in the middle of an important change in RamDiskUI.

[Victor]

The exit code can be set to anything if the process is killed

It is not a random thing, though.
In this case exit code will be passed as parameter to TerminateProcess(HANDLE hProcess, UINT uExitCode) function by another process that want to kill your process.
Windows in this case sets ExitCode=1. Yes, you can write program that kills processes and sets zero as ExitCode, but i don't see why anyone want to do this.
And chances that your process will be killed by such weird program is almost nonexistent.
IMHO such a rare and weird case can be disregarded and you can, actually, rely on exit code.
BTW I found another use case that can lead to data deletion in Hard disk folder by Sync process when RamDisk size < Hard disk folder size.
For example, RamDisk: 100MB, Hard disk folder: 300MB, on startup xcopy copies 100MB and returns ExitCode=4 (Insufficient disk space), on shutdown Sync process will delete 200MB from Hard disk folder. Oops!
So, you should check those exit codes.
If you are going to use the proposed solution with registry flag ("finished"/"NOT finished"), then i can suggest the following behavior:
Set this flag to "NOT finished" at the very beginning of your Startup process, because it can be killed at any time by early shutdown.
If your Startup process finished all steps successfully and xcopy exit code is zero then set flag to "finished".
If xcopy exit code is non-zero then leave this flag with "NOT finished" state.
When Sync is about to start check this flag. If it is in "NOT finished" state don't delete anything with Sync. If state is "finished" do Sync as usual.

[v77]

But if exit code is 1, it is not supposed to be an issue because a user can want to copy only empty folders.
And there is no documentation about the value 1 returned in case where the xcopy process is terminated by Windows.
We need something more robust than that.
In fact, the use of xcopy is not a good idea... But when I wrote that, the synchronization service did not exist.

[Victor]

Yes, that is another good solution. Just copy all files/folders with your own code and don't depend on xcopy and it's exit codes. But still you should somehow deal with the possibility that your process can be killed in the middle of copying files.

[v77]

I said that xcopy can return 1 in case of empty folders, but in fact it's only what the documentation says. I just checked and the return value is 0.
And like you, 4 with insufficient space, and 1 if it is killed by the Task Manager.
In case where the process is killed during shutdown, it seems someone already looked at that.

So it seems that we can rely on the exit code. The rest will be easy.