[pywin32-bugs] [ pywin32-Bugs-3501250 ] Possible bug in win32file.SetFileTime()

[SourceForge.net <no...@so...>]

Bugs item #3501250, was opened at 2012-03-10 10:34
Message generated for change (Settings changed) made by mhammond
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=551954&aid=3501250&group_id=78018

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: win32
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: http://jamercee.myopenid.com/ ()
Assigned to: Nobody/Anonymous (nobody)
Summary: Possible bug in win32file.SetFileTime()

Initial Comment:
It appears there may be a bug in the win32file.SetFileTime() method (or else there's a big bug in my understanding of how this pywin32 method is intended to work).

We needed the ability to set the ctime,atime,mtime tuple of windows files. The python standard library method os.utime() only works for ctime,mtime. So we turned to pywin32 -- and foudn the SetFileTime seemed to be exactly what we needed.

But it appears win32file.SetFileTime() performs time-zone related transformations that are tripping us up. Specifically, it alters the datetimes it is passed based on the local-system timezone.

As a test, we passed the results from win32file.GetFileTime() win32file.SetFileTime(). In our view, this should have done nothing -- but instead it altered the ctime,atime,mtime of the files by advancing the times. 

I think a quick example can highlight the issue more clearly. Assume the following code is filetest.py:

	import win32file
	hnd = win32file.CreateFile('file.txt',
			win32file.GENERIC_READ | win32file.GENERIC_WRITE,
			0, 
			None, 
			win32file.OPEN_EXISTING, 
			0, 
			None)
	ct, at, mt = win32file.GetFileTime(hnd)
	win32file.SetFileTime(hnd, ct, at, mt)
	hnd.close()

We run it like this:

	C:\> ECHO hi > file.txt
	C:\> DIR file.txt
	03/10/2012  12:55 PM     5 file.txt
               1 File(s)      5 bytes
	
	C:\> PYTHON filetest.py
	C:\> DIR file.txt
	03/10/2012  05:55 PM     5 file.txt
               1 File(s)      5 bytes	

Our local timezone is EST (which today is GMT-5). 

Notice the last-write time is now 5-hours in the future (note: so are ctime, and atime, but for brevity, I've omitted the listing). It looks like SetFileTime() assumed it was passed a collection of times that represented localtime, and performed local -> GMT conversion.

But this is counter to the documentation available with GetFileTime(). According to MSDN, a FILETIME structure (like those returned from GetFileTime()) is:

	"...a 64-bit value representing the number of 
	100-nanosecond intervals since January 1, 1601 (UTC)..."

	http://msdn.microsoft.com/en-us/library/windows/desktop/ms724284(v=vs.85).aspx

After spending some head scratching time on this, we hacked up a quick C version to confirm our understanding was correct about GetFileTime/SetFileTime:

	#include <stdio.h>
	#include <windows.h>
	int main() {
		HANDLE hnd = CreateFile("file.txt",
			GENERIC_READ | GENERIC_WRITE,
			0, 
			NULL, 
			OPEN_EXISTING, 
			0, 
			NULL);
		if (hnd == INVALID_HANDLE_VALUE) {
			printf("error, couldn't open file.txt\n");
			return -1;
			}
		FILETIME ctime, atime, mtime;
		if (GetFileTime(hnd, &ctime, &atime, &mtime) == 0) {
			printf("unable to retrieve filetime(s)");
			return -1;
			}
		if (SetFileTime(hnd, &ctime, &atime, &mtime) == 0) {
			printf("unable to set filetime(s)");
			return -1;
			}
		CloseHandle(hnd);
		return 0;
		}

This program works exactly as we expected and it does not alter any of the the file times.

So we spent some additional time drilling down in the win32file.i swig code.  The SetFileTime() declaration starts on line 718.  On line 740, it begins processing the arguments passed (with code sections for each of the three times).  Here's the one for ctime

	if (!PyWinObject_AsFILETIME(obTimeCreated, &LocalFileTime))
		  return NULL;
	// This sucks!  This code is the only code in pywin32 that
	// blindly converted the result of AsFILETIME to a localtime.
	// That doesn't make sense in a tz-aware datetime world...
	if (PyWinTime_DateTimeCheck(obTimeCreated))
		  TimeCreated = LocalFileTime;
	else
		  LocalFileTimeToFileTime(&LocalFileTime, &TimeCreated);
	lpTimeCreated= &TimeCreated;

(NOTE: we did NOT add the somewhat offensive "sucks" comment. That was left over from whoever edited this code last)

There's quite a bit of moving parts at this level of depth in the pywin32 code -- but it would appear to our superficial understanding that the pywin32 code at this point is converting the time passed to localtime.  Is that correct? 

Has anyone else encountered a problem with win32file.SetFileTime() -- or are we just completely off our nut here?

 
Our test setup:
	Windows 7 (64-bit)
	Running Python 2.7.2 (32-bit)
	pywin32-217
	
The filesystem we are writting to is NTFS



----------------------------------------------------------------------

>Comment By: Mark Hammond (mhammond)
Date: 2012-11-23 22:03

Message:
Build 218 allows a new param to indicate whether the times are UTC - that
defaults to false for b/w compat reasons, but should still allow this
function to be used correctly.

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2012-10-14 21:19

Message:
That sounds like a great compromise!

----------------------------------------------------------------------

Comment By: Roger Upole (rupole)
Date: 2012-10-14 21:00

Message:
This is a leftover from older systems (95 &98) that stored these dates in
local time.  I seem to remember this having been discussed before
somewhere, and the problem with fixing it was that it was already being
used by pre-adjusting the time passed in.  Changing it to no longer convert
to local time will cause problems for anyone already using it that way. 
Also, I think FAT file systems may still use local time even now.

Maybe we could add a flag to do localtime conversion, and defaults to True
so current code continues to work as-is, but new code can use UTC by
passing False.


----------------------------------------------------------------------

Comment By: Michael Vincent (venkman11)
Date: 2012-10-02 14:11

Message:
I'm seeing this behavior also, very annoying.  Maybe the word "possible"
should be removed from the defect title?  It's certainly there.  

My use case for using this functionality as follows:  I'm needing to set
file creation times to their modification time so that media file viewers
which can't extract this metadata from files (e.g. home videos) can get it
from the timestamp.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=551954&aid=3501250&group_id=78018