#478 Inserting special character causes source file deletion

[Steve C]

From within PythonWin editor, inserting a string containing a certain special character (–, as seen in "Char to paste.txt") into a python string object, and then attempting to save the file, causes the source file to be erased completely, from outside the editor.

Using the 2 attached files, copy the character in "Char to paste.txt" to the clipboard. Open "Bug Demo.py" with PythonWin, and paste the character into the second string in the same location as the hyphen in the first string. Save the file.

At the very minimum, this error should be handled much more gracefully. I nearly lost a significant amount of code when this happened initially, and fortunately I chose NOT to reload the (now empty) file in the editor, so I had the opportunity to copy the code.

Environment:
PyWin32 build 214
64-bit Windows 7 Ultimate

[Steve C]

Zip file containing the two file referenced in Details.

[Mark Hammond]

What tool did you use to *copy* the character? The tool used is probably important for me to reliably reproduce this.

[Steve C]

Windows Clipboard.... Initially from Visual Studio 2005, and also occurs when copying from Notepad. I think as long as the correct character gets pasted, it is the process of saving that triggers the issue.

[Roger Upole]

Here's the relevant part of the traceback:
Traceback (most recent call last):
File "J:\Python31\Lib\site-packages\pythonwin\pywin\framework\editor\document.py", line 77, in OnSaveDocument
self.SaveFile(fileName)
File "J:\Python31\Lib\site-packages\pythonwin\pywin\scintilla\document.py", line 54, in SaveFile
ok = view.SaveTextFile(fileName)
File "J:\Python31\Lib\site-packages\pythonwin\pywin\scintilla\view.py", line 394, in SaveTextFile
doc._SaveTextToFile(self, f)
File "J:\Python31\Lib\site-packages\pythonwin\pywin\scintilla\document.py", line 156, in _SaveTextToFile
f.write(s.encode(source_encoding))
UnicodeEncodeError: 'latin-1' codec can't encode character '\u2013' in position 139: ordinal not in range(256)

Looks like we should default the file encoding to something other than latin-1. This would also mean inserting a coding declaration where one doesn't already exist. Alternately, we could catch the error and just pop up a message box saying than an encoding should be specified in the source file. Either way, the actual encoding needs to be attempted before opening the file to avoid losing any existing data.

[Mark Hammond]

Fixed by Roger on 2010/05/06