Re: [openMSX-devel] Unicode on Windows issue?

[Wouter V. <wou...@sc...>]

On 10/24/2010 12:44 AM, Max Attar Feingold wrote:
> I've reproduced the problem, and checked in a fix.
>
> That said, it's not a fix I particularly like.  I suspect it will not work
> well with true unicode characters instead of characters like é.  However,
> there is no workaround for that until gzopen() is reimplemented to expect
> UTF8 or UTF16.

Thanks a lot for figuring this out and fixing it.

I agree it's not an ideal fix. I did some googling and found other
projects that hit this same problem. Some fix it by using gzdopen()
instead of gzopen(). I've implemented something similar in the patch
below. As a bonus there's no more platform specific code in this
source file (well, it's hidden in FileOperations).

I've tested this code on Linux, and it seems to work fine. Max, can
you give it a try on Windows? I'm not sure about the fileno()
function on windows. Can you also try with non-ansi unicode chars?

Thanks.

Wouter




diff --git a/src/serialize.cc b/src/serialize.cc
index 621216d..71a5105 100644
--- a/src/serialize.cc
+++ b/src/serialize.cc
@@ -10,7 +10,7 @@
  #include "lzo.hh"
  #include "MemBuffer.hh"
  #include "StringOp.hh"
-#include "utf8_checked.hh"
+#include "FileOperations.hh"
  #include <cstring>
  #include <zlib.h>
  #include <limits>
@@ -261,14 +261,19 @@ void MemInputArchive::serialize_blob(const char*, void* data, unsign
  XmlOutputArchive::XmlOutputArchive(const string& filename)
         : current(new XMLElement("serial"))
  {
-#ifdef _WIN32
-       // gzopen wants an ansi-encoded string, not utf8
-       string ansifilename = utf8::utf8toansi(filename);
-       file = gzopen(ansifilename.c_str(), "wb9");
-#else
-       file = gzopen(filename.c_str(),     "wb9");
-#endif
+       // Note: using gzopen() would seem simpler, though internally gzopen()
+       // uses fopen() and on windows fopen() expects a ansi-encoded filename
+       // (on unix fopen() expects UTF8). The 'filename' parameter of this
+       // constructor is UTF8 encoded. Therefor we use gzdopen(), it's very
+       // similar to gzopen(), but instead of a filename it takes an already
+       // opened file as parameter.
+       FILE* f = FileOperations::openFile(filename, "wb");
+       if (!f) {
+               throw XMLException("Could not open compressed file \"" + filename + "\"");
+       }
+       file = gzdopen(fileno(f), "wb9");
         if (!file) {
+               fclose(f);
                 throw XMLException("Could not open compressed file \"" + filename + "\"");
         }
  }