winmerge-translate

[Winmerge-translate] Puzzles about po-file-encoding and rc-file-compiling by 2.7.5.2 and 2.7.5.3

[Wei-cheng <wc...@pc...>]

Hi all,

Since 2.7.5.2 the encoding of .po file were changed into utf-8. And
CreateRcFilesFromPoFiles.vbs produces a rc file which contains utf-8
encoded strings. I am not sure that vc6's rc-compiler can support
utf-8. VC6 cannot display the strings in UTF-8 encoded rc files
correctly. Somehow I had an impression that VC6 cannot support unicode.

In order to see the result of translation I used vc6 to compile rc
files (2.7.5.2, 2.7.5.3) through their correspondent
BuildOneLanguage.vbs and compiled MakeResDll.exe. It failed.

So I made some experiments. I converted the encoding of po file from
utf-8 to local one with some converting program, set its encoding to
local one ("charset=..."), generated a rc, and compiled it. It failed
still.

In the same circumstances I can compile rc files of 2.7.4.or 2.7.5.1.
which are encoded with local encoding. And their WinMerge.exe accept
the resulted lang files happily. (I am happy too.) I hope that I
desrcibe the matters clearly enough to let you reproduce the situations.

Wei-cheng

Re: [Winmerge-translate] Puzzles about po-file-encoding and rc-file-compiling by 2.7.5.2 and 2.7.5.3

[Kimmo V. <ki...@wi...>]

Hi,

> Since 2.7.5.2 the encoding of .po file were changed into utf-8. And
> CreateRcFilesFromPoFiles.vbs produces a rc file which contains utf-8
> encoded strings.

This sounds like a bug. The header in PO files says its CP-xxx, so there 
shouldn't be UTF-8 content. Also I think .rc files don't even support UTF-8.

We still support W98 so we can't rely on UTF-8 anyways.

Tim?

Regards,
Kimmo

Re: [Winmerge-translate] Puzzles about po-file-encoding and rc-file-compiling by 2.7.5.2 and 2.7.5.3

[Tim G. <ti...@ge...>]

Kimmo Varis wrote:
>> Since 2.7.5.2 the encoding of .po file were changed into utf-8. And
>> CreateRcFilesFromPoFiles.vbs produces a rc file which contains utf-8
>> encoded strings.
> 
> This sounds like a bug. The header in PO files says its CP-xxx, so there 
> shouldn't be UTF-8 content. Also I think .rc files don't even support UTF-8.
> 
> We still support W98 so we can't rely on UTF-8 anyways.
> 
> Tim?

Sorry for the long silence, but I was on vacation! ;)

I didn't change the CODEPAGE of the PO files. The PO files in SVN looks 
all normal and my editor SciTE <http://scintilla.org/SciTE.html> says 
there are not UTF-8. I have also no problems to compile the PO files 
with MakeResDll.exe. (I use VC6 too!) So I can't reproduce your problem...

How do you get the PO file? Do you use a SVN client like TortoiseSVN 
<http://tortoisesvn.net/>? Or do you download the latest source code 
archives from SF.net? Or use the "SVN Browse" page from SF.net 
<http://winmerge.svn.sourceforge.net/viewvc/winmerge/>? Maybe your 
browser or archive program change the file to UTF-8?

Greetings,
Tim

Re: [Winmerge-translate] Puzzles about po-file-encoding and rc-file-compiling by 2.7.5.2 and 2.7.5.3

[Wei-cheng <wc...@pc...>]

Tim,

Thank you. I feel embarrassed. (Mir ist peinlich.) You are right, the
po is encoded with local encoding. I made a mistake somehow. And after
several days I can not remember what I have done before. So I tried to
reproduce those operations today. And the compiliation problem remains.

I had the following operations:

1. downloaded WinMerge-2.7.5.3-src.7z, unpacked it,
2. updated its ChineseTraditional.po with English.pot, revise some
translations,
3. opened \WinMerge-2.7.5.3-src\Tools\MakeResDll\MakeResDll.dsw to
build MakeResDll.exe. The messages in pane of vc6 show as follows:
--------------------Configuration: MakeResDll - Win32
Debug--------------------
Compiling resources...
Compiling...
StdAfx.cpp
Compiling...
MakeResDll.cpp
c:\program
files\winmerge-src\winmerge-2.7.5.3-src\tools\makeresdll\makeresdll.cpp(1=
87)
: warning C4786:
'std::reverse_iterator<std::basic_string<char,std::char_traits<char>,std:=
:allocator<char>
> const
*,std::basic_string<char,std::char_traits<char>,std::allocator<char>
>,std::basic_string<char,std::char_traits<char>,std::allocator<char> >
const
&,std::basic_string<char,std::char_traits<char>,std::allocator<char> >
const *,int>' :
identifier was truncated to '255' characters in the debug information

c:\program
files\winmerge-src\winmerge-2.7.5.3-src\tools\makeresdll\makeresdll.cpp(1=
87)
: warning C4786:
'std::reverse_iterator<std::basic_string<char,std::char_traits<char>,std:=
:allocator<char>
> *,std::basic_string<char,std::char_traits<char>,std::allocator<char>
>,std::basic_string<char,std::char_traits<char>,std::allocator<char> >
&,std::basic_string<char,std::char_traits<char>,std::allocator<char> >
*,int>' :
identifier was truncated to '255' characters in the debug information

RegKey.cpp
c:\program files\microsoft visual studio\vc98\include\vector(39) :
warning C4786:
'std::vector<std::basic_string<char,std::char_traits<char>,std::allocator=
<char>
>,std::allocator<std::basic_string<char,std::char_traits<char>,std::alloc=
ator<char>
> >
>::vector<std::basic_string<char,std::char_traits<char>,std::allocator<ch=
ar>
>,std::allocator<std::basic_string<char,std::char_traits<char>,std::alloc=
ator<char>
> > >' : identifier was truncated to '255' characters in the debug
information

c:\program files\microsoft visual studio\vc98\include\vector(60) :
warning C4786:
'std::vector<std::basic_string<char,std::char_traits<char>,std::allocator=
<char>
>,std::allocator<std::basic_string<char,std::char_traits<char>,std::alloc=
ator<char>
> >
>::~vector<std::basic_string<char,std::char_traits<char>,std::allocator<c=
har>
>,std::allocator<std::basic_string<char,std::char_traits<char>,std::alloc=
ator<char>
> > >' : identifier was truncated to '255' characters in the debug
information

UnicodeString.cpp
Linking...
LINK : LNK6006: program database C:\Program
Files\WinMerge-src\WinMerge-2.7.5.3-src\BuildTmp\MakeResDll___Win32_Debug=
\MakeResDll.pdb
missing; performing full link

MakeResDll.exe - 0 error(s), 4 warning(s)
(The same compiliation by 2.7.5.1 and 2.7.4 did not show warnings.)

4. copied MakeResDll.exe to \WinMerge-2.7.5.3-src\Src\Languages
5. started CreateRcFilesFromPoFiles.vbs to get MergeChineseTraditional.rc=

6. started BuildOneLanguage.vbs to compile MergeChineseTraditional.rc.
It failed, and the error message is: "Unhandled exception in
MakeResDll.exe in OxC0000005: Access Violation".

ChineseTraditional.po and MergeChineseTraditional.rc are attached.

I have SciTE too. SciTE cannot detect encoding in any situation
automatically. Which encoding it adopts depends in a certain degree on
the user-setting. After choosing encoding the user can see the change
on screen, and determine which encoding a file in question uses. It is
difficult for you to detect local encodings because your PC may not
have the required font. To detect unicode-encoding is in most
situations easier owing to BOM even though lacking the required font.

Wei-cheng


Tim Gerundt =E6=8F=90=E5=88=B0:
> Kimmo Varis wrote:
>>> Since 2.7.5.2 the encoding of .po file were changed into utf-8. And
>>> CreateRcFilesFromPoFiles.vbs produces a rc file which contains utf-8
>>> encoded strings.
>>
>> This sounds like a bug. The header in PO files says its CP-xxx, so
>> there shouldn't be UTF-8 content. Also I think .rc files don't even
>> support UTF-8.
>>
>> We still support W98 so we can't rely on UTF-8 anyways.
>>
>> Tim?
>=20
> Sorry for the long silence, but I was on vacation! ;)
>=20
> I didn't change the CODEPAGE of the PO files. The PO files in SVN looks=

> all normal and my editor SciTE <http://scintilla.org/SciTE.html> says
> there are not UTF-8. I have also no problems to compile the PO files
> with MakeResDll.exe. (I use VC6 too!) So I can't reproduce your problem=
=2E..
>=20
> How do you get the PO file? Do you use a SVN client like TortoiseSVN
> <http://tortoisesvn.net/>? Or do you download the latest source code
> archives from SF.net? Or use the "SVN Browse" page from SF.net
> <http://winmerge.svn.sourceforge.net/viewvc/winmerge/>? Maybe your
> browser or archive program change the file to UTF-8?
>=20
> Greetings,
> Tim
>=20
>=20

Re: [Winmerge-translate] Puzzles about po-file-encoding and rc-file-compiling by 2.7.5.2 and 2.7.5.3

[Kimmo V. <ki...@wi...>]

I'm only commenting to the compile problem.

> I had the following operations:
> 
> 1. downloaded WinMerge-2.7.5.3-src.7z, unpacked it,
> 2. updated its ChineseTraditional.po with English.pot, revise some
> translations,
> 3. opened \WinMerge-2.7.5.3-src\Tools\MakeResDll\MakeResDll.dsw to
> build MakeResDll.exe. The messages in pane of vc6 show as follows:
> --------------------Configuration: MakeResDll - Win32
> Debug--------------------

[..cut..]

> RegKey.cpp
> c:\program files\microsoft visual studio\vc98\include\vector(39) :

This suggests you are not using Platform SDK, as the instructions in 
/Docs/Developers/readme-developers.html suggest. We know from experience 
that there are some surprising differences in headers coming with Visual 
Studio and headers in PSDK. And so we suggest using PSDK.

Can you install PSDK and try again if the crash still happens?

[..cut..]
> (The same compiliation by 2.7.5.1 and 2.7.4 did not show warnings.)

Yes, because MakeResDll.exe code has been changed quite a lot after 
2.7.5.1 - it doesn't anymore use MFC. And well, soon we don't need the 
MakeResDll.exe at all...

> 4. copied MakeResDll.exe to \WinMerge-2.7.5.3-src\Src\Languages
> 5. started CreateRcFilesFromPoFiles.vbs to get MergeChineseTraditional.rc
> 6. started BuildOneLanguage.vbs to compile MergeChineseTraditional.rc.
> It failed, and the error message is: "Unhandled exception in
> MakeResDll.exe in OxC0000005: Access Violation".

Unfortunately this doesn't tell much to us. Only way to find out why it 
crashes is to run it in debugger. And the end result might be just a 
bug/difference in VC6 headers...

Regards,
Kimmo

Re: [Winmerge-translate] Puzzles about po-file-encoding and rc-file-compiling by 2.7.5.2 and 2.7.5.3

[Wei-cheng <wc...@pc...>]

Hi Kimmo,

Thank you for your answer.

As for problem of building .lang I will try your advise, read
readme-developers.html again, and examine in which step I went wrong.
(I had installed psdk before)Recently I have another things to do.

I saw in Tracker/Patches that Jochen is developing a new system for
localization based on the current po-system. Maybe I understand that
not quitely. It seems to be able to generate lang file without
building, rather through replacing strings. If it is so, I expect
that. Because it reduce the "equipments" of translators quite a lot
without losing the speed of loading lang.

Wei-cheng

Kimmo Varis =E6=8F=90=E5=88=B0:
> I'm only commenting to the compile problem.
>=20
>> I had the following operations:
>>
>> 1. downloaded WinMerge-2.7.5.3-src.7z, unpacked it,
>> 2. updated its ChineseTraditional.po with English.pot, revise some
>> translations,
>> 3. opened \WinMerge-2.7.5.3-src\Tools\MakeResDll\MakeResDll.dsw to
>> build MakeResDll.exe. The messages in pane of vc6 show as follows:
>> --------------------Configuration: MakeResDll - Win32
>> Debug--------------------
>=20
> [..cut..]
>=20
>> RegKey.cpp
>> c:\program files\microsoft visual studio\vc98\include\vector(39) :
>=20
> This suggests you are not using Platform SDK, as the instructions in=20
> /Docs/Developers/readme-developers.html suggest. We know from experienc=
e=20
> that there are some surprising differences in headers coming with Visua=
l=20
> Studio and headers in PSDK. And so we suggest using PSDK.
>=20
> Can you install PSDK and try again if the crash still happens?
>=20
> [..cut..]
>> (The same compiliation by 2.7.5.1 and 2.7.4 did not show warnings.)
>=20
> Yes, because MakeResDll.exe code has been changed quite a lot after=20
> 2.7.5.1 - it doesn't anymore use MFC. And well, soon we don't need the =

> MakeResDll.exe at all...
>=20
>> 4. copied MakeResDll.exe to \WinMerge-2.7.5.3-src\Src\Languages
>> 5. started CreateRcFilesFromPoFiles.vbs to get MergeChineseTraditional=
=2Erc
>> 6. started BuildOneLanguage.vbs to compile MergeChineseTraditional.rc.=

>> It failed, and the error message is: "Unhandled exception in
>> MakeResDll.exe in OxC0000005: Access Violation".
>=20
> Unfortunately this doesn't tell much to us. Only way to find out why it=
=20
> crashes is to run it in debugger. And the end result might be just a=20
> bug/difference in VC6 headers...
>=20
> Regards,
> Kimmo
>=20
> -----------------------------------------------------------------------=
--
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.=

> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> Winmerge-translate mailing list
> Win...@li...
> https://lists.sourceforge.net/lists/listinfo/winmerge-translate
>=20
>=20

Re: [Winmerge-translate] Puzzles about po-file-encoding and rc-file-compiling by 2.7.5.2 and 2.7.5.3

[Tim G. <ti...@ge...>]

Kimmo Varis wrote:
> Unfortunately this doesn't tell much to us. Only way to find out why it 
> crashes is to run it in debugger. And the end result might be just a 
> bug/difference in VC6 headers...

Mhh, I try the new MakeResDll version with VC6 too and get the same 
problem like Wei-cheng. With older versions I had no problems and could 
also compile Wei-cheng's "MergeChineseTraditional.rc".

The debugger jumps to the file "%ProgramFiles%\Visual 
Studio\VC98\CRT\SRC\OUTPUT.C" and set the cursor to a while loop ("while 
(i-- && *p)" in line 685):

-------------------------8<-------------------------
#else  /* _UNICODE */
                 if (flags & (FL_LONG|FL_WIDECHAR)) {
                     if (text.wz == NULL) /* NULL passed, use special 
string */
                         text.wz = __wnullstring;
                     bufferiswide = 1;
                     pwch = text.wz;
                     while ( i-- && *pwch )
                         ++pwch;
                     textlen = pwch - text.wz;
                     /* textlen now contains length in wide chars */
                 } else {
                     if (text.sz == NULL) /* NULL passed, use special 
string */
                         text.sz = __nullstring;
                     p = text.sz;
                     while (i-- && *p)
                         ++p;
                     textlen = p - text.sz;    /* length of the string */
                 }

#endif  /* _UNICODE */
------------------------->8-------------------------

Unfortunately this is too much C/C++ for me! ;)

Greetings,
Tim