Menu

#2364 UI translation without compiled lang files

Trunk
closed-accepted
nobody
5
2008-01-12
2007-09-29
No

This patch proposes a way to make new UI translation system work without compiled lang files. It reads translated strings directly from .po files.

Alongside with English.pot, CreateMasterPotFile.vbs creates MergeLang.rc, which contains Merge.rc line number references in place of texts. Compiling MergeLang.rc yields MergeLang.dll, which WinMerge loads in place of current lang files. Then, after loading resources from MergeLang.dll, WinMerge knows line number references, which lead it the way to translated strings in .po files.

The patch as is does the job for frame menues and open dialog, just to show how things work and allow for a well-founded decision.

Discussion

1 2 3 4 > >> (Page 1 of 4)
  • Kimmo Varis

    Kimmo Varis - 2007-09-29

    Logged In: YES
    user_id=631874
    Originator: NO

    Wow! I'm speechless...

    This looks pretty in code (we need only that one Translate*() call?), and it is elegant and solid in design. I've been hoping we have something like this, but I never thought it could be something this elegant. Wonderful job, I'm all for this patch!

    Are there any way we can check PO files contain all the strings? I mean, with this patch people can just start editing PO files and errors happen more easily. Though I'm not even sure if we care, as fallback to English string is always present.. But maybe some warning for translators would be good..

     
  • Jochen Tucht

    Jochen Tucht - 2007-09-30

    Logged In: YES
    user_id=766060
    Originator: YES

    > Are there any way we can check PO files contain all the strings?

    With the English.pot at hand, a .vbs or so could do a sanity check on .po files. (I'm sure Tim will do a good job at that.)

    BTW, translating status bar messages isn't terribly hard either. CFrameWnd's virtual GetMessageString() is our friend:

    [CODE]
    void CMainFrame::GetMessageString(UINT nID, CString& rMessage) const
    {
    // load appropriate string
    const String s = theApp.LoadString(nID);
    if (!AfxExtractSubString(rMessage, s.begin(), 0))
    {
    // not found
    TRACE1("Warning: no message line prompt for ID 0x%04X.\n", nID);
    }
    }
    [/CODE]

    Most of the effort will go into converting all occurences of LoadString(), including those wrapped into AfxFormatString*() and the like. This shouldn't be one big patch. Too many files involved. But that also means translation system won't be fully operational for some time.

    As changes to existing code are quite straightforward, I'd like to first apply this patch as is, and then apply ad-hoc fixes to other files without submitting new patches. I will post about progress in this thread, though.

     
  • Kimmo Varis

    Kimmo Varis - 2007-09-30

    Logged In: YES
    user_id=631874
    Originator: NO

    Tim worked his system in a separate development branch. Maybe this change should be worked in its own branch also. That way you for example avoid need of merging patches every time we change something in other code. When the code in branch works ok, then merge to trunk can be one commit. And all history is preserved in the branch.

    Yes, a bit more work and time, but I think it is after all easier to work in "stable" branch where other patches don't interfere.

     
  • Jochen Tucht

    Jochen Tucht - 2007-09-30

    Logged In: YES
    user_id=766060
    Originator: YES

    Separate branch is just overkill. I'd rather do an intermediate version of CLanguageSelect class that loads compiled lang files as usual, NOPs for Translate*() methods, and delegates LoadString() to MFC. Then I can do necessary changes in trunk without breaking existing translation system. Finally, we can make quick switch to new system by replacing CLanguageSelect class.

     
  • Kimmo Varis

    Kimmo Varis - 2007-09-30

    Logged In: YES
    user_id=631874
    Originator: NO

    Ok, that sounds like a good plan.

    Generally we can break trunk sometimes for short time, but with pretty fundamental changes like this we should be quite careful. Anyway, your suggestion about how to forward with this in trunk sounds good, go ahead.

     
  • Jochen Tucht

    Jochen Tucht - 2007-10-03

    Logged In: YES
    user_id=766060
    Originator: YES

    Intermediate version of CLanguageSelect class is in trunk now.

     
  • Kimmo Varis

    Kimmo Varis - 2007-10-03

    Logged In: YES
    user_id=631874
    Originator: NO

    Please add SVN revision number also if you bother adding comment.. :)

    I now see compile errors with VS2003.net and UnicodeDebug build:
    g:\WinMerge\WinMerge_SVN\Src\Common\LanguageSelect.cpp(312): error C2440: 'initializing' : cannot convert from 'std::basic_string<_Elem,_Traits,_Ax>::iterator' to 'char *'
    with
    [
    _Elem=char,
    _Traits=std::char_traits<char>,
    _Ax=std::allocator<char>
    ]
    g:\WinMerge\WinMerge_SVN\Src\Common\LanguageSelect.cpp(360): error C2679: binary '-' : no operator found which takes a right-hand operand of type 'std::basic_string<_Elem,_Traits,_Ax>::iterator' (or there is no acceptable conversion)
    with
    [
    _Elem=char,
    _Traits=std::char_traits<char>,
    _Ax=std::allocator<char>
    ]
    g:\WinMerge\WinMerge_SVN\Src\Common\LanguageSelect.cpp(848): error C3861: 'RTL_NUMBER_OF': identifier not found, even with argument-dependent lookup

     
  • Jochen Tucht

    Jochen Tucht - 2007-10-04

    Logged In: YES
    user_id=766060
    Originator: YES

    // $Id: LanguageSelect.cpp 4580 2007-10-04 10:11:09Z jtuc $

    VS2003 build should be fixed. (It seems STL strings have been further optimized in VS2003.)

    I still wonder how it comes that RTL_NUMBER_OF doesn't work for you...

     
  • Kimmo Varis

    Kimmo Varis - 2007-10-04

    Logged In: YES
    user_id=631874
    Originator: NO

    Thanks, your commit fixed my build problems.

    > I still wonder how it comes that RTL_NUMBER_OF doesn't work for you...
    No idea. I'm using header files and libraries that came with VS2003.Net. Maybe you have PSDK versions? I know there are some differences in versions coming with VS and with PSDK...

     
  • Jochen Tucht

    Jochen Tucht - 2007-10-04

    Logged In: YES
    user_id=766060
    Originator: YES

    > Maybe you have PSDK versions?

    Yep, VS2003 headers define these RTL_ macros only if _WIN32_WINNT > 0x0500. This is fixed in February 2003 SDK, which I'm using with both VS6 and VS2003.

     
  • Jochen Tucht

    Jochen Tucht - 2007-10-05

    Logged In: YES
    user_id=766060
    Originator: YES

    // $Id: OpenDlg.cpp 4583 2007-10-05 09:03:29Z jtuc $
    // $Id: MainFrm.cpp 4583 2007-10-05 09:03:29Z jtuc $
    // $Id: MainFrm.h 4583 2007-10-05 09:03:29Z jtuc $
    // $Id: Merge.cpp 4583 2007-10-05 09:03:29Z jtuc $
    // $Id: Merge.h 4583 2007-10-05 09:03:29Z jtuc $

     
  • Jochen Tucht

    Jochen Tucht - 2007-10-05

    Logged In: YES
    user_id=766060
    Originator: YES

    Modified: svnroot2\trunk\Src\DirActions.cpp
    Modified: svnroot2\trunk\Src\DirCmpReportDlg.cpp
    Modified: svnroot2\trunk\Src\FileFiltersDlg.cpp
    Modified: svnroot2\trunk\Src\FileOrFolderSelect.cpp
    Modified: svnroot2\trunk\Src\FileOrFolderSelect.h
    Modified: svnroot2\trunk\Src\MainFrm.cpp
    Modified: svnroot2\trunk\Src\MergeDoc.cpp
    Modified: svnroot2\trunk\Src\PatchDlg.cpp
    Modified: svnroot2\trunk\Src\PropArchive.cpp
    Modified: svnroot2\trunk\Src\PropBackups.cpp
    Modified: svnroot2\trunk\Src\PropCodepage.cpp
    Modified: svnroot2\trunk\Src\PropColors.cpp
    Modified: svnroot2\trunk\Src\PropColors.h
    Modified: svnroot2\trunk\Src\PropCompare.cpp
    Modified: svnroot2\trunk\Src\PropEditor.cpp
    Modified: svnroot2\trunk\Src\PropGeneral.cpp
    Modified: svnroot2\trunk\Src\PropLineFilter.cpp
    Modified: svnroot2\trunk\Src\PropRegistry.cpp
    Modified: svnroot2\trunk\Src\PropSyntaxColors.cpp
    Modified: svnroot2\trunk\Src\PropSyntaxColors.h
    Modified: svnroot2\trunk\Src\PropTextColors.cpp
    Modified: svnroot2\trunk\Src\PropVss.cpp
    Sending content: C:\svnroot2\trunk\Src\PropLineFilter.cpp
    Sending content: C:\svnroot2\trunk\Src\FileFiltersDlg.cpp
    Sending content: C:\svnroot2\trunk\Src\DirActions.cpp
    Sending content: C:\svnroot2\trunk\Src\DirCmpReportDlg.cpp
    Sending content: C:\svnroot2\trunk\Src\PropColors.cpp
    Sending content: C:\svnroot2\trunk\Src\PropColors.h
    Sending content: C:\svnroot2\trunk\Src\PropTextColors.cpp
    Sending content: C:\svnroot2\trunk\Src\PropCodepage.cpp
    Sending content: C:\svnroot2\trunk\Src\PropRegistry.cpp
    Sending content: C:\svnroot2\trunk\Src\PropSyntaxColors.cpp
    Sending content: C:\svnroot2\trunk\Src\FileOrFolderSelect.cpp
    Sending content: C:\svnroot2\trunk\Src\PropVss.cpp
    Sending content: C:\svnroot2\trunk\Src\PropGeneral.cpp
    Sending content: C:\svnroot2\trunk\Src\PropSyntaxColors.h
    Sending content: C:\svnroot2\trunk\Src\FileOrFolderSelect.h
    Sending content: C:\svnroot2\trunk\Src\PropArchive.cpp
    Sending content: C:\svnroot2\trunk\Src\MergeDoc.cpp
    Sending content: C:\svnroot2\trunk\Src\PatchDlg.cpp
    Sending content: C:\svnroot2\trunk\Src\PropEditor.cpp
    Sending content: C:\svnroot2\trunk\Src\PropCompare.cpp
    Sending content: C:\svnroot2\trunk\Src\PropBackups.cpp
    Sending content: C:\svnroot2\trunk\Src\MainFrm.cpp
    Completed: At revision: 4588

    Modified: svnroot2\trunk\Src\Common\PreferencesDlg.cpp
    Sending content: C:\svnroot2\trunk\Src\Common\PreferencesDlg.cpp
    Completed: At revision: 4589

     
  • Jochen Tucht

    Jochen Tucht - 2007-10-05

    Logged In: YES
    user_id=766060
    Originator: YES

    Modified: svnroot2\trunk\Src\Languages\CreateMasterPotFile.vbs
    Sending content: C:\svnroot2\trunk\Src\Languages\CreateMasterPotFile.vbs
    Completed: At revision: 4590

     
  • Kimmo Varis

    Kimmo Varis - 2007-10-05

    Logged In: YES
    user_id=631874
    Originator: NO

    You don't need to paste every file versions, they are identical in SVN, just this kind of line:
    > Completed: At revision: 4590
    is what we need. And why it is good to paste the revision - it helps to find the revision from SVN logs. The log entry then tells what files were changed and so on..

    Once you get this patch completed and I've understood how to use the new system, I'll do a next experimental release. There now seems to be few pretty active translators so I'm sure we get immediate testing and feedback this time.

    Also, as this is big change to installations (no more LANG files) and how translations work we'll need to document this well. I can help with that.

     
  • Kimmo Varis

    Kimmo Varis - 2007-10-06

    Logged In: YES
    user_id=631874
    Originator: NO

    Regarding your last commits, I spotted couple of places where you used std::wstring unconditionally. I wonder if that causes problems in ANSI build? As std::wstring maps to wchar, right?

     
  • Jochen Tucht

    Jochen Tucht - 2007-10-07

    Logged In: YES
    user_id=766060
    Originator: YES

    I was unable to post here yesterday, so this is yesterday's news...

    Completed: At revision: 4594
    Completed: At revision: 4595

    > std::wstring

    New LoadDialogCaption() method, which became necessary for translating titles of property pages, uses std::wstring because strings in dialog resource are always wide.

    Sample usage in FileFiltersDlg::FileFiltersDlg():

    [CODE]
    m_strCaption = theApp.LoadDialogCaption(m_lpszTemplateName).c_str();
    [/CODE]

    In ANSI build, CString::operator=(LPCWSTR) cares about conversion.

     
  • Jochen Tucht

    Jochen Tucht - 2007-10-07

    Logged In: YES
    user_id=766060
    Originator: YES

    Completed: At revision: 4596
    Completed: At revision: 4597

    Kimmo, please look at CompareStats::GetCount(). I felt it was safe to remove thread synchronization from that method so it can be declared const, but I may have missed something.

     
  • Kimmo Varis

    Kimmo Varis - 2007-10-07

    Logged In: YES
    user_id=631874
    Originator: NO

    Yes, I think removing thread synchronization from CompareStats::GetCount() was the right thing to do. Looking at version control history, that code was added in my initial commit for that class. Back then (before 2.4 release) the system was quite a different, and I think even then the GetCount() synchronization was done just to be sure.

     
  • Tim Gerundt

    Tim Gerundt - 2007-10-08

    Logged In: YES
    user_id=652377
    Originator: NO

    Wow, this sounds really interesting! It would be really easy for translators to test their translations!

    The only problem I see at the moment is, that we need a way to update all PO files after change at the Merge.rc. If you for example add or delete lines to/from Merge.rc and don't updates the PO files, you can't find the correct line number reference in the PO files. Or I understood something wrong?

    I will look for a way without poEdit and gettext (msgmerge)...

     
  • Jochen Tucht

    Jochen Tucht - 2007-10-09

    Logged In: YES
    user_id=766060
    Originator: YES

    Yes, line number references would have to be updated in PO files. I have done this manually by WinMerging existing German.po with new English.pot. Feasible but boring. Anyway, doesn't this same issue apply if we stay with compiled lang files, but use PO files as intermediate step for translation?

     
  • Tim Gerundt

    Tim Gerundt - 2007-10-09

    Logged In: YES
    user_id=652377
    Originator: NO

    > done this manually by WinMerging existing German.po
    > with new English.pot.

    poEdit <http://www.poedit.net/> has a function to update a PO file from a POT file. ;)

    > Anyway, doesn't this same issue apply if we stay with
    > compiled lang files, but use PO files as intermediate
    > step for translation?

    No, since I use the whole English string as key. The "File:Linenumber" is only bonus for translators. In poEdit you can say "Goto reference" and it shows you the line in the file to get the context of the translation.

    Could you use the whole English string as key too? If not I will try to write a VBScript to do the update job...

     
  • Jochen Tucht

    Jochen Tucht - 2007-10-11

    Logged In: YES
    user_id=766060
    Originator: YES

    > No, since I use the whole English string as key.

    Then how would you handle strings that translate differently depending on context?

     
  • Kimmo Varis

    Kimmo Varis - 2007-10-11

    Logged In: YES
    user_id=631874
    Originator: NO

    What do you mean? Strings are constants in resource. I've understood its 1:1 mapping in strings merge.rc file. Every string has its counterpart in po file. If some string is changed in RC file, then it needs to be changed in po file too. Even when there are strings wihh same content "string-data", they all get their own id, so if another changes, it doesn't affect to others.

    Or did I completely misunderstood something?

     
  • Jochen Tucht

    Jochen Tucht - 2007-10-11

    Logged In: YES
    user_id=766060
    Originator: YES

    > Even when there are strings wihh same content "string-data", they all get their own id...

    This is what I assumed (and how this patch works). But Tim's post, as I understand it, reads different.

     
1 2 3 4 > >> (Page 1 of 4)

Log in to post a comment.