UI translation without compiled lang files
Windows visual diff and merge for files and directories
Brought to you by:
christianlist,
grimmdp
This patch proposes a way to make new UI translation system work without compiled lang files. It reads translated strings directly from .po files.
Alongside with English.pot, CreateMasterPotFile.vbs creates MergeLang.rc, which contains Merge.rc line number references in place of texts. Compiling MergeLang.rc yields MergeLang.dll, which WinMerge loads in place of current lang files. Then, after loading resources from MergeLang.dll, WinMerge knows line number references, which lead it the way to translated strings in .po files.
The patch as is does the job for frame menues and open dialog, just to show how things work and allow for a well-founded decision.
Logged In: YES
user_id=631874
Originator: NO
Wow! I'm speechless...
This looks pretty in code (we need only that one Translate*() call?), and it is elegant and solid in design. I've been hoping we have something like this, but I never thought it could be something this elegant. Wonderful job, I'm all for this patch!
Are there any way we can check PO files contain all the strings? I mean, with this patch people can just start editing PO files and errors happen more easily. Though I'm not even sure if we care, as fallback to English string is always present.. But maybe some warning for translators would be good..
Logged In: YES
user_id=766060
Originator: YES
> Are there any way we can check PO files contain all the strings?
With the English.pot at hand, a .vbs or so could do a sanity check on .po files. (I'm sure Tim will do a good job at that.)
BTW, translating status bar messages isn't terribly hard either. CFrameWnd's virtual GetMessageString() is our friend:
[CODE]
void CMainFrame::GetMessageString(UINT nID, CString& rMessage) const
{
// load appropriate string
const String s = theApp.LoadString(nID);
if (!AfxExtractSubString(rMessage, s.begin(), 0))
{
// not found
TRACE1("Warning: no message line prompt for ID 0x%04X.\n", nID);
}
}
[/CODE]
Most of the effort will go into converting all occurences of LoadString(), including those wrapped into AfxFormatString*() and the like. This shouldn't be one big patch. Too many files involved. But that also means translation system won't be fully operational for some time.
As changes to existing code are quite straightforward, I'd like to first apply this patch as is, and then apply ad-hoc fixes to other files without submitting new patches. I will post about progress in this thread, though.
Logged In: YES
user_id=631874
Originator: NO
Tim worked his system in a separate development branch. Maybe this change should be worked in its own branch also. That way you for example avoid need of merging patches every time we change something in other code. When the code in branch works ok, then merge to trunk can be one commit. And all history is preserved in the branch.
Yes, a bit more work and time, but I think it is after all easier to work in "stable" branch where other patches don't interfere.
Logged In: YES
user_id=766060
Originator: YES
Separate branch is just overkill. I'd rather do an intermediate version of CLanguageSelect class that loads compiled lang files as usual, NOPs for Translate*() methods, and delegates LoadString() to MFC. Then I can do necessary changes in trunk without breaking existing translation system. Finally, we can make quick switch to new system by replacing CLanguageSelect class.
Logged In: YES
user_id=631874
Originator: NO
Ok, that sounds like a good plan.
Generally we can break trunk sometimes for short time, but with pretty fundamental changes like this we should be quite careful. Anyway, your suggestion about how to forward with this in trunk sounds good, go ahead.
Logged In: YES
user_id=766060
Originator: YES
Intermediate version of CLanguageSelect class is in trunk now.
Logged In: YES
user_id=631874
Originator: NO
Please add SVN revision number also if you bother adding comment.. :)
I now see compile errors with VS2003.net and UnicodeDebug build:
g:\WinMerge\WinMerge_SVN\Src\Common\LanguageSelect.cpp(312): error C2440: 'initializing' : cannot convert from 'std::basic_string<_Elem,_Traits,_Ax>::iterator' to 'char *'
with
[
_Elem=char,
_Traits=std::char_traits<char>,
_Ax=std::allocator<char>
]
g:\WinMerge\WinMerge_SVN\Src\Common\LanguageSelect.cpp(360): error C2679: binary '-' : no operator found which takes a right-hand operand of type 'std::basic_string<_Elem,_Traits,_Ax>::iterator' (or there is no acceptable conversion)
with
[
_Elem=char,
_Traits=std::char_traits<char>,
_Ax=std::allocator<char>
]
g:\WinMerge\WinMerge_SVN\Src\Common\LanguageSelect.cpp(848): error C3861: 'RTL_NUMBER_OF': identifier not found, even with argument-dependent lookup
Logged In: YES
user_id=766060
Originator: YES
// $Id: LanguageSelect.cpp 4580 2007-10-04 10:11:09Z jtuc $
VS2003 build should be fixed. (It seems STL strings have been further optimized in VS2003.)
I still wonder how it comes that RTL_NUMBER_OF doesn't work for you...
Logged In: YES
user_id=631874
Originator: NO
Thanks, your commit fixed my build problems.
> I still wonder how it comes that RTL_NUMBER_OF doesn't work for you...
No idea. I'm using header files and libraries that came with VS2003.Net. Maybe you have PSDK versions? I know there are some differences in versions coming with VS and with PSDK...
Logged In: YES
user_id=766060
Originator: YES
> Maybe you have PSDK versions?
Yep, VS2003 headers define these RTL_ macros only if _WIN32_WINNT > 0x0500. This is fixed in February 2003 SDK, which I'm using with both VS6 and VS2003.
Logged In: YES
user_id=766060
Originator: YES
// $Id: OpenDlg.cpp 4583 2007-10-05 09:03:29Z jtuc $
// $Id: MainFrm.cpp 4583 2007-10-05 09:03:29Z jtuc $
// $Id: MainFrm.h 4583 2007-10-05 09:03:29Z jtuc $
// $Id: Merge.cpp 4583 2007-10-05 09:03:29Z jtuc $
// $Id: Merge.h 4583 2007-10-05 09:03:29Z jtuc $
Logged In: YES
user_id=766060
Originator: YES
Modified: svnroot2\trunk\Src\DirActions.cpp
Modified: svnroot2\trunk\Src\DirCmpReportDlg.cpp
Modified: svnroot2\trunk\Src\FileFiltersDlg.cpp
Modified: svnroot2\trunk\Src\FileOrFolderSelect.cpp
Modified: svnroot2\trunk\Src\FileOrFolderSelect.h
Modified: svnroot2\trunk\Src\MainFrm.cpp
Modified: svnroot2\trunk\Src\MergeDoc.cpp
Modified: svnroot2\trunk\Src\PatchDlg.cpp
Modified: svnroot2\trunk\Src\PropArchive.cpp
Modified: svnroot2\trunk\Src\PropBackups.cpp
Modified: svnroot2\trunk\Src\PropCodepage.cpp
Modified: svnroot2\trunk\Src\PropColors.cpp
Modified: svnroot2\trunk\Src\PropColors.h
Modified: svnroot2\trunk\Src\PropCompare.cpp
Modified: svnroot2\trunk\Src\PropEditor.cpp
Modified: svnroot2\trunk\Src\PropGeneral.cpp
Modified: svnroot2\trunk\Src\PropLineFilter.cpp
Modified: svnroot2\trunk\Src\PropRegistry.cpp
Modified: svnroot2\trunk\Src\PropSyntaxColors.cpp
Modified: svnroot2\trunk\Src\PropSyntaxColors.h
Modified: svnroot2\trunk\Src\PropTextColors.cpp
Modified: svnroot2\trunk\Src\PropVss.cpp
Sending content: C:\svnroot2\trunk\Src\PropLineFilter.cpp
Sending content: C:\svnroot2\trunk\Src\FileFiltersDlg.cpp
Sending content: C:\svnroot2\trunk\Src\DirActions.cpp
Sending content: C:\svnroot2\trunk\Src\DirCmpReportDlg.cpp
Sending content: C:\svnroot2\trunk\Src\PropColors.cpp
Sending content: C:\svnroot2\trunk\Src\PropColors.h
Sending content: C:\svnroot2\trunk\Src\PropTextColors.cpp
Sending content: C:\svnroot2\trunk\Src\PropCodepage.cpp
Sending content: C:\svnroot2\trunk\Src\PropRegistry.cpp
Sending content: C:\svnroot2\trunk\Src\PropSyntaxColors.cpp
Sending content: C:\svnroot2\trunk\Src\FileOrFolderSelect.cpp
Sending content: C:\svnroot2\trunk\Src\PropVss.cpp
Sending content: C:\svnroot2\trunk\Src\PropGeneral.cpp
Sending content: C:\svnroot2\trunk\Src\PropSyntaxColors.h
Sending content: C:\svnroot2\trunk\Src\FileOrFolderSelect.h
Sending content: C:\svnroot2\trunk\Src\PropArchive.cpp
Sending content: C:\svnroot2\trunk\Src\MergeDoc.cpp
Sending content: C:\svnroot2\trunk\Src\PatchDlg.cpp
Sending content: C:\svnroot2\trunk\Src\PropEditor.cpp
Sending content: C:\svnroot2\trunk\Src\PropCompare.cpp
Sending content: C:\svnroot2\trunk\Src\PropBackups.cpp
Sending content: C:\svnroot2\trunk\Src\MainFrm.cpp
Completed: At revision: 4588
Modified: svnroot2\trunk\Src\Common\PreferencesDlg.cpp
Sending content: C:\svnroot2\trunk\Src\Common\PreferencesDlg.cpp
Completed: At revision: 4589
Logged In: YES
user_id=766060
Originator: YES
Modified: svnroot2\trunk\Src\Languages\CreateMasterPotFile.vbs
Sending content: C:\svnroot2\trunk\Src\Languages\CreateMasterPotFile.vbs
Completed: At revision: 4590
Logged In: YES
user_id=631874
Originator: NO
You don't need to paste every file versions, they are identical in SVN, just this kind of line:
> Completed: At revision: 4590
is what we need. And why it is good to paste the revision - it helps to find the revision from SVN logs. The log entry then tells what files were changed and so on..
Once you get this patch completed and I've understood how to use the new system, I'll do a next experimental release. There now seems to be few pretty active translators so I'm sure we get immediate testing and feedback this time.
Also, as this is big change to installations (no more LANG files) and how translations work we'll need to document this well. I can help with that.
Logged In: YES
user_id=631874
Originator: NO
Regarding your last commits, I spotted couple of places where you used std::wstring unconditionally. I wonder if that causes problems in ANSI build? As std::wstring maps to wchar, right?
Logged In: YES
user_id=766060
Originator: YES
I was unable to post here yesterday, so this is yesterday's news...
Completed: At revision: 4594
Completed: At revision: 4595
> std::wstring
New LoadDialogCaption() method, which became necessary for translating titles of property pages, uses std::wstring because strings in dialog resource are always wide.
Sample usage in FileFiltersDlg::FileFiltersDlg():
[CODE]
m_strCaption = theApp.LoadDialogCaption(m_lpszTemplateName).c_str();
[/CODE]
In ANSI build, CString::operator=(LPCWSTR) cares about conversion.
Logged In: YES
user_id=766060
Originator: YES
Completed: At revision: 4596
Completed: At revision: 4597
Kimmo, please look at CompareStats::GetCount(). I felt it was safe to remove thread synchronization from that method so it can be declared const, but I may have missed something.
Logged In: YES
user_id=631874
Originator: NO
Yes, I think removing thread synchronization from CompareStats::GetCount() was the right thing to do. Looking at version control history, that code was added in my initial commit for that class. Back then (before 2.4 release) the system was quite a different, and I think even then the GetCount() synchronization was done just to be sure.
Logged In: YES
user_id=652377
Originator: NO
Wow, this sounds really interesting! It would be really easy for translators to test their translations!
The only problem I see at the moment is, that we need a way to update all PO files after change at the Merge.rc. If you for example add or delete lines to/from Merge.rc and don't updates the PO files, you can't find the correct line number reference in the PO files. Or I understood something wrong?
I will look for a way without poEdit and gettext (msgmerge)...
Logged In: YES
user_id=766060
Originator: YES
Yes, line number references would have to be updated in PO files. I have done this manually by WinMerging existing German.po with new English.pot. Feasible but boring. Anyway, doesn't this same issue apply if we stay with compiled lang files, but use PO files as intermediate step for translation?
Logged In: YES
user_id=652377
Originator: NO
> done this manually by WinMerging existing German.po
> with new English.pot.
poEdit <http://www.poedit.net/> has a function to update a PO file from a POT file. ;)
> Anyway, doesn't this same issue apply if we stay with
> compiled lang files, but use PO files as intermediate
> step for translation?
No, since I use the whole English string as key. The "File:Linenumber" is only bonus for translators. In poEdit you can say "Goto reference" and it shows you the line in the file to get the context of the translation.
Could you use the whole English string as key too? If not I will try to write a VBScript to do the update job...
Logged In: YES
user_id=766060
Originator: YES
> No, since I use the whole English string as key.
Then how would you handle strings that translate differently depending on context?
Logged In: YES
user_id=631874
Originator: NO
What do you mean? Strings are constants in resource. I've understood its 1:1 mapping in strings merge.rc file. Every string has its counterpart in po file. If some string is changed in RC file, then it needs to be changed in po file too. Even when there are strings wihh same content "string-data", they all get their own id, so if another changes, it doesn't affect to others.
Or did I completely misunderstood something?
Logged In: YES
user_id=766060
Originator: YES
> Even when there are strings wihh same content "string-data", they all get their own id...
This is what I assumed (and how this patch works). But Tim's post, as I understand it, reads different.