bashburn-info

[Bashburn-info] Gettext support

[Anders L. <and...@gm...>]

My reply to Steven was sent only to him. I'm reposting the message here.

> As the changes to bb start to wind down, there are new text strings 
> creeping in and old ones that might want to change. They should be
handled 
> in less of an ad hoc manner; If I change a string now, I'm afraid
it'll 
> just leave the other languages out of sync.
> 
> Which brings up a new topic... The way that we're handling the
different 
> language problems right now works but it feels somewhat unwieldy. We
have 
> strings in lang/English/somefile.lang and then we have copies of the
file 
> in lang/!English/somefile.lang . Some of the files are mostly
translated. 
> (Good job BTW.) Some languages are there in good shape, but if a
string 
> changes, I don't see a system for *knowing* that the other languages
have 
> to be followed up. Also, we have a lot of strings that are duplicated,
and 
> changing one instance begs the questions whether followup can be
managed.
> 
> So here's the $64 question: Does anyone here have any experience with 
> gettext/msgfmt/po files/pot files/ etc? Should we keep going the way
we 
> are? I ask this with no prior experience using pot files.
> 


All experience with po files I have is that I helped translate an
application to Swedish a bunch of years ago. I have no experience in
actually adding support for it though.

I agree with you though Steven. The translation system is sort of a
mess. It works yes but it's kind of awkward to work with (All credit to
me I suppose). I think it would be a good idea to look into using
gettext. I'm sure it would make translators lives a lot easier. 

-- 
Anders Lindén
http://bashburn.sf.net

Re: [Bashburn-info] Gettext support

[Markus K. <mar...@on...>]

Anders Lindén wrote:
> My reply to Steven was sent only to him. I'm reposting the message here.
>
>   
>> As the changes to bb start to wind down, there are new text strings 
>> creeping in and old ones that might want to change. They should be
>>     
> handled 
>   
>> in less of an ad hoc manner; If I change a string now, I'm afraid
>>     
> it'll 
>   
>> just leave the other languages out of sync.
>>
>> Which brings up a new topic... The way that we're handling the
>>     
> different 
>   
>> language problems right now works but it feels somewhat unwieldy. We
>>     
> have 
>   
>> strings in lang/English/somefile.lang and then we have copies of the
>>     
> file 
>   
>> in lang/!English/somefile.lang . Some of the files are mostly
>>     
> translated. 
>   
>> (Good job BTW.) Some languages are there in good shape, but if a
>>     
> string 
>   
>> changes, I don't see a system for *knowing* that the other languages
>>     
> have 
>   
>> to be followed up. Also, we have a lot of strings that are duplicated,
>>     
> and 
>   
>> changing one instance begs the questions whether followup can be
>>     
> managed.
>   
>> So here's the $64 question: Does anyone here have any experience with 
>> gettext/msgfmt/po files/pot files/ etc? Should we keep going the way
>>     
> we 
>   
>> are? I ask this with no prior experience using pot files.
>>
>>     
>
>
> All experience with po files I have is that I helped translate an
> application to Swedish a bunch of years ago. I have no experience in
> actually adding support for it though.
>
> I agree with you though Steven. The translation system is sort of a
> mess. It works yes but it's kind of awkward to work with (All credit to
> me I suppose). I think it would be a good idea to look into using
> gettext. I'm sure it would make translators lives a lot easier. 
>
>   
Hm, I have just began some weeks before to check/playing and learning to 
do gettext for bashburn.
I stoped the work, because I thought it is a great thing,  but one must 
have support for gettext in the lib's of the system.
And I was not shure whether it would be worth for us to do - I thought 
at first the priority to improve bashburn functionality would be more 
interesting for the most.

But I see now the discussion here, which is interesting. What do the 
others think? Should we rely that a bashburn user has/need i18n 
(internationalization) support?

If the most agree to use gettext, i can start working and experimenting 
and to find out a strategy we can best implement gettext in bashburn, if 
it is wished.

Markus

Re: [Bashburn-info] Gettext support

[Steven W. O. <st...@sy...>]

On Wednesday, Sep 10th 2008 at 15:39 -0000, quoth Markus Kollmar:

=>Hm, I have just began some weeks before to check/playing and learning to 
=>do gettext for bashburn.
=>I stoped the work, because I thought it is a great thing,  but one must 
=>have support for gettext in the lib's of the system.
=>And I was not shure whether it would be worth for us to do - I thought 
=>at first the priority to improve bashburn functionality would be more 
=>interesting for the most.
=>
=>But I see now the discussion here, which is interesting. What do the 
=>others think? Should we rely that a bashburn user has/need i18n 
=>(internationalization) support?
=>
=>If the most agree to use gettext, i can start working and experimenting 
=>and to find out a strategy we can best implement gettext in bashburn, if 
=>it is wished.

I took a look at how this might look.

A po file is a single language mapping file.  So you need multiple po 
files, one file per language.  Then using gettext() you can translate from 
English to whatever foreign language you want based on the setlocale() 
language setting. (This means that BBLANG would just go away.)

Note that the po files are generated from (I think) the pot files. IOW, 
the pot files are src code and the po files are the result of a Makefile.

So in your bashcode you might have something like:

  printf $(gettext "Hello, %s, you have %d messages waiting.\n" \
	name msgs)

and then your po file would contain the mapping of this string to the
language specific translations.  For example es.po would contain:

#: ../src/foo.c:244
msgid "Hello, %s, you have %d messages waiting.\n"
msgstr "Hola %s, usted tiene esperar de %d mensajes.\n"

Your question about lib support is a non-issue. Literally every system has 
all of the support needed. The part that I'm not yet comfortable with is 
the idea of whether this is of value in a shell script since these 
utilities were designed originally for C code. My feeling is that this is 
not a problem, but since I've never been wrong before, it's bound to 
happen sometime. ;-)

Markus, since you've actually looked at this before, can you take a stab 
at implementing a HelloWorld.sh?

-- 
Time flies like the wind. Fruit flies like a banana. Stranger things have  .0.
happened but none stranger than this. Does your driver's license say Organ ..0
Donor?Black holes are where God divided by zero. Listen to me! We are all- 000
individuals! What if this weren't a hypothetical question?
steveo at syslang.net

Re: [Bashburn-info] Gettext support

[Markus K. <mar...@on...>]

Steven W. Orr wrote:
> On Wednesday, Sep 10th 2008 at 15:39 -0000, quoth Markus Kollmar:
>
> =>Hm, I have just began some weeks before to check/playing and learning to 
> =>do gettext for bashburn.
> =>I stoped the work, because I thought it is a great thing,  but one must 
> =>have support for gettext in the lib's of the system.
> =>And I was not shure whether it would be worth for us to do - I thought 
> =>at first the priority to improve bashburn functionality would be more 
> =>interesting for the most.
> =>
> =>But I see now the discussion here, which is interesting. What do the 
> =>others think? Should we rely that a bashburn user has/need i18n 
> =>(internationalization) support?
> =>
> =>If the most agree to use gettext, i can start working and experimenting 
> =>and to find out a strategy we can best implement gettext in bashburn, if 
> =>it is wished.
>
> I took a look at how this might look.
>
> A po file is a single language mapping file.  So you need multiple po 
> files, one file per language.  Then using gettext() you can translate from 
> English to whatever foreign language you want based on the setlocale() 
> language setting. (This means that BBLANG would just go away.)
>
> Note that the po files are generated from (I think) the pot files. IOW, 
> the pot files are src code and the po files are the result of a Makefile.
>
> So in your bashcode you might have something like:
>
>   printf $(gettext "Hello, %s, you have %d messages waiting.\n" \
> 	name msgs)
>
> and then your po file would contain the mapping of this string to the
> language specific translations.  For example es.po would contain:
>
> #: ../src/foo.c:244
> msgid "Hello, %s, you have %d messages waiting.\n"
> msgstr "Hola %s, usted tiene esperar de %d mensajes.\n"
>
> Your question about lib support is a non-issue. Literally every system has 
> all of the support needed. The part that I'm not yet comfortable with is 
> the idea of whether this is of value in a shell script since these 
> utilities were designed originally for C code. My feeling is that this is 
> not a problem, but since I've never been wrong before, it's bound to 
> happen sometime. ;-)
>
> Markus, since you've actually looked at this before, can you take a stab 
> at implementing a HelloWorld.sh?
>
>   
Ok, I will try.
Please wait.

Markus

Re: [Bashburn-info] Gettext support

[Steven W. O. <st...@sy...>]

On Wednesday, Sep 10th 2008 at 21:51 -0000, quoth Markus Kollmar:

=>Steven W. Orr wrote:
=>> On Wednesday, Sep 10th 2008 at 15:39 -0000, quoth Markus Kollmar:
=>> 
=>> =>Hm, I have just began some weeks before to check/playing and learning to
=>> =>do gettext for bashburn.
=>> =>I stoped the work, because I thought it is a great thing,  but one must
=>> =>have support for gettext in the lib's of the system.
=>> =>And I was not shure whether it would be worth for us to do - I thought
=>> =>at first the priority to improve bashburn functionality would be more
=>> =>interesting for the most.
=>> =>
=>> =>But I see now the discussion here, which is interesting. What do the
=>> =>others think? Should we rely that a bashburn user has/need i18n
=>> =>(internationalization) support?
=>> =>
=>> =>If the most agree to use gettext, i can start working and experimenting
=>> =>and to find out a strategy we can best implement gettext in bashburn, if
=>> =>it is wished.
=>> 
=>> I took a look at how this might look.
=>> 
=>> A po file is a single language mapping file.  So you need multiple po files,
=>> one file per language.  Then using gettext() you can translate from English
=>> to whatever foreign language you want based on the setlocale() language
=>> setting. (This means that BBLANG would just go away.)
=>> 
=>> Note that the po files are generated from (I think) the pot files. IOW, the
=>> pot files are src code and the po files are the result of a Makefile.
=>> 
=>> So in your bashcode you might have something like:
=>> 
=>>   printf $(gettext "Hello, %s, you have %d messages waiting.\n" \
=>> 	name msgs)
=>> 
=>> and then your po file would contain the mapping of this string to the
=>> language specific translations.  For example es.po would contain:
=>> 
=>> #: ../src/foo.c:244
=>> msgid "Hello, %s, you have %d messages waiting.\n"
=>> msgstr "Hola %s, usted tiene esperar de %d mensajes.\n"
=>> 
=>> Your question about lib support is a non-issue. Literally every system has
=>> all of the support needed. The part that I'm not yet comfortable with is the
=>> idea of whether this is of value in a shell script since these utilities
=>> were designed originally for C code. My feeling is that this is not a
=>> problem, but since I've never been wrong before, it's bound to happen
=>> sometime. ;-)
=>> 
=>> Markus, since you've actually looked at this before, can you take a stab at
=>> implementing a HelloWorld.sh?
=>> 
=>>   
=>Ok, I will try.
=>Please wait.
=>
=>Markus
=>

:-)

I have a few extra notes to add:

po files are a part of the gettext solution for handling 
internationalization (i18n).

They can work quite well. Here's a thumbnail overview.

You create  plain text message files with an extension of .po (This is 
different from what I wrote earlier.)

The content of these looks like this:

msgid "base language message"
msgstr "This is the base language message"

The idea is that the file contains a list of pairs of msgid and msgstr 
values. Typically there is a brief msgid value, which is what you'd use 
within your code for the message lookup, and a full sentence for the 
msgstr value. Note that using this technique makes it easier to actually 
do translations.

You need one po file per language. The files are typically setup in a 
particular language directory structure. Here's the usual setup:

A locale directory that contains one subdirectory per locale. These are 
usually two letter language codes such as en, fr, de, it, ja, zh, es, 
and so forth. Each locale directory then contains a directory name 
LC_MESSAGES, yes in upper case just like that, and within this directory 
goes your po file.

A po file must be pre-processed into an mo file that is binary data. The 
command to do this is:

msgfmt -o messages.mo messages.po

We also have the problem where position information is language dependent.

In one language you might say
"You have 10 items in your shopping cart worth $50 total"
In another language "Your total of $50 is for item count ten"

More succinctly, my grandfather always used to say "Vut iss here going 
on!" :-)

If we were writing C code, we would take advantage of the fact that printf 
allows for a positional notation using (something like) %1$d

#include <stdio.h>
main()
{
    int number1=44;
    int number2=55;
    printf("number1 is %d, and number2 is %d.\n", number1, number2);
    printf("number1 is %2$d, and number2 is %1$d.\n", number2, number1);
} 

>From bash however, we are given an extra obstacle. There are two printf's. 
One is the builtin inside bash:

1519 > printf 'number1 is %2$s, and number2 is %1$s.\n' number2, number1
bash: printf: `$': invalid format character
number1 is 1520 >

The other is the binary that's seperate from the builtin:

1520 > /usr/bin/printf 'number1 is %2$s, and number2 is %1$s.\n' \
		number2  number1
number1 is /usr/bin/printf: %$: invalid directive
1521 >

So we can see that they both don't work. Don't let that bother you. Just 
pretend that the functionality is really there. Worst case I can write a 
printf that does what we want. z.b., 

awk 'BEGIN {printf "number1 is %2$s, and number2 is %1$s.\n", "number2", 
"number1"}'
number1 is number1, and number2 is number2.

-- 
Time flies like the wind. Fruit flies like a banana. Stranger things have  .0.
happened but none stranger than this. Does your driver's license say Organ ..0
Donor?Black holes are where God divided by zero. Listen to me! We are all- 000
individuals! What if this weren't a hypothetical question?
steveo at syslang.net

Re: [Bashburn-info] Gettext support

[Nick W. <ni...@uk...>]

On Wed, 10 Sep 2008 22:30:05 -0400 (EDT)
"Steven W. Orr" <st...@sy...> wrote:

> >From bash however, we are given an extra obstacle. There are two
> >printf's. 
> One is the builtin inside bash:
> 
> 1519 > printf 'number1 is %2$s, and number2 is %1$s.\n' number2,
> number1 bash: printf: `$': invalid format character
> number1 is 1520 >
> 
> The other is the binary that's seperate from the builtin:
> 
> 1520 > /usr/bin/printf 'number1 is %2$s, and number2 is %1$s.\n' \
> 		number2  number1
> number1 is /usr/bin/printf: %$: invalid directive
> 1521 >
> 
> So we can see that they both don't work. Don't let that bother you.
> Just pretend that the functionality is really there. Worst case I can
> write a printf that does what we want. z.b., 
> 
> awk 'BEGIN {printf "number1 is %2$s, and number2 is %1$s.\n",
> "number2", "number1"}'
> number1 is number1, and number2 is
> number2.
> 

I just found this:

http://www.linuxtopia.org/online_books/advanced_bash_scripting_guide/localization.html

I don't know if it helps here.

Nick
-- 
Free Software Foundation Associate Member 5508

Re: [Bashburn-info] Gettext support

[Steven W. O. <st...@sy...>]

On Thursday, Sep 11th 2008 at 03:11 -0000, quoth Nick Warne:

=>On Wed, 10 Sep 2008 22:30:05 -0400 (EDT)
=>"Steven W. Orr" <st...@sy...> wrote:
=>
=>> >From bash however, we are given an extra obstacle. There are two
=>> >printf's. 
=>> One is the builtin inside bash:
=>> 
=>> 1519 > printf 'number1 is %2$s, and number2 is %1$s.\n' number2,
=>> number1 bash: printf: `$': invalid format character
=>> number1 is 1520 >
=>> 
=>> The other is the binary that's seperate from the builtin:
=>> 
=>> 1520 > /usr/bin/printf 'number1 is %2$s, and number2 is %1$s.\n' \
=>> 		number2  number1
=>> number1 is /usr/bin/printf: %$: invalid directive
=>> 1521 >
=>> 
=>> So we can see that they both don't work. Don't let that bother you.
=>> Just pretend that the functionality is really there. Worst case I can
=>> write a printf that does what we want. z.b., 
=>> 
=>> awk 'BEGIN {printf "number1 is %2$s, and number2 is %1$s.\n",
=>> "number2", "number1"}'
=>> number1 is number1, and number2 is
=>> number2.
=>> 
=>
=>I just found this:
=>
=>http://www.linuxtopia.org/online_books/advanced_bash_scripting_guide/localization.html
=>
=>I don't know if it helps here.

Nick, you're so modest! This is officially sexy. :-)

-- 
Time flies like the wind. Fruit flies like a banana. Stranger things have  .0.
happened but none stranger than this. Does your driver's license say Organ ..0
Donor?Black holes are where God divided by zero. Listen to me! We are all- 000
individuals! What if this weren't a hypothetical question?
steveo at syslang.net

Re: [Bashburn-info] Gettext support

[Anders L. <and...@gm...>]

tor 2008-09-11 klockan 10:11 -0400 skrev Steven W. Orr:
> On Thursday, Sep 11th 2008 at 03:11 -0000, quoth Nick Warne:
> 
> =>On Wed, 10 Sep 2008 22:30:05 -0400 (EDT)
> =>"Steven W. Orr" <st...@sy...> wrote:
> =>
> =>> >From bash however, we are given an extra obstacle. There are two
> =>> >printf's. 
> =>> One is the builtin inside bash:
> =>> 
> =>> 1519 > printf 'number1 is %2$s, and number2 is %1$s.\n' number2,
> =>> number1 bash: printf: `$': invalid format character
> =>> number1 is 1520 >
> =>> 
> =>> The other is the binary that's seperate from the builtin:
> =>> 
> =>> 1520 > /usr/bin/printf 'number1 is %2$s, and number2 is %1$s.\n' \
> =>> 		number2  number1
> =>> number1 is /usr/bin/printf: %$: invalid directive
> =>> 1521 >
> =>> 
> =>> So we can see that they both don't work. Don't let that bother you.
> =>> Just pretend that the functionality is really there. Worst case I can
> =>> write a printf that does what we want. z.b., 
> =>> 
> =>> awk 'BEGIN {printf "number1 is %2$s, and number2 is %1$s.\n",
> =>> "number2", "number1"}'
> =>> number1 is number1, and number2 is
> =>> number2.
> =>> 
> =>
> =>I just found this:
> =>
> =>http://www.linuxtopia.org/online_books/advanced_bash_scripting_guide/localization.html
> =>
> =>I don't know if it helps here.
> 
> Nick, you're so modest! This is officially sexy. :-)
> 
 I agree. This is awesome. 

A question for you guys. Do you think we should release 3.0 before or
after we have gettextified BashBurn? I'm thinking that this translation
might take a while and so it will be quite some time before a new
version is released. 
On the other hand we want 3.0 to be as good as it can get, and this new
translation system would make our baby kick ass.

So what do you think? Stabilize what we have now and wait with the new
translation system for 3.1, or wait with a release until the translation
system is in place and then release 3.0? 

I'm leaning towards waiting but I'd like to hear your opinions as well.

-- 
Anders Lindén
http://bashburn.sf.net

Re: [Bashburn-info] Gettext support

[Nick W. <ni...@uk...>]

On Thu, 11 Sep 2008 22:40:37 +0200
Anders Lindén <and...@gm...> wrote:


> A question for you guys. Do you think we should release 3.0 before or
> after we have gettextified BashBurn? I'm thinking that this
> translation might take a while and so it will be quite some time
> before a new version is released. 
> On the other hand we want 3.0 to be as good as it can get, and this
> new translation system would make our baby kick ass.
> 
> So what do you think? Stabilize what we have now and wait with the new
> translation system for 3.1, or wait with a release until the
> translation system is in place and then release 3.0? 
> 
> I'm leaning towards waiting but I'd like to hear your opinions as
> well.

I concur - a good functional release of the new code FIRST.  Then let
it bed in a bit to ensure no issues.

Then take the time to get the gettext stuff done. 

BUT, we still need to test, test, test and test what we have here.  I
can't reiterate enough, nobody seems to test if BB still works?

Nick
-- 
Free Software Foundation Associate Member 5508

Re: [Bashburn-info] Gettext support

[Markus K. <mar...@on...>]

Anders Lindén wrote:
> tor 2008-09-11 klockan 10:11 -0400 skrev Steven W. Orr:
>   
>> On Thursday, Sep 11th 2008 at 03:11 -0000, quoth Nick Warne:
>>
>> =>On Wed, 10 Sep 2008 22:30:05 -0400 (EDT)
>> =>"Steven W. Orr" <st...@sy...> wrote:
>> =>
>> =>> >From bash however, we are given an extra obstacle. There are two
>> =>> >printf's. 
>> =>> One is the builtin inside bash:
>> =>> 
>> =>> 1519 > printf 'number1 is %2$s, and number2 is %1$s.\n' number2,
>> =>> number1 bash: printf: `$': invalid format character
>> =>> number1 is 1520 >
>> =>> 
>> =>> The other is the binary that's seperate from the builtin:
>> =>> 
>> =>> 1520 > /usr/bin/printf 'number1 is %2$s, and number2 is %1$s.\n' \
>> =>> 		number2  number1
>> =>> number1 is /usr/bin/printf: %$: invalid directive
>> =>> 1521 >
>> =>> 
>> =>> So we can see that they both don't work. Don't let that bother you.
>> =>> Just pretend that the functionality is really there. Worst case I can
>> =>> write a printf that does what we want. z.b., 
>> =>> 
>> =>> awk 'BEGIN {printf "number1 is %2$s, and number2 is %1$s.\n",
>> =>> "number2", "number1"}'
>> =>> number1 is number1, and number2 is
>> =>> number2.
>> =>> 
>> =>
>> =>I just found this:
>> =>
>> =>http://www.linuxtopia.org/online_books/advanced_bash_scripting_guide/localization.html
>> =>
>> =>I don't know if it helps here.
>>
>> Nick, you're so modest! This is officially sexy. :-)
>>
>>     
>  I agree. This is awesome. 
>
> A question for you guys. Do you think we should release 3.0 before or
> after we have gettextified BashBurn? I'm thinking that this translation
> might take a while and so it will be quite some time before a new
> version is released. 
> On the other hand we want 3.0 to be as good as it can get, and this new
> translation system would make our baby kick ass.
>
> So what do you think? Stabilize what we have now and wait with the new
> translation system for 3.1, or wait with a release until the translation
> system is in place and then release 3.0? 
>
> I'm leaning towards waiting but I'd like to hear your opinions as well.
>
>   
Yes Nick, this is better then every "hello_world" or "Hallo_Welt" script 
I could make (because even the gettext maintainer  gives his comments).  ;-)

This is a good question Anders.
I personally think (like Nick), we should improve bashburn with the old 
translation system to get a working system.
There are many things that have to be done - even after the great work 
that already has been done.
Then like Nick said, we need time to test and fix errors. After that we 
could release "bashburn 3.0".

And version 3.1 or so could be the one with gettext (since we and the 
translators first must be familiar with the gettext system).

gettext short summary:
Yes the gettext system for bash has mainly 3 steps.

The existing SOURCE_FILE (bashburn) is source for:

1. SOURCE.POT (bashburn)
2. SOURCE.PO (german), SOURCE.PO (lang_xy), ...
3. SOURCE.MO (german), SOURCE.MO (lang_xy), ...

I' m currently working on a simple makefile/infrastructure where we can 
generate this files automatically (e.g. by calling "make mo")  for a 
little gettext translation system.
But I think it is not usefull to check it into the current bashburn 
branch in. Is this the moment we should create a new branch or so, for 
the bashburn version 3.1?
So one part can work on the current version 3.0, and parallel we have 
the future version 3.1 with gettext support.


Markus