Menu

Get rid of the Base Language

2008-11-10
2013-03-08
  • karsten-thiemann

    Hi,

    on our german community meeting this weekend we discussed problems of the actual ADempiere language support architecture. The main problem for non-english users is that ADempiere has a hard coded base language which is en_US (american english). Of course you can add other languages as system languages to have a translated user interface and to print out orders/invoices in those languages but there are many differences between the base language and a system language:

    - When you create new data (e.g. product) you enter the fields for the base language (en_us). You have to add translations for the rest of the system languages but there is no way to use ADempiere _without_ english as an enabled language.

    - When you open a window (e.g. product window) all data are shown from the tab's table (M_Product). So the name and description you will see comes from the M_Product table and the language is en_us - no matter what your login language is. There is no way in the actual ADempiere version to have a fully german/french/italian.. user interface that shows only the translated values.

    - You can't search for values in your actual login language - search will always use the fields of the main table (M_Product) and will ignore the _TRL table.

    - The base language is system wide - you can't have two clients with different base languages (even if you manage to switch the hard coded base language..)

    Some of this problems are solved by a contribution from metas that allows you to switch the base language from en_us to another system language but we agreed in our discussion that it would be much better to get rid of the base language and switch to a language neutral architecture.

    The main idea is to save/update/load all fields that are translated directly to/from the *_TRL tables, depending on the actual login language. AND to create _TRL entries for ALL system languages (so also for en_us) - so every language would be treated the same way.

    A nice benefit from this approach would be that we could throw away all views that have also a tranlated view - we just keep the translated views since every language is a translation then..

    We started to create a (quick and dirty) prototype and we managed to show that the concept works. You can see (and try) the code from here:
    http://adempiere.svn.sourceforge.net/viewvc/adempiere/contributions/Localizations/Germany/no_baselanguage/

    It can save/load data in every enabled system language depending on your actual login language, you can search by the login language (only simple search by now) and it will show all the data in you actual login language. That allows us to provide abcomletly translated gui that can be used with different languages and every user will see all data in his/her chosen language.

    There is still a lot of work to do - the provided code is as ugly as the original code which is a real pain in the a** in this area. We need to do some code refacturing here (there are many many getWhereSQL() methods all over the place and so on..). But we think it is worth the effort.

    So - to say it with Teo's words - what do you think?

    Regards,
    Karsten

    P.S. If you want to test the code you have to make some tricks:
    - remove the base language flag for en_us directly in the database
    - Login with spain (MX) as login language as System Administrator
    - navigate to the Language window and run the create missing translations for en_us - now you have all english data in the *_TRL tables and you are ready to go :)

     
    • Trifon (An ADempiere founder)

      Hi Karsten,

      >So - to say it with Teo's words - what do you think?

      Very nice initiative.
      Thank's again to the Germany community for the leading role and willingens to cooperate.

      I'm joining this effort. In fact not me, but it is my customer who require this functionality...

      Kind regards,
      Trifon

       
    • Redhuan D. Oon

      Redhuan D. Oon - 2008-11-15

      I like to make a suggestion. Since this is a big change and we have experimental branch to experiment with, why not put this in this branch for after 4.0 release?

      There we can then have more developers testing out the whole branch as a whole.

      I am there now mostly doing the Query changes which have NOT undergone real testing.

      What do you think?

      red1

       
      • Carlos Ruiz

        Carlos Ruiz - 2008-11-17

        Hi Karsten,

        I think every installation needs a base language.

        You're right this must be defined in a tenant basis - but I'm not sure the right thing to do is dropping the base language.

        I need to put an example to show better my point.

        Currently I have this in my own adempiere:

        base language: en_US (compelled by system)
        system language: es_CO

        I have a product called "Development Services" and a spanish translation for the product called "Servicios de Desarrollo".

        My users can look for the product with both names - but in the window just the base language is shown.

        I support your idea of dropping en_US as the base language - and making it tenant selectable.

        But, I don't agree with the idea of dropping base language.
        My guess is every installation needs a base language (well, every tenant in this case).
        Yes - en_US must be just another language - and if the tenant language is es_CO then this must be the base language.

        But I can't imagine the nightmare trying to support a system where every person looks the product name in a different language.

        I would prefer all my users see "Servicios de Desarollo" as the product name - so every user of my installation talks about the same product.
        But having users that know a product like "Servicios de Desarrollo" and others knowing such product as "Development Services" can make the installation really hard to support - and I think it can make the installation very "terminology-error-prone".

        About dropping _V views and leaving just _VT views!  Great idea.  It will simplify code and will make support easier.

        ---------------------

        I'm not sure about mixing experiments in the same branch - but that's something we must learn.
        There was a discussion about the convenience of migrating to Query class - and we still don't have the "performance-numbers" to support such movement, in fact I read several comments about this is not impacting performance, or it can impact negatively.
        And I also saw some comments about other possibilities to move the code - and about other things we need to discuss before making such change.  But none of those numbers, discussions, thinking have appeared (at least not in forums - and please take account I have still 198 e-mails to read).

        Regards,

        Carlos Ruiz

         
        • Trifon (An ADempiere founder)

          Hi Carlos,

          >But I can't imagine the nightmare trying to support a system where every person
          >looks the product name in a different language.

          Product name is not the most important thing in a system.
          But it is very, very important.
          My customers request every company operating in different language to see product names in it's native language.
          Searching by native product name is very important and users request it.

          Products have one common thing: Value column (Search key). Value column is not translated and should be kept as non translated.
          Users with different languages can use it when refereing to product names in different languages.

          If we look at design of OsCommrece we can find the same functionality.
          Users see and search by products in their native language.

          If we look from Technology point of view product name, description and other transaltable fields do not have any meaning for the program. They are in the program just for human readability.

          Kind regards,
          Trifon

           
          • Carlos Ruiz

            Carlos Ruiz - 2008-11-17

            Thanks Trifon,

            Let's separate things

            Searching by translated name - what is currently supported AFAIR

            Showing translated names in reports and documents - what is currently supported with one problem, base language MUST be en_US - so, I see very useful to allow changing base language on a tenant basis

            Showing translated name on windows searches, etc - as I understand from you and Karsten this is desirable for some installations, and as I pointed it could be undesirable for others - for example I would prefer if every user in the system calls me for support with the same product name.  I'm talking from a IT support point of view, obviously there must be different POV's on different persons and companies.

            So, my guess is that I support totally the second (allowing change of en_US as base language on a tenant basis) - and I think the third could be configurable.
            But maybe as you pointed tables with search key (non-translatable) can solve the problem, but - as we also discussed today via e-mail list - not all tables have search key.

            Regards,

            Carlos Ruiz

             
            • karsten-thiemann

              Hi Carlos,

              I would like to differentiate between the technical base language (having the values in the standard tables instead of _trl tables, using own views) and the usage base language (the base language of a company). I would like to throw away the _technical_ base language and prefer a system without differentiation between languages on the database level. So all translateble values have to be _only_ in *_trl tables (normalization of the db).

              But I think it wouldn't be a problem to have a 'base language' flag in the language table and a systemconfig flag showValuesInBaseLanguage to achieve both behaviours. In fact it should be easy since we have to add the wanted language to the SELECT statement anyway.

              Best regards,
              Karsten

               
    • Colin Rooney

      Colin Rooney - 2008-11-18

      Well even many of those using English are using translations as we need to enter the en_GB or en_AU (or en_IE in my case) to ensure the correct date formats and at times different word usages; vendor or supplier for example.
      So I expect nearly everyone (outside of the US) could benefit from such changes.

      I do have one query, just to be 100% about the new functionality.  Is it still possible to have 1 installation in use by users using different languages? 

      From my own experience with pan-European helpdesks this was always a most tricky issue.  For example, someone logs an issue at a French helpdesk, but it cannot be resolved so it goes to the 2nd line support who are based in Germany.  If this issue is confirmed as a new generic problem it would go into a knowledge base, or something like it, to be dispersed to all front line helpdesks.  So the original Helpdesk worked in French, the 2nd line in Germany and the Knowledge base in a 3rd language.  The only solution I every saw was a translation to a "neutral" language which typically was English since most people had it as a second language... but it depended on the countries involved (if it were Germany, Austria & Switzerland for example it most probably would be German). 

      So I Just wanted to confirm, the application is still multilingual? What is proposed is to allow languages other than en_US to be base?  And the advantage is the search functionality only searches the base language?

      But just like my helpdesk example above must not one language always have to be base? i.e. it is developed in one and translated into the others?

      Perhaps the base language is a distraction here and the real issue is the searching does not take into account the language the user logged in with?

      Besides the question of searching what are the other benefits of a language being the base language?

      As I said to begin with, I too must use a translated version so I am in support of making adempiere as multilingual friendly as possible ... I'm just not sure I understand fully :)

      colin

       
      • karsten-thiemann

        Hi Colin,

        >So I Just wanted to confirm, the application is still multilingual?

        Yes. And the plan is to have a real multilangual and language neutral architecture. My feelings regarding the actual language support solution is that it is a kind of add-on and that the first version was a single language architecture. At least the actual architecture does not support a single language usage if the language isn't american english (and if you plan to use more languages later).

        >What is proposed is to allow languages other than en_US to be base?
        Yes (and no) - The proposal is to have all languages with the same 'rights'. But you (and Carlos) are right, we need the possibility to have one main languge and we need it for the development too. But the implementation would be as a language pack. So when you download and install ADempiere you would have one language (pack) installed (en_us) - but it would be easy to use other languages as the base language.

        >And the advantage is the search functionality only searches the base language?
        Now you can _only_ search in the base language - the search ignores the *_trl tables. Metas did an enhancement to let the user search in his login language but with the actual version you will always see the translateble fields in the base language - no matter which language you login. So you can not have a ADempiere GUI with _only_ german values (product names etc.). This can be what you want (as Carlos said) - but it can also be annoying..

        Other advantages are more on the technical side (no *_V _and_ *_VT, cleaner code..). But I think the main benefit is for the (non english) small business market. If you want to establish a product for a 10-20 person company - customers expect a consistent (german, italian, whatever) UI in their language and they don't want to be forced to enter all names etc. twice. But of course the system should be extensible in an easy way later if additionay languages are needed.

        Regards,
        Karsten

         
        • Carlos Ruiz

          Carlos Ruiz - 2008-11-19

          Hi, I think we have common ground now, but just to be sure let me clarify this:

          > Now you can _only_ search in the base language - the search ignores the *_trl tables

          Just tried in my production DB.  340s+patches.
          I have this product called "Services of implementation" and translated to spanish as "Servicios de implementación"

          I logged in with es_CO language.
          On invoice line, product field, I entered english name and it returned the product and showed me the product in spanish name.
          In other words I entered "Services of implementation" and the product field was filled with "Servicios de implementación"
          I also entered "Servicios de implementación" and it worked again.

          So - I don't understand completely the issue about searching - currently the system is executing searchs with base and translated.

          Please note I have "multilingual documents" enabled in my AD_Client record.

          In fact what I'm talking about potential problem is present currently in the system.  I would prefer a selectable way to show the base product name in field "Services of implementation" instead of showing me the translated name.
          [Now I remember why the support nightmare reference - it's in adempiere :-)   I've spent some time reviewing "bugs" reported for wrongly translated products]

          > customers expect a consistent (german, italian, whatever) UI

          Yes - that's what I'm understanding - we're looking for a way to have a base language that can be different than en_US.  And additionally I'm looking to establish a base language for all users.

          Regards,

          Carlos Ruiz

           
          • Heng Sin

            Heng Sin - 2008-11-19

            Hi Carlos,

            I think the default/base language should be definable at tenant or orgnization level. Organization is a very loose term in Adempiere - the default/base language can be different when Organization is use to represent Entity/Company within a Group.

            Regards,
            Low

             
          • Mark Ostermann

            Mark Ostermann - 2008-11-19

            Hi Carlos,

            > So - I don't understand completely the issue about searching - currently the system is executing searchs with base and translated.

            Yes, that works. But that is not the search mentioned here. Try to search the Product in Product-Info with Baselanguage and Translation. I believe that you'll just be able to get a result for the Baselanguage.

            Greetings from Sankt Augustin
            Mark

             
            • Carlos Ruiz

              Carlos Ruiz - 2008-11-19

              Right Mark, I think if we agree on the goal there's no problem at all - there will be more points in adempiere where we're going to need such translated searches.

              Important thing is that we all agree that Adempiere must be better in language neutrality allowing a base different than en_US.

              Regards,

              Carlos Ruiz

               
              • Teo Sarca

                Teo Sarca - 2008-11-19

                Just for documentation purposes, if when change the base language to de_DE for example, then ADempiere will expect that all application dictionary elements (e.g. AD_Element.Name, AD_Field.Name, AD_Process_Para.Name) to be in base language.
                So in this case you need to update all those elements to have proper translation...

                More, have you considered AD_Client.ISMULTILINGUALDOCUMENT flag ?

                Best regards,
                Teo Sarca - www.arhipac.ro

                 
                • Trifon (An ADempiere founder)

                  Hi Teo,

                  >Just for documentation purposes, if when change the base language to de_DE for example, then >ADempiere will expect that all application dictionary elements (e.g. AD_Element.Name, AD_Field.Name, >AD_Process_Para.Name) to be in base language.
                  >So in this case you need to update all those elements to have proper translation...

                  NO.

                  General idea is to move all Name, Description and human readable column to _Trl tables.
                  That't why name of this initiative is get rid of Base Language.
                  All languages must be equal.

                  Kind regards,
                  Trifon

                   
                  • Redhuan D. Oon

                    Redhuan D. Oon - 2008-11-19

                    Would this mean all present translations have to be upgraded to this new format? If so, we need a simple guide on how to do this, so that all our translators can understand and make the changes.

                    Or a good clear sample.

                    WDYT?

                    red1

                     
                • karsten-thiemann

                  Hi Teo,

                  we already have the AD_Element_TRL, AD_Field_TRL etc. The idea is to get the values from the _trl tables for every language.

                  That's 'all' :) - of course it's not that easy..

                  By the way (red1 asked this in another post), the creation of an en_US language pack is easy - just change the base language flag directly in the db from en_US to another installed language and you can make a language export for en_US.

                  Regards,
                  Karsten

                   
        • Colin Rooney

          Colin Rooney - 2008-11-19

          thanks Karsten.
          Ok so it looks to me like we are all thinking the same way... there is just some discussion (to be sure) of the approach. But since I don't know much about the current translation implementation approach I shall defer to your collective judgment. 

          colin

           
    • Redhuan D. Oon

      Redhuan D. Oon - 2008-11-19

      It would also be nice that when one selects his/her locale, meaning language, all the needed locale settings comes along with it:

      1) Country/Region defaulted to nation of that language.

      2) Taxation settings

      3) Local govt regulationary workflow and report formats

      So perhaps we can make allowance in our design for such in future.

      What Do You Think?

      red1

       
  • Redhuan D. Oon

    Redhuan D. Oon - 2011-11-26

    Trying to test this nice feature here in Germany with local group. Will keep this thread posted of any positive news.

     
  • Ricardo Santana

    Ricardo Santana - 2011-11-30

    Hi Guys,

    Long time since Karsten started this thread.

    I have the same problem with translation. So in my case I did some changes to Adempiere to sync the main record to _Trl according to client language. Let me use an example to explain:

    My Scenario:
    Base Language = New record called xx_XX
    System Languages = en_US and pt_BR (Brazilian Portuguese)
    AD_Client Language (My Client) = pt_BR

    When I create a new Record in M_Product table, two records (pt_BR and en_US) are created automatically to _Trl table. If you change anything at the main record (M_Product table) then the system will update the corresponding record in _Trl table based in your client language, in this case pt_BR.

    Since you cannot login with xx_XX language it became a dummy language, and all translations will work at the same way (_Trl table). Using this approach we can discuss about dropping _V views and keep only _VT.

    Pros:
    - All data in master records (eg. M_Product table) are in the same language of system language
    - All translations are treated in the same way

    Cons:
    - Duplicate information from master record to _Trl

    If someone is interested I can push my code to a branch so it can be tested by you.

    Regards,
    Ricardo Santana
    www.kenos.com.br

     
  • Mark Ostermann

    Mark Ostermann - 2011-12-01

    Hello Ricardo,
    thanks for your informations about how you are solving the ADempiere "Base Language" issue.

    It would be great to get this topic moving again, so I would like to invite you to push your code into a development branch. I'm eager to test, discuss and help pushing it forward. Let's see if the different appraoches can be combined or if we have to decide for one of them.

    WDYT?

    Greetings from Sankt Augustin.
    Mark

     

Log in to post a comment.