[eGroupWare-translation] New translation tools roadmap and changes.

[<om...@es...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi, all.


As some of you may know, I'm working in (what I consider) a lot of
improvements for translation tools. I didn't formally request to take
over the maintenance of it because I'm still learning egw internals

These changes will be available in the development (aka trunk or HEAD)
branch as soon a core developer places some necessary changes in the api
that I've requested, and I would update the rest of the changes for the
new translation tools to work. Don't expect it to be available in any
1.2.x release (because it's a _stable_ release, aimed to fix bugs only,
not to include new features or major changes).

There isn't a _100%_ clean and accurate info about the changes and some
things into my mind, but I have what I think is a "good" draft with the
status of the written features and, at least, more expected changes. I
think this draft covers about 90% of the data, so I think it's very good
for a first approach of the idea. Here's a copy & paste of it:



=3D=3D=3D=3D=3D=3D=3D=3D BEGINNING OF PASTE =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D


Enhancements for translations tools 1.3
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

This new rework of translation tools has as main core the rewrite of the
table structure that handles translations, being the main point to
reduce the impact in the db by assigning numeric ids instead of long and
"expensive" varchars up to 128 bytes length. The new table structure of
the affected tables is this:

		'egw_lang_en' =3D> array(
						'fd' =3D> array(
										'content_id' =3D> array('type' =3D> 'auto','precision' =3D>
'11','nullable' =3D> False),
										'app_name' =3D> array('type' =3D> 'varchar','precision' =3D>
'32','nullable' =3D> False),
										'parent_app_name' =3D> array('type' =3D> 'varchar','precision' =
=3D>
'32','nullable' =3D> False),
										'message_id' =3D> array('type' =3D> 'varchar','precision' =3D>
'128','nullable' =3D> False)
						),
						'pk' =3D> array('content_id'),
						'fk' =3D> array(),
						'ix' =3D> array(),
						'uc' =3D> array(array('app_name','message_id'))
		)
		'egw_lang' =3D> array(
						'fd' =3D> array(
										'lang' =3D> array('type' =3D> 'varchar','precision' =3D>
'5','nullable' =3D> False,'default' =3D> ''),
										'content_id' =3D> array('type' =3D> 'int','precision' =3D>
'11','nullable' =3D> False),
										'content' =3D> array('type' =3D> 'text','nullable' =3D> False)
						),
						'pk' =3D> array('lang','content_id'),
						'fk' =3D> array(),
						'ix' =3D> array(),
						'uc' =3D> array()
		),

The egw_lang_en table uses the numeric content_id field to assign
numeric ids to every message_id. The fields are:
 - parent_app_name: the directory where the file is originally read
 - "app_name": exactly the same, being one of $parent_app_name, 'admin',
'common', 'preferences' or 'login'. Maybe all values not being
$parent_app_name could be handled by using only 'common', but that is
not crucial at this time. It is taken from the second field in the
phpgw_en.lang file.
 - message_id: exactly the same, a string up to MAX_MESSAGE_ID_LENGTH
characters (currently 128) that contains the 'entity' to be translated,
taken from the first field in the phpgw_en.lang file. The current role
is to initially identify the source string, and assign a numeric id to
that string inside the parent_app_name (there could be two uses or
contexts of the same message_id in the different parent_app_name, but
NOT in the same app_name -i.e. 'common'-).
 - content_id: the main philosophy behind is to have referencial
integrity in the lang table. A translated string is now identified in
the db by the content_id (currently automatically assigned by the db
engine when inserting a message_id), the lang and the translation itself
('content' fieldname), avoiding the previous schema of "uncontrolled"
and accumulated strings, because indexing several columns doesn't
prevent unused strings, along with unnecesary long rows (and there are
more than 7000 message_id, or strings per language). This field in the
language table uses the same field in the id table as foreign key.

When the new table structure is implemented in head (1.3.x), then it
will be possible to port this rework to public (doing it right now would
break the current translation tools). Also, this needs to work in
conjunction with the translation class, that is affected by these changes=
.

Below is my working (draft?) to keep track, together with the status
between parenthesis.

New features
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
- - (TESTING) Enhancement of the display of the "Application: <app>" bein=
g
translated. It's usual to "forget" the app being translated, and
sometimes there's no easy way to find out. The current fix is enhancing
by using an <h3> title, and duplicate it at the bottom of the strings.
- - Add comment support for phrases (developers, please use this by givin=
g
hints about context to help translators!!):
  - (FIXED) Parsing and writing comment line(s) in egw_en.lang file
  - UI for editing comment (FIXED - using javascript prompt() function).
TODO: replace current javascript prompt() with a subform, like calendar
entries.
  - (FIXED) Show the comment in the UI (useful for devs and translators)
  - (TODO) Use ACL to allow edit comments and write english files
- - (TESTING) Rename or rewrite of existing functions for better
comprension / management. Also, get rid of outdated, unused or
duplicated functions, tags, etc from developer tools:
	- (TESTING)A wrong function in phpgwapi/inc/common_functions.inc.php
uses the phpgw_lang table, but it seems it's not used.
	- (DONE) Rename 'Save', 'Update' and button names in the edit app screen=
.
	- (DONE) Take a lot of code in uilangfile.edit to bolangfile, because
there are no UI functions.
- - (DONE) Reduce source language list to installed langs. This fixes the
load of source language.
- - (TESTING) Improve statistics module in site manager
- - (DONE) New phrases added at top of the page rather than at alphabetic
location. When writing, they're properly sorted. Test comments.
- - (DONE) Prevent file overwrites of the lang file if the content doesn'=
t
differ
- - (DONE) Update db button checked by default
- - (FIXED) Allow searches and uploads to be translated before being adde=
d
to the whole translation
- - Take some screens to popup (as calendar entries). The target is to no=
t
lose the edit screen, always useful:
	- Add phrase
	- Search missing phrases
	- Upload file
- - (FIXED) Show times langs take to be installed when enabling extra
messages. This affects setup/lang.php.
- - (DONE) Add upload feature
- - (DONE) Show in the main screen the status of every app (graph just
like statistics)



Intermediate tasks to get the desired final result, ordered by theorical
priority:

High
=3D=3D=3D=3D
- - (FIXED) Use the new table schema, along with all the necessary files
for installation or upgrade. The new table name containing the
message_id's is "egw_lang_en", so if you think of another table name,
the sooner, the better :).
- - (DONE) Include the content_id field, (FIXED)trying to use it
- - (FIXED) Find a way to link 'common' with the source app. This has bee=
n
done by adding a 'parent_app_name' field in the "egw_lang_en" table,
containg the app (EGW_ROOT/app/phpgw_en.lang) the message_id belongs to.
Again, the sooner the better for changing the field name. This small
field is very useful to improve searches logic.
- - (TESTING) Check for the langfiles to be writable at every load,
warning about it
- - (LATER) Make install or uninstall the selected apps only in /setup,
instead of reinstall all apps
- - (TESTING) Make sure every parent_app_name has itself as
parent_app_name and common as app_name, removing from phpgwapi (or any
other)


Medium
=3D=3D=3D=3D=3D=3D
- - Add admin options:
   - Type of language load:
      * Full load (default, merge source file + db + search phrases,
syncing all phrases)
      * Source + db (no search phrases). Current default.
      * Db only
      * Source only
   - Use ACL. Default not write english file
   - Try to make the 'search new phrases' useless by performing it at
load time.
- - (FIXED) Sync the add and delete actions with all languages installed
- - Search for global/per app untranslated phrases
- - Integrate paging. Get the globalpreference for the number of lines.
- - Separate sub-apps like htmlarea, jscalendar and others. The best mayb=
e
a subdir in setup with the sub-app name itself.


Low
=3D=3D=3D

- - Edit of global missing strings in the same screen (with a threshold)
- - (TESTING)Walk to egw_$lang.lang naming convention. (DONE)The
EGW_LANGFILE_PREFIX constant has been defined in both
setup/inc/class.setup_translation.inc.php and in translation class, but
current value is still 'phpgw_'. This affects setup/inc.
- - Autotranslation capabilities or search for existing similar phrases.
- - Move "loginscreen" and "mainscreen" to somewhere else (maybe config),
because they're not apps
- - Split mutisentences phrases into single sentences.


=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D END OF PASTE =3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D


I hope your comments about all this. The installation is at
http://www.omgs.net/~oscar/egroupware/, but your accounts (nor any demo
account) aren't available yet. Anyway, at least at this time, this
installation isn't intended to commit changes to 1.2 branch, but to test
the new translation tools, and help to debug your own translations.


Regards.

- --

|----------------------------------------------------------------------|
| http://counter.li.org info: Linux user: 92390 - Linux machine: 39301 |
|        Oscar Manuel G=F3mez Senovilla - omgsATescomposlinux.org        =
|
|               GPG Key at http://pgp.escomposlinux.org                |
|----------------------------------------------------------------------|
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFEjcBDQpr3kykd/aQRAjpYAJ9uPXUacNUnBsWNlbKMwVFnlsC1jwCfbymF
YKA3jkqAoeEDe1YwsT1uZPg=3D
=3DNXCD
-----END PGP SIGNATURE-----