[Indic-computing-users] FEATURE: The Penguin's Mantra...

[Frederick N. <fr...@by...>]

THE PENGUIN'S MANTRA: Efforts are on to find Indian language solutions to
GNU/Linux in the subcontinent that matters

By Frederick Noronha

Some call Indic and other South Asian scripts the final challenge to
computer vendors for full I18n support. It has taken its time in coming, the
challenges are not that simple, and successes have been few and far between.
Yet optimism is high now. Can this frontier be conquered early?

South Asia -- home to nearly one-sixth of humanity -- is struggling to
attain regional-language solutions that would make computing accessible to
the common man. It would throw open the keyboard to hundreds of millions.
Even if most are poor and have low purchasing power, this could open the
floodgate to greater computing power and much-needed efficiency in a
critical area of the globe.

India has 18 official languages written in 10 different alphabets:
Devanagari (used for Hindi, Marathi, and others), Bengali, Gurmukhi
(Punjabi), Gujarati, Oriya, Malayalam, Kannada, Tamil, Telugu, and Latin
(English). In addition, more than 800 other languages spoken in India do not
have official status. Mandrake Linux, one popular distribution, includes
keyboards and fonts for Bengali, Devanagari, Gujarati, Gurmukhi, and Tamil,
five of the nine Indic writing systems. Unfortunately, many applications do
not accept these characters, and those that accept them may not handle them
correctly.

For sure, GNU/Linux is making headway. Even if all this has so far largely
failed to get the attention it deserves. Keep your fingers crossed,
GNU/Linux could throw up some interesting surprises here.

Come September-end, a number of key proponents of Indianization plan to meet
up in the southern city of Bangalore, which is sometimes called India's
Silicon Valley. They plan to bring together energetic young developers
working on developing local language development tools, applications, and
content.  It would be a small and informal workshop, with 20-25 core
participants, including a lot of free discussion intended to spur "creative
and passionate thought" about the future of local language computing
technologies.

It's not difficult to understand the implications of this endeavour. Some
Indian language regions are larger than most countries. Take Hindi with its
366 million speakers, second worldwide in number only to Chinese Mandarin;
Telugu with 69 million; Marathi's 68 million; and Tamil with 66 million.
There are another 13 Indian languages in the top-70 global languages with
over 10 million speakers.

Other languages spoken in India are also spoken beyond: like Bengali (207
million speakers in India and Bangladesh), and Urdu (60 million in Pakistan
and India). Naturally, this linguistic space needs to be closely watched.

Range of initiatives
--------------------

Varied initiatives are currently on in various parts of the country.

The most exciting project is the plan to create the Simputer, a simple and
relatively inexpensive computing device that would benefit groups of even
illiterate villagers.

GNU/Linux enthusiasts are optimistic about its potential, not only because
this planned computing device runs on their favourite OS (operating system).
The creation of the Simputer is in 'open' design format. This innovation
from India gives a new meaning to 'Open Source' hardware.

"Palm tops can't compute in Indian languages and don't have text-to-speech
interfaces for Indian languages. They are also not aimed for the mass market
that Simputer is and still have a more elitist user community," says the
early-twenties developer Abhas Abhinav of Deep Root Linux in Bangalore.

Dhvani is a text-to-speech system for Indian languages developed by the
Simputer Trust developers and others. Enhancements in the works include a
better phonetic engine, a Java port, and a language independent framework.
(For more information http://sourceforge.net/projects/dhvani/ )

Meanwhile, IMLI is a browser created by the Simputer Trust, using the IML
markup language. IMLI is designed for small format display and easy creation
of Indian language and content and is integrated with the text-to-speech
engine. This browser can be independently installed on any computer running
GNU/Linux.

In national capital New Delhi and the western touristic state of Goa,
campaigners are struggling to take GNU/Linux to the classroom.
Indian-language solutions could obviously take such a project much farther
than software restricted to English.

Pankaj Kaushal <pa...@sa...> is an early-twenties free software
activist and web-master at the Sarai.net in Delhi. He says that he will soon
be working on a Hindi solution for "either KDE or Gnome or both". "Right now
I am collecting information about what needs to be done," he says.

Kerala is a southern state with an impressive 90% literacy rate and
Malayalam, the language of Kerala, is spoken by 35 million people. Senior
local government official Ajay Kumar <kum...@ya...> is leading an
initiative to make GNU/Linux friendly to the South Indian language of
Malayalam. He says: "We propose to develop a renderer for our language.
Specifically, we are looking for a renderer for Pango (the generic-engine
used with the GTK toolkit)."

They are working to get persons who have worked on Malayalam and Unicode to
offer some of their work for this project, specially in fonts. Ajay Kumar
adds that in nine months time, "we want to create an atmosphere where
language computing in Malayalam improves". Says he: "We are confident that
once we deliver the basic framework, others will start localizing more
applications in Malayalam."

Other initiatives have also come up, like the GNU India Translation Project
(GTP) by gnu_india. It aims at the localization of GNU/Linux program into
the native languages of india. See http://sourceforge.net/projects/gnuindia/

Rahul Jindal, some time back, announced a Hindi-speaking chat robot. Deepti
in intended to be a Hindi speaking bot, on the lines of Alice
(www.alicebot.org). "We shall use or develop a Hindi TTS (text-to-sppech)
for the output and more frills as time permits," he announced. See
http://sourceforge.net/projects/deepti/

The itrans package by Avinash Chopde prints texts in Indian languages. It
uses Latin-encoded text for input. For output, it supports the Devanagari
script (used for writing Hindi and some other Indian languages), Gujarati,
Telugu, Kannada, Bengali, Tamil, Punjabi, and Romanized Sanskrit. Input
files can be in TeX, LaTeX, HTML, or PostScript format, and Unicode output
is supported. See http://www.aczone.com/itrans/

More importantly, international efforts are also helping India. Yudit, with
its recent 2.5.4 release, announced in recent weeks that it was offering
support in three south Indian languages -- Malayalam, Kannada and Telugu.
Delhi-based GNU/Linux veteran Raj Mathur commented: "The current version of
Yudit has complete support for Malayalam and other indic languages. It can
also use Opentype layout tables of Malayalam fonts. I think Yudit is the
first application that can use Opentype tables for Malayalam."

K Ratheesh was a student of the Indian Institute of Technology-Madras (at
the south Indian town of Chennai) when he worked on enabling the GNU/Linux
console for local languages, a couple of years ago.

"As the (then) current PSF format didn't support variable width
fonts, I have made a patch in the console driver so that it will
load a user-defined multi-glyph mapping table so that multiple
glyphs can be displayed for a single character code. All editing
operations will also be taken care of," Ratheesh commented.

Further, as he pointed out, for Indian languages, there are various
consonant/vowel modifiers which result in complex character clusters. "So I
have extended the patch to load user defined context sensitive parse rules
for glyphs and character codes as well. Again, all editing operations will
behave according to the parse rule specifications," he commented.

Says he: "Even though the patch has been developed keeping Indian languages
in mind, I feel it will be applicable to many other languages (for eg.
Chinese) which require wider fonts on console or user defined parsing at I/O
level." The package, containing the patch, some documentation, utilities and
sample files then weighed around 100 KB.

There are even projects aimed at helping to explore Indian holy books,
written in the ancient religious language of Sanskrit.
http://sourceforge.net/projects/ved/

Strategy suggested
------------------

One Indic-Computing Strategy Document, prepared in May 2002, noted that
India faces a unique local-language computing standardization and
capacity-building problem -- apart from other factors. This is simply due to
the wide variety of regional and local languages in wide use. Then, there's
also the organizational and regional obstacles inherent in the effort to
standardize this rich variety of languages.

It mooted a strategy of creating a hierarchy of participatory consortia, to
facilitate broad regional and local participation in standardization and
development from a range of stake-holders with differing areas of expertise.

"It is important that these consortia be participatory and inclusive to
properly represent the viewpoint of local developers, users and other
stake-holders.  We recommend the formation of state-level (regional)
consortia for each regional language, which should include participants from
the following key member groups: developers, technolgists,
users/practitioners, linguistic groups...," it added.

Tapan S. Parikh <ta...@ya...> is a US-educated Indo-American who has set
his heart on finding language computing solutions for the land of his
ancestors. says he is working with with colleagues to pull together some
linguistic info for Indian languages, document that and post it to the web.

Says he: "Basically the idea now is to put these guidelines out there and
solicit a lot of feedback on this info from the general community for each
language. From that we can collate the best results and publish in the
handbook.  So we don't want so many questions that people will be
overwhelmed, and we should accept partial responses and the collate the best
answers to each area on our side, but we still want to be comprehensive."

At the end of September's Bangalore meeting, organisers hope to have
assembled a community of technically informed and motivated people to
organize and lead the Indic-computing development effort into the future.

"The leadership of this community should be individually driven, technically
motivated, and entrenched with youth, vitality and a progressive vision,"
say Tapan Parikh, one of the organisers.

"We also hope that this broad coalition would play a facilitatory role in
helping local language groups interact more effectively with international
standards processes and forums, such as the Unicode Consortium, Li18nux and
W3C," say the organisers.

Which ones first?
----------------

Many issues need to be tackled in the search for solutions. For instance,
which languages need be tackled first?

HP's Bangalore-based technical consultant Joseph Koshy <JK...@Fr...>
argues that the north Indian "Hindi family" promises the greatest reach
population-wise. However, he feels the southern languages -- Kannada,
Telugu, Tamil and Malayalam -- offer the greatest promise of real-world
deployability.  They enjoy better support infrastructure needed to deploy an
effective IT solution.

Outside his work-life at HP, Koshy is a volunteer-developer of the FreeBSD
operating system and one of the founders of the Indic-Computing project on
SourceForge. Says he: "What I am interested in is helping make
standards-based, interoperable computing for Indian languages a reality.
This dream is bigger than any one operating system or any one computing
platform.  I want to see pagers, telephones, PDAs and other devices that
have not been invented yet interacting with our people in our native
languages."

But others have different views. Says C V Radhakrishnan
<cv...@ri...>, who himself works out of South India: "I think most
of the South Indian languages should pose much problems for its non-linear
nature, for example, to create conjunct glyphs one has to go back and forth,
while North Indian languages do not have this problem. Malayalam has
peculiar characters called half consonants ('chillu'), there is no
equivalent for this in other languages. This raises severe
computing/programming challenges."

Radhakrishnan is a TeX programmer, who runs the River Valley Technologies
out of Thiruvananthapuram in South India. RVT is solely dependent on free
software with GNU/Linux as the sole operating system and has been in the
business for the last eight years with publishers like Elsevier Science and
Institute of Physics as lead clients. "We've reversed the paradigm that
GNU/Linux is unfit for commercial companies!" says a proud Radhakrishnan.

He has also written four packages in TeX which are licensed under GPL or
LGPL and are available at Comprehensive TeX Archive Network (CTAN --
official repository of TeX software). Among these is pdfscreen (a
presentation software) which is very widely used in the TeX world.

While this debate goes on -- and the proof would lie in the actual solutions
that come up -- it's clear that some could be 'difficult' languages.

Edward Cherlin of Web for Humans, an international Web development company
based in Cupertino, California, says, "The problems of rendering each
standard Indic script are reasonably well understood, and will be solved
soon in Pango. The real difficulty is with languages that have never been
written, or are written in non-standard variants of the official scripts.
The only organization I know of that has been working seriously on this
problem is Summer Institute of Linguistics (SIL), and their work is stalled
for lack of funds." Cherlin is active in Unicode, L18nux, Pango, Free
Standards Group, and other organizations working on Indic and other
unsupported writing systems, especially on the problem of getting all of the
interested parties into contact with each other.

FreeBSD developer Koshy notes that the official Census of India lists 114
'major' languages in the sub-continent.  Linguists, who discriminate more
finely than the Census officials, peg the number of living languages in
India at 850+, he says.

The information is catalagued at Ethnologue, http://www.ethnologue.com/,
which is maintained by SIL.

Wish lists
----------

What would be the applications and solutions required for a good start?

Radhakrishnan's wish-list reads thus: "X window support for local language
(a promising project in this direction is Indix). A good editor that
supports Unicode is the prime requirement. Although 'yudit' supports
Unicode, it is highly insufficient as an editor. Multi-lingual typesetting
system -- Omega (16bit extension of TeX) is a good candidate for this.
Simple mail client like pine or mutt. Browser extended to support local
languages with local language menus."

Says Koshy: "The usual 'paper consumption' uses (i.e.  word processing,
printing etc) are always there. But I think that the greatest demand would
be for what I call 'relevant information' for the lack of a better name."

Content is also critical. Requirements vary widely.

It all depends on where computers are used, argues Koshy. For instance, the
Garhwal region could need a matrimonial service uniting its people scattered
around the world. Those in eastern town of Assansol might need information
about tobacco or tea markets, the locally-important produce.

Some stress the need for the basics -- enabling the user to type, save and
print documents in his language(s), the ability to share files with others,
a chance to read and send e-mail, and the opportunity to browse and search
the Net in her language.

Other wants come up fast too: Indian GNOME, KDE, Mozilla, Galeon, and
Konqueror; an office suite; and instant messaging solutions.

There has been much debate over where in X should the language stuff go --
whether in the client side or server side. Server side implementation like
Pango, Indix and the IIT-Madras Xlib work good, but find limited
acceptability so far. Modifications they have made have not been accepted
into mainstream X servers. This means mainstream distros like Redhat,
Mandrake, Debian do not have it, though back in India it can be an add-on.

There is the X extension approach eg XOM (X output method), being worked on
by a team in Sun microsystems, according to G. Karunakar. li18nux is working
on standards for keyboard input, among other things, in conjunction with the
Linux Standard Base of the Free Standards Group. They have focused first on
Input Methods for Chinese, Japanese, and Korean, but when Pango's Indic
support is complete they will extend their standard to include it.

At the toolkit level, Gtk and Qt are the most used toolkits. This helps. Gtk
already has a good framework through Pango project, and basic level support
for Indian languages. Qt also now has Unicode level support for all
languages, but rendering is not yet ready.

However, Pango is independent of Gtk, and can be used with Qt or any other
software.

On the font level, there is no font-encoding standard. ISFOC was aimed to be
one, but it has become synonymous as a CDAC encoding & lack of a document
describing it has left it being ignored in GNU/Linux solutions.

GNU, Li18nux and Pango are focusing on Opentype, which is the only font
format that provides the glyph mapping tables needed to support Indic
conjuncts. GOTE, the GNU OpenType Editor, will be the essential tool for
this effort when it is completed.

What applications are needed in languages of India? "Everything.  Don't take
'No' for an answer", argues Edward Cherlin.  On GNU/Linux, Cherlin points
out, you can volunteer to Indicize any application. In the future, when font
management and rendering are standardized, all applications will run in
Indian languages for input and output without further ado, and anyone will
be able to create a localization file to customize the user interface.
Volunteers are also needed to translate documentation.

Other OSs
---------

Experts in the field are also studying the progress of other OSs.

Some argue today that only Microsoft's WinXP has any kind of Indian language
support worth speaking about, even though Apple has provided Indian language
kits for many years. There is confusion about Unicode support for Indic
writing systems, since Unicode does not provide character codes for conjunct
glyphs.

Many in India still think that this is a design flaw in Unicode, whereas the
Unicode designers argue that it is a necessary design decision so that we
can escape from the current broken Indic rendering techniques.

The set of conjuncts is needed is not determined solely by the writing
system and language. It is font-specific, and can therefore only be
supported by font glyphs, not character encoding.  Unfortunately, PostScript
and TrueType fonts do not support the correct mapping tables, and the
problem can only be solved with Opentype fonts.

In contrast, rendering Indic scripts using PostScript or TrueType fonts
requires encoding the conjuncts directly in the text stream, rather than the
letters composing them, and requires non-standard software to translate
between the sequence of letters from the keyboard and the sequence of
conjunct characters in a non-standard font. The result is text that cannot
be sorted and searched properly, where spelling and grammar checkers cannot
operate. It is hard on users to have to wait so long for proper support of
Indic scripts through Unicode, but the results are guaranteed to justify the
delay.

Microsoft faces other problems too.

"When Microsoft came out with the South Asian edition of MS-Word, the fonts
had a lot of problems. Mostly, words were rendered as separate letters with
space in between and not combined together as is the case with most Indian
languages," says PicoPeta language technology specialist Kalika Bali.
PicoPeta is one of the firms working to create the Simputer.

Support for Indian languages in the open-source OSes is today confined to a
series of hacks and ad-hoc 'solutions', argues Koshy. Unicode support in the
open-source OSes has been slow to get started, but now has the goal in
sight.

Dr. U.B. Pavanaja <pav...@vi...>, a former scientist now
widely noticed for his determined work to push computing in the influential
south Indian language of Kannada (see www.vishvakannada.com), however sees
the progress as being "quite remarkable, compared with the scene about two
years ago". Says Pavanaja: "Current pricing and product activation of XP may
become a boon for GNU/Linux (since software 'piracy' would be more
difficult)."

Cherlin is optimistic. Says he: "By next year, the Pango project should
support all nine official Indic scripts, so the answer (to which languages
should be tackled at this stage) is 'all of them'."

As Cherlin argues, Indic and other South Asian scripts are the final
challenge to computer vendors for full I18n support. "Progress is slow at
Microsoft and Apple. Linux should pass them by the end of the year, or early
in 2003," he feels.

"Apple and Microsoft are not willing simply to support typing, display, and
printing. They will not release language and writing system support until
they have complete locales built, preferably including a dictionary and
spelling checker. Linux is under no such constraints," says Cherlin.

His points out that the Free Standards Group together with Li18nux.org are
proposing to rationalize and simplify I18n support under X, including a
common rendering engine, shared font paths, and other standards that will
greatly simplify the business of supporting all writing systems and all
languages. Cherlin points out that Yudit and emacs both support several
Indic scripts, and could be extended with only moderate effort on the part
of a few experts.

Mandrake Linux includes Bengali, Gujarati, Gurmukhi, Hindi Devanagari, and
Tamil out of the box. That leaves Oriya, Malayalam, Telugu, and Kannada
still to be done, along with the Indic-derived Lao, Sinhala, Myanmar, and
Khmer. Tibetan and Thai are moderately well supported, Cherlin contends.

"Recently, localization efforts are picking up," agrees
scientist-free software advocate in Mumbai, Nagarajuna G.

"Other operating systems have their own funds for R&D.  GNU/Linux depends on
volunteers and external financial support.  If the government or other
funding agencies can spare even some amount to bodies like Free Software
Foundation of India, and others who are active in the localization
initiative the developers can work with obsession and make this happen very
fast.  FSFIndia is presently working with Kerala government to produce
Malayalam support to the GNOME desktop," notes Nagarajuna.

Incidentally, the Indian TeX Users Group have a project now to fund font
designers in all the Indian languages who are ready to write fonts and
donate under GPL to TUGIndia.

They've thus secured 'Keli' a Malayalam font family in various weights and
shapes written by Hashim and released under GPL. "We do hope to get more
fonts in other languages to fill up the gaps. We hope to use the savings
generated with TUG2002 (to be held in India in September 2002) exclusively
for this purpose," says Radhakrishnan in Thiruvananthapuram.

Technical challenges
--------------------

Technical challenges are certainly not few: the X rendering model is too
simple for Indic scripts (but an upcoming tutorial on the Indic Computing
site will have the nitty-gritty). Input for Indian languages is an open
issue. Most `keyboard solutions' available today for X are fragile and are
really more work-arounds than solutions.

In Cherlin's view, the principal problem is rendering conjuncts without
proper rendering engines and properly encoded fonts. Users want to type a
sequence of characters, and not concern themselves with the details of
rendering.  This requires fonts with appropriate tables giving the possible
character sequences and the glyphs for rendering each, and an engine that
knows to read the
 tables.

Once this is accomplished, standard keyboard layouts will work correctly and
robustly.

Recently, at the user-interface level, GNOME/Gtk teams tried rendering
Unicode encoded Devanagari (Hindi's script).  But this is specific to GTK
and doesn't extend to the other X toolkits, adds Koshy.

Says Koshy: "I don't know of any non-X user interface toolkits that support
Indian languages. General text processing toolkits -- a toolkit or library
that helps in manipulating Indian language text, for sorting, searching,
storage and retrieval -- are not known to me.  We don't even have the
necessary technical information about 90% of our languages that we can use
to get started on such a toolkit."

In current font technology, only Opentype is capable of handling Indic
scripts correctly.

Availability of good-quality fonts is another matter; but, as Koshy puts it,
this is not really a show stopper. Display technology for embedded devices
(pagers, small devices) for Indian languages is not well developed.

Languages like Urdu and Sindhi have right-to-left scripts which look similar
to Arabic but are, in fact, different, argues Prakash Advani who some years
back launched the FreeOS.com initiative. Urdu is the main language of
Pakistan, but is also used in India.

"I have found a great problem with typesetting of technical documents and
school/college text books particularly in the disciplines of math, physics
and chemistry. Why this happens is the lack of local language support for
TeX, the world's best math typesetting system. When an operating system does
not support the education in local language, the purpose of usage of
computers is extremely diluted," argues argues C V Radhakrishnan.

Satish Babu <sb...@in...>, a Free Software enthusiast and Vice President of
InApp, an Indo-US software company dealing with Free & Open Source
solutions, points to other problems: Collation (sorting) order confusion
(oftentimes, there is no unique 'natural' collation order, and one has to be
adopted through standardization).

Then there's also the non-availability of dictionaries and thesauri in
Indian languages and issues arising out of multiple correct spellings for
words; encoding standardization (Unicode) that will, inter alia, facilitate
transliteration between Indian languages programme support (database,
spreadsheet) for sorting/searching two-byte strings; lack of support of some
languages (eg., Tulu, Konkani, Haryanvi, Bhojpuri) which are the
mother-tongues for some sections of the Indian population.

Ravikant <rav...@sa...> who taught History at Delhi University before
moving to the Language and New Media project of sarai.net, has experimented
and worked on a wide range of Indian laguage possible solutions for his own
needs. He says: "The long term solution is of course Unicode and the package
Yudit already works on both Linux and Windows. Using the package you can
write e-mails, by cut-paste on any of the browsers -- the new Mozilla and
IE; host web-pages -- in short, write html."

For short-term measures, he suggests working towards developing the existing
packages "in a manner that people can use them with freedom from OS's and
fonts". ITRANS and WRITE32, written by Indians settled abroad, are
transliteration packages which already do so. The LATEX-Devnag package is
being used and promoted by Mahatma Gandhi International University, Delhi.

Then, there are packages that, according to Ravikant, do not offer OS
freedom. These are for Windows only: Baraha (www.baraha.com), I-Leap and
IndiaPage (mithi.com).

Cherlin suggests, for those ready to pay for it, Unitype Global Office, an
add-on to Microsoft Office which supports Hindi, Marathi, Nepali, Sanskrit,
Punjabi, Gujarati, Bengali, Assamese, Tamil, Telugu, Maldivian, Kannada,
Malayalam, Urdu, Pashto, Dari, and many other languages. See
http://www.unitype.com/globaloffice.htm. Although it uses non-Unicode
encoded fonts and a non-standard rendering engine, Global Office and
Microsoft Office together are capable of writing Unicode files that can be
viewed correctly with Opentype Indic fonts when they become available.

Says Prakash Advani <pr...@ne...> of the freeos.com initiative:
"There is definitely a market for Indian language computing that exists
today but there is a huge untapped market. 95% of the population don't
read/write English. If we can provide them a low cost Indian language
computer, it will be a killer." But as important as the office suite and
browser is getting across Indian-language content, says he.

Adds Advani: "The biggest challenge in not technical but lack of standards.
Until Unicode happened, there was a complete lack of standards. Everyone was
following their own standards of input, storage and output of data."

Unicode brings in standardisation. But not all is hunky dory. Issues remain,
like: not everyone agrees to Unicode even though it is an international
standard; not all the applications are Unicode-enabled but things are
getting there; most Indian language web sites don't support Unicode; not all
operating systems support Unicode.

There is a lack of free Indian language fonts. "There are over 5000
commercial Indian language fonts but there are probably 10 Free (GPL/royalty
free) Indian language fonts. This is a serious issue and more efforts should
be made to release free fonts," says Prakash Advani.

SIL's Fonts in Cyberspace pages at http://www.sil.org/computing/fonts/ and
Alan Wood's Unicode Resources at
http://www.hclrss.demon.co.uk/unicode/fontsbyrange.html both list fonts for
every major writing system, but hardly any are GPL-ed. This is about to
change, according to Cherlin. "Several projects and numerous individuals are
working on Free Unicode fonts, now that commercial Opentype font editors
such as Tiro Typeworks Fontlab 4.0 are available. Finishing the GNU OpenType
Editor (GOTE) will speed things up much more."

One other view is that GNU/Linux's GUI is a soup of various protocols and
toolkits and there is no single point where Indian languages can be
incorporated. GTK and Qt has separate projects for i18n, but neither one is
sufficient. IndiX takes a different route and works at the X level. Over all
the whole process is awkward, in one word.

However, Pango is not bound to GTK, and will be usable anywhere.
Furthermore, it will be mandated by the Free Standards Group and Li18nux
standard for internationalization (I18n) now in development.

Besides, others point out, Fonts are another mess altogether. Most of the
current implementations rely on glyph locations to display and store
information. For instance, to represent the latter "a" what is stored is the
position of "a" in some particular font used by that package. This is
different from normal English where the ASCII standard specifies that to
represent "a" the number 97 has to be used. No such standard exists for
Indian languages and thus one document written in one language cannot be
opened in another application. This is also the reason why in Indian Web
pages particular fonts specified by the author must be used.

Such a situation is also used by vendors to lock in their customers to a
particular product. This also hampers the exchange of email to situations
only where both the parties have same web interface or program to use an
Indian language email.

TUGIndia, which Raj represents, has procured a Malayalam font (Keli) from
font designer Hashim and will convert it to Opentype and distribute it under
GNU GPL. The project is expected to be completed by September 2002. Raj
works as an engineer at Linuxense Information systems, and leads the Indian
TeX users group's localization project.

Says G. Karunakar, the soft-spoken untiring Mumbai-based campaigner for
Indianisation of computing: "There are very few people in India who
understand Fonts technology completely, so most fonts that are available are
buggy. Due to lack of font standard, our fonts are not tagged as a Indian
language font."

Right now a general consensus is building on Opentype Fonts as the suitable
technology for Indian language fonts. There is already a free Devanagari
font ('Raghu' by Dr RK Joshi, NCST and used in Indix), a Kannada Opentype
from KGP, also for Malayalam, Telugu and Bengali.

This leaves Gujarati, Gurmukhi, Oriya, and Tamil still to be done in
Opentype.

"A lot of know how is existing in rare books, which are difficult to get.
Lot of research work done by scholars, linguists, typographers etc is going
untapped, as we don't know of it, or the people who know it," adds
Karunakar.

Lack of information
-------------------

Call it challenges, or call them difficulties; they are waiting to be
solved.

Koshy says: "Well the *biggest* problem I see today is the lack of
information, in a form useful to a software developer.  Most of the
developers for open-source projects (and this holds true for closed-source
companies too) are not Indians."

"Though we Indians claim to be a software `super power', we apparently
aren't very good at producing working code.  For example, the core work in
bringing Devanagari support into GTK has been done by a few Europeans; the
`Indian' contribution has been in providing translations of application
messages," as he puts it.

Given this situation, campaigners at the ground level are saying its
imperative that information needed to implement language support be made
widely available so that whoever is interested -- be they Czech or
Scandinavian or Bengali -- can add Indian language support to the code base
that they maintain.

In term of voice synthesis and recognition, Indian languages face challenges
too. Language technology specialist Kalika Bali points to the lack of easily
available annotated speech corpora to train language/statistical models for
creating state-of-art TTS and ASR engines.

"This is especially the case for ASR as one would need to train the models
for dialectal variation if they were to be deployed in a semi-urban
environment. How many people actually use the standard Sanskrit-influenced
'Doordarshan' version of Hindi (used by the Indian government's main TV
channel) for their daily interaction, for example?" Bali asks.

Free Software advocate for education and research, the Mumbai-based Dr.
Nagarjuna G. lists the problem bluntly: "Lack of standards, lack of good
quality fonts available in the public domain.  Governments are spending lots
of tax payers money in the development of technologies and fonts, which
either are not following standards are the products are not freely
available."

Shrinath <shr...@ko...>, a senior staff scientist at
Mumbai's NCST, which has done some interesting work on this subject, says:
"We want Indian language programming to be as simple as programming for
English is today. Almost every company has to reinvent the wheel or buy
costly solutions from others. In English, the OS supports it. It's a chicken
and egg problem. If there are apps in indic, the OS vendors will build the
fundamental capabilities into the OS, and if the capabilities are built in,
there will be more apps."

"English has been de-facto language for software development as well as
usage. So there is a long way to go. China is working fast on that end as it
appears, so can we," argues Girish S, an electronics engineer from the
Central Indian region of Madhya Pradesh and set up apnajabalpur.com.

There are other needs too: dictionaries and spelling checkers, of course.
Word-breaking doesn't operate the same way in Indic scripts as in the Latin
alphabet. And fine typography, which you don't find in consumer or office
applications in any language.

Adobe has not created a Unicode version of FrameMaker, or for that matter a
Linux version of any kind. Omega TeX will be wonderful when it is completed,
but it needs a lot of volunteer work.

One major challenge are the sheer numbers. India is believed to have 1652
mother tongues, of which 33 are spoken by people numbering over a hundred
thousand. (END OF MAIN ARTICLE)

*******************************************************************
SEPARATE BOX ITEM PLEASE:
Finding an Indian tongue for the Penguin
*******************************************************************

Support for Indian languages is coming in slowly. But there are several
efforts going towards this end:

* IndLinux project
http://www.indlinux.org  / http://www.sourceforge.net/projects/indlinux A
volunteer group working at the desktop level (KDE/GNOME), using Unicode, but
ISCII, the Indian standard character interface and South Asian equivalent of
ASCII, will also be supported by providing converter tools, Current focus on
Opentype fonts development, translations for GNOME 2.0 .

This group aims to play the integrating role, on putting all the pieces
together to make it usable. Now, a distributed approach is being taken, to
encourage encourage people to take up localization for their language. There
are now volunteers from more remote areas like Bhopal, Jabalpur, Nainital
etc apart from regular centres like Mumbai, Pune, Hyderabad, Bangalore.

Current work is on  Gnome 2 translations, to make it simple to use so that
all the user needs to do is either change his language or keyboard layout.
Also in progress is having making a Hindi enabled version of upcoming Redhat
7.3.93 (Limbo) which will probably become Redhat 8.0. So you will now have
option of installing in Hindi.

* IndiX  http://rohini.ncst.ernet.in/indix/
A modified X server to support Indian languages using opentype fonts. Uses
Unicode. Seeks to bring Indic support at the OS level on GNU/Linux.  Others
too agree that NCST's (the Govt of India's Mumbai-based National Centre for
Software Technology) localization work is promising, both for Indix and
OpenOffice in Hindi.

* IITM indlinux  http://www.tenet.res.in/Donlab/Indlinux/
From the prestigious IIT-Madras in the south Indian city of Chennai. They
have modified X & console in kernel to support Indian languages. Use ISCII
encoding only.

* Linux Localization Initiative ( LLI ) - lli.linux-bangalore.net
A volunteer group working on translating  LDP documentation (starting with
HOWTOs) to Indian languages.

* Indic-computing project.  indic-computing.sourceforge.net
Aims to create a resource center for all Indian language issues in
computing. It is aggregating all language info in one place, so that its lot
easier for developers in future.

* Language Technology Resource center (LTRC), IIIT Hyderabad. -
http://www.iiit.net/ltrc/index.html They have developed language
dictionaries, plug-in for viewing ISCII, and font converters. Also doing a
machine based translation tool (Anusaaraka). Most of their work is release
under GNU GPL.

Indix, IITM and IIIT-Hyderabad's work is supported by the Government of
India. The rest are volunteer-based and "looking for funds".

J. Patricia Annie JebaMalar <pa...@la...> updates us on some
interesting GNU/Linux and other OS work happening in South India, in the
Tamil language heartland and nearby.

* http://www.chennaikavigal.com
* http://www.tamillinux.org

Some other projects earning notice:

A team doing good work is the IITM team [http://acharya.iitm.ernet.in/]; the
algorithms/approach are interesting.

Says scientist-turned-editor Dr Pavanaja: "Mithi Technologies, the
Pune-based firm, have done good job on the web server front. This is quite
thought-off effort, as the majority of the web servers run on Linux-Apache.
They have good future. Let us hope best for them."

There are also international projects that could benefit Indian computer
users: Pango, Graphite, Li18nux, Free Standards. Mandrake Linux, which
emphasizes multi-lingual support and welcomes offers of help. And of course
India's own attempt at building a people-friendly low-cost computing device,
the Simputer (www.simputer.org).

Pango http://www.pango.org
Graphite http://www.sil.org/computing/graphite/
Li18nux http://www.li18nux.org
Free Standards Group http://www.freestandards.org/
Mandrake http://www.mandrake.com

http://www.parabaas.com/Parabaas_Axar/index.html (Bangla editor for Linux,
Java-based, runs on all platforms)

There's also the Indian language work by a team in the International
Institute for Information Technology (IIIT) Hyderabad. They have been doing
good work in areas of machine translation, linguistics, dictionaries etc.
and much their work is available under GNU GPL.

There are two international projects to create a complete rendering engine:
Pango (Pango.org, Li18nux.org) and Graphite (sil.org). India could gain from
these. They also have plans for complete sets of Unicode fonts (including
not just the Unicode characters, but also all of the non-character glyphs
for rendering Indic scripts. -FN (END OF BOX ITEM)
-- 
FOOTNOTE: Thanks to Edward Cherlin for his generous help in revising this
article. Responsibility for errors and inaccuracies rest however with the
author. (ENDS)