#1321 Add translation feature using Glosbe translation API
Translation would be a nice feature to add to Stendhal. Unfortunately, the biggest player in terms of translation APIs, Google, stopped offering its service for free in 2011 and now it costs money. Microsoft offers it for free as an API up to a certain number of characters per day. But if you go above that, it costs money. There are several other services offering translation APIs, some of which are free if you stay under a certain limit of characters. However, if you go over that limit it either costs money or you can get banned from using their translation service. This is discussed on this site: http://stackoverflow.com/questions/6151668/alternative-to-google-translate-api
But on that site I found an answer. There is a service called Glosbe that offers a free translation API, here I found a blog post by someone who discovered it and has C# code for how to use it: http://networkprogramming.wordpress.com/2013/08/31/translation-api-without-an-api-key/ So this translation is done without an API key. What is good about this is, if we integrate some translation service into the Stendhal client, then the translation API, if it is tracking requests, will see the number of requests from the IP address of each Stendhal client, rather than the total from the entire Stendhal game or Stendhal server, which would be a higher number. Also, Glosbe is a service offered by people who are into open-source and they are very fond of the idea of their service being used by open source software. Here is the website about their API: http://glosbe.com/a-api
Anyway, now that we know which API to use, there is the matter of how to integrate translation into the Stendhal game. Well my idea is, there are NPCs such as Harold and the Postman who have ways of interacting with characters. So I propose adding a new NPC to the game, a non-player character who players can talk to, who is a language expert and knows many, many languages. This NPC would be similar to the Postman, in that you could use IRC-like commands to send messages to that NPC, and the NPC would explain this to anyone who talks to him or her. However, unlike the Postman, messages sent to and from this NPC using IRC-like commands would not be handled server-side but rather client-side. So the code for sending a message to someone, it would check to see if the username is the same as the username that will be reserved for this NPC.
Anyway the reason I am proposing this is many players of the game have native languages other than English. There are many German, Polish, Portuguese, and Spanish speakers who play Stendhal. This NPC, you would be able to send messages, to translate text from one language to another. The NPC if you walk up to them in the game or message them with something that isn't a command they understand, would explain the command syntax to players. The command syntax I propose is "translate oldlanguage-code newlanguage-code text to be translated". The language codes to use are 3-letter ISO 639-3 language codes (see http://en.wikipedia.org/wiki/List_of_ISO_639-3_codes for a full list). Anyway here are some of the most common ones:
ARA - Arabic
DEU - German
ENG - English
FRA - French
ITA - Italian
JPN - Japanese
POL - Polish
POR - Portuguese
RUS - Russian
SPA - Spanish
ZHO - Chinese
So if I send the message "translate eng por hello" to that NPC, its response would be "olá", as an example. It translates words and phrases but doesn't do full-fledged machine translation of sentences.
Also, the legal terms of use for Glosbe are fully compatible with use in a GNU GPL game. Some people use hacks even today to use Google Translate results in their software, however this violates the terms of service of Google Translate. But this Glosbe translation service is free. At the bottom of their API page it says: "API is free to use, regarding indicated data source license. There is a limit of call that may be done from one IP in fixed period of time, to prevent from abuse." That is the terms of service. And then it says: "If you're using or planning to use our API: why don't you contact us? It's always nice to hear that someone is using our service. Consider also sharing your job as an OpenSource project. It's good to share."
So, they are very much open to having their service put to use in open source projects such as this one. The only problem is it isn't really a full-fledged machine translation service, but rather merely a database of translations for words and phrases, and if you put in a phrase that isn't in its database, it can't translate it. However, if that happens, the phrase can be broken up into separate words and each of them can be translated individually by the Stendhal client. This would, of course, lead to incorrect grammar, since if you translate every word in an English sentence into the equivalent Portuguese word, for instance, it will not be correct grammar. But, that is not such a bad problem, since a translation with the correct words but bad grammar is better than nothing, and people can figure out what it means. Also bad grammar can be funny, if you find sentences such as "All your base are belong to us" funny.
Oh and also, since this would all be done within the client, the only thing that would be done on server-side is adding the NPC who explains how this works and has the same name as the NPC that, on the client side, if you message, then instead of going to the Stendhal server as a message, the client directly uses the Glosbe API to do the translation. It would appear in the Stendhal client just as if it is a private message. So basically this would not increase the server load on Stendhal servers at all. Instead it would increase server load on Glosbe's servers, but only slightly, since not that many people play Stendhal.
Anyway, if you would prefer a full-fledged machine translation, well that is a more complicated problem, since Glosbe doesn't provide this. There is a free translation API with no limits provided by another service called Yandex. However its legal terms of service (see here: http://legal.yandex.com/translate_api/), I don't think they are compatible with this game. Google Translate costs money to use unless you use hacks that violate its terms of service. Microsoft's Bing translation API IS free to use if you are below a certain number of characters used in their translation service in a given period of time. However, they require a developer ID key. And all of the requests, even if they are done client-side, would show up as coming from the same developer ID key, and count towards that limit. This could potentially lead to a financial liability vulnerability: if a player or group of players did a whole lot of translations in their Stendhal clients, the Stendhal developers would end up having to pay money to Microsoft. Also, since this game is open source, having source code available to the public that contains the developer ID key in it would allow anyone in the world to use that developer ID key and do translations with it, which the Stendhal developers would have to pay for, if the volume of translations went above the limit for the Bing translation API free service. So I would advise against using the Bing machine translation from Microsoft, as it could potentially lead to having to pay them money. So, that is why I am recommending Glosbe, as it is the only translation service I know of with a 100% free-to-use API that is also 100% legal to use in open-source software, without any restrictions that would go against the spirit of open-source software, and without any security vulnerabilities that would be introduced by something like having a developer ID key. Yes, we probably could get better translation results that would also be free, by using one of the hacks to use Google Translate for free, but that is not legal. And we COULD use Yandex but that would require turning both the Stendhal website and the Stendhal game client into advertisements for Yandex and its services, the kind of commercialism and advertising which really goes against the spirit of open source and would really not look very nice on the website or in the game client. Globse is the only translation API provider I know of that is really compliant with the requirements of free, open-source software.
Discussion
Anonymous
Glosbe is a nice dictionary to lookup individual words or fixed phrases. It gives multiple translations and assings context to it.
It is not suitable to translate sentences because it is not able to extract context. Splitting a sentence up into words and translating them individually will result in a product that needs to be condensed:
Automatic translation is an interesting and useful suggestion. Without going to the technicalities of a passable translator (just assume there's a service we can use), I'll comment a bit about the UI.
I'd make it simply work such that let the user select an area in the chat log, then choose "translate", without doing any apparent NPC interaction. I'd add that item to both the main menu, and to the context menu. The user could select the preferred target and source languages in the settings, but both of them need a convenient way to use something else. (Unless the service can reliably detect the source language - then only the target needs a switch that's always available).
I estimate the number of unique chat lines to contain 25,000,000 characters per year. Bing offers 2,000,000 characters per month for free.
I guess we do not want to translate every chat line, which reduces the number of characters. But we may want to translate a line into multiple languages, which will increase the number of characters.
It may be possible to only offer translations from English and into English and cache the results. If we start with a number of beta testers for a month or two, the cache would be filled in advance.
I have not tested the quality of Bing translation server, yet.
I agree with Kimmo, that translation should be handled by the user-interface without NPC interaction.
I have the feeling, that it should be handled by the server instead of the client. In any case, we have to warn people that their chat is transmitted to an external service for translations. And that this service is hosted in a country with extremely week privacy laws.
Handling it through the server would certainly solve the issue with the AppIDs. It would be a part of the server configuration, and not need to be distributed with the client