Does not support UTF8 resource file

[KW Yan]

Midlet 3.0 Alpha 3 and its previous version (including version 2.x) does not support resource file is in UTF8 format. I have created my own procedure to handle it:

    public static String readline(InputStream inputstream) {
       StringBuffer buffer = new StringBuffer();
       char c;
       try {
          InputStreamReader resource = new InputStreamReader(inputstream, "UTF-8");
          do {
             c = (char)resource.read();
             if (c == (char)10) {
                break;
             }
             if ((int)c != 13 && (int)c != (char)65279) {
                buffer.append (C);
             }
          } while (true);
       } catch (Exception exception) {
       }
       return (buffer.toString());
    }

I raised this request since I saw the 3.0 Alpha 3 mentioned about supporting utf8 in IDE. It will be nice if the whole software, including resource file, is fully support UTF8.

[Javier Santo Domingo]

Yes, MP needs to support UTF-8 also in resources, I had to deal with this problem in the past too. There are many things to enhance in the compiler, but since the MP 3.0 goal is to-provide-at-least-the-same-functionality-than-MP-2.0 and we already decided the migration of the compiler from c to pascal (that means the current compiler will be dropped just like the 2.0 IDE was), it seems this kind of enhancements will have to wait at least until, lets say, MP 4.0.

Just for the record: the only enhancements the MP 3.0 compiler has, from Niksa's original, and specially at the language level, are those which Artem (abcdef) added rewriting the lexer (shr and shl keywords, inline function to generate java byte code, smart string concat, and max array size up to 32000). My modifications were not focused in the feature-side, I only ported it to GCC, make it work as a DLL, made many bugfixes in different areas (lexer char literals bug, memory leaking bug, etc) and added the stuff needed to work with the new IDE (working with callbacks, enhanced messages, etc).

But be sure we all agree that UTF-8 support in resources handling it's pretty important and should be added as soon as possible when the compiler rewrite takes place.

Thanks for your feature request!