[ jEdit-devel ] [ jedit-Patches-1687684 ] Improve encoding handling especially for UTF-16, UTF-32

[SourceForge.net <no...@so...>]

Patches item #1687684, was opened at 2007-03-24 20:33
Message generated for change (Comment added) made by sf-robot
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=300588&aid=1687684&group_id=588

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
>Status: Closed
Resolution: Accepted
Priority: 5
Private: No
Submitted By: Kazutoshi Satoda (k_satoda)
Assigned to: Björn Kautler (vampire0)
Summary: Improve encoding handling especially for UTF-16, UTF-32

Initial Comment:
This patch is a large improvement about handling of
text encodings. This is also a preparation to improve
encoding auto detection. If this patch is accepted, I
will submit a next one which depends on this.

* User visible changes
  - User can control the endian of UTF-16 or UTF-32
    via new encoding names, x-UTF-16LE-BOM,
    X-UTF-32LE-BOM, X-UTF-32BE-BOM. These names are
    supported by Java SE 6. For older environments,
    jEdit also emulates these encodings.
    (This reverts r9092.)
  - BOM for UTF-32 is detected.
  - [Reload with Encoding] supports extended encodings.
    This possibly fixes the bug #1683494.
  - Loading with wrong encoding is reported as an
    error. (This might be disputable about usability.
    But silent data loss should be avoided in
    principle.)

* Developer visible changes
  - Extended encodings can be added via service.xml.
    This means a plugin can add special encodings.
    (I would like to see an encoding which performs
    Unicode Escapes for Java props file.)
  - Encoding can be completely specified by just one
    string name. This is important because the name is
    saved to a file and typed by user into a TextField.
  - BOM handling codes are encapsulated into one class.
    No more unprincipled public constants.


----------------------------------------------------------------------

>Comment By: SourceForge Robot (sf-robot)
Date: 2007-04-15 19:20

Message:
Logged In: YES 
user_id=1312539
Originator: NO

This Tracker item was closed automatically by the system. It was
previously set to a Pending status, and the original submitter
did not respond within 14 days (the time period specified by
the administrator of this Tracker).

----------------------------------------------------------------------

Comment By: Alan Ezust (ezust)
Date: 2007-03-29 10:37

Message:
Logged In: YES 
user_id=935841
Originator: NO

I was not able to adequately test this feature, since I rarely work with
foreign fonts or even other character sets. I ask that both you and Vampire
test it, since you both like to use lots of foreign characters.

committed to 9308. Assigning to Vampire to test. 




----------------------------------------------------------------------

Comment By: Kazutoshi Satoda (k_satoda)
Date: 2007-03-28 10:48

Message:
Logged In: YES 
user_id=1483238
Originator: YES

Please use this: myname.zip

This file contains some files which all contains
my real name in Japanese. The name of each file is the
encoding of the file. You might have to change the
font to see the correct result.

I hope this is funny enough for you.

Following is some casual test suits.

If auto detection is enabled:
All files except MS932.txt should be opened with
correct encoding because they have BOMs. Saving them
should reproduce exact same bytes.

If auto detection is disabled:
All files which can't be interpreted by the global
default encoding should fail to load. Using
[Reload with Encoding] with correct encoding should
avoid the error. Saving them should reproduce exact
same bytes.

File Added: myname.zip

----------------------------------------------------------------------

Comment By: Alan Ezust (ezust)
Date: 2007-03-27 16:08

Message:
Logged In: YES 
user_id=935841
Originator: NO

I'd like to apply this, but I need a file as a testcase to use "reload
with encoding". Can you please attach a file which has one of the encodings
and some funny characters in it as a testcase for me to verify the patch
with?



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=300588&aid=1687684&group_id=588