Menu
▾
▴
#27 Using unicode characters does wierd things in editor
Using (entering, pasting) unicode characters into the editor, either in a script or a notecard, does wierd things, like suddenly changing fonts.
I used pasting characters from the tables available there : http://www.tamasoft.co.jp/en/general-info/unicode.html
Moreover, if i activate windows unicode entry by setting the registry key HKEY_CURRENT_USER\Control Panel\Input Method\EnableHexNumpad to 1 (as described in http://en.wikipedia.org/wiki/Unicode_input ), typing alt 0 3 (num keypad) immediately crashes LSLEditor.
I just tried copy/pasting from the tables you linked to, without any unexpected problems. No font changes or any other weirdness. Can you confirm the version number you are using please?
I'm not aware that unicode entry needs to be "enabled", at least not on my XP and 7 versions.
Any how, I confirm that ALT+0+3 crashes the editor. That is likely because it is an invalid unicode point, but I do not know why .NET doesn't filter out invalid characters. I will look into whether this can be dealt with in LSL Editor.
I will post a screenshot of the font switching thingy when I get home. I am using the latest 2.46 version of LSL editor (on a Windows7-64bit machine). In case this is of importance, i am using the "original color scheme".
As to unicode enabling, this is in fact necessary to reach the higher unicode characters as hexadecimal input : you can then press ALT and type + (the plus sign from the numerical keypad) and four hexadecimal digits. I don't think it's related to the described bug though. (rebooting windows or at least relogging is necessary after setting said registry key).
A screenshot will not be of much help ;-) I need to be able to reproduce the problem. Please give detailed steps needed to reproduce the problem.
I'd initially misread your description, thinking you were referring to the normal ALT+numpad functionality.That's what I get for reading bug reports while tired. I've enabled the hex input method now and tried it on basic characters to get a feel for it. Please give me the steps to reproduce your problem.
example of a file triggerring the font bug
OK here is how you can (hopefully) reproduce the font problem. I uploaded a screenshot anyway :P
1) Open LSL Editor. Version check ; 2.46.
2) Open http://www.tamasoft.co.jp/en/general-info/unicode.html with google chrome (latest version).
3) Copy first table (characters 0000 to 3FFF) and paste in place of the default script.
4) Copy second table (characters 0400 to 07FF) and paste after the first table in editor.
5) Notice that the font changes (see screenshot, see the numbers in the first column of the table) after character 0540.
6) type some random text after the table... normal font show again.
7) save the file as new.lsl and close LSL editor
8) open the file with a double click... expand LSL window to full screen... font artifact is still there !
This is not an issue with the number of unicode characters used ; i saw this happen with very few unicode characters (black triangles figuring arrows, see characters in the line 25A0) in a string inside a regular script.
Perhaps I should have said "minimal reproduction".
The reproduction needs to be very simple and in the normal operation of the program. I need to know which character or combinations of characters cause a problem, your reproduction is too broad and has way too many variables to try diagnosing with
If you manage to find a much simpler reproduction, please add it here and I will look into it..
simpler file with same problem
I added a second file which is just the first one with only two lines and the problem showing up. Consider it as a notecard. Editing a notecard is normal operation.
This is just a rendering problem really, some unicode character is understood as some kind of control sequence...
I dont think it can get much simpler than that, but will continue to try and refine it.
Hi.
Again, an example of the problem does not help me to identify it much. As I asked before, I need a set of small steps to reproduce the problem. The process of reproduction is the best bet I have to identify the actual problem, not just its symptoms.
Your original post mentioned "using (entering, pasting) unicode characters...", the entering is my best bet for identifying the problem I think.
The second file narrows the problem down some, but not enough. I looked at the file in a hex editor and it is valid UTF-8, so no reason to render incorrectly because of bad codes. It is possible that the file is being interpreted as some other encoding than UTF-8, which I will try to test with a debug build.
I forgot to ask, has this problem only begun since 2.46 or were you seeing it before? If before, since which version?
I understand your point of view about trying to reproduce the problem. However, how I got this file put together does not really matter in my opinion, since the problem appears just by loading an otherwise valid file.
So the simple steps are just : load the new2.lsl file and see :) Indeed i first wrote the report stating entering and pasting unicode, but i could just rephrase it as loading a simple unicode file.
I had not witessed the problem with older versions but it's because i haven't really worked with unicode files within LSLEditor before.
I just tested it (loading new2.lsl) with LSLEditor 2.45.1.10325 and the same problem appears. BTW, the configuration for this older build is the default (including the source coloring scheme). I will try with even older versions and keep you informed about this.
I tested the new2.lsl file against different versions of LSLEditor as freshly downloaded from sourceforge.
From 2.40.0 to 2.44.0 inclusive, the problem does not appear when loading, as the new2.lsl file is assumed to be ANSI and not unicode. The problem appears by loading in version 2.44.1.
Copying and pasting the contents of the file does not seem to produce the appropriate unicode content in versions prior to 2.44.0 (only question marks - is there something to do in the options in those versions ? i didn't find anything appropriate). In version 2.44.0, the problem appears if i copy and paste the contents of new2.lsl.
From the user point of view, I agree it doesn't seem to matter, however from a bug finding point of view, general problems rarely lead to specific fixes. This is a good example of that, your apparently simple problem appears to me to be at least two different problems:
1) File IO is incorrectly identifying the file encoding (a vaild UTF-8 file appears to be interpreted as an ANSI one).
2) Pasting text is not being properly converted to Unicode characters.
Admittedly these seem related, but only by their symptoms, and that one allows the other to propagate.
The code processing these two problems is separate, not directly.unconnected.
I will have to dig deeper into both sets of code to identify the problem.
Thanks for testing with the older versions to narrow down the cause.
I don't think there are any options to help in < v2.44.1.