How do I activate utf-8 output for the console? In Scite I just uncomment
code.page=65001
output.code.page=65001
which works. I can't get it to work in Npp. Is it Windows-related?:
a) Choosing a unicode font for the console does not work;
b) opening a console and typing "chcp 65001" gives "Process failed";
c) creating chcp.bat with "chcp 65001" creates an infinite loop and I have to forcefully close NppExec (thankfully Npp does not crash)
For the Windows command prompt I went into the registry and added support for courier new, but still no satisfaction there....
Doing "chcp 65001" at a windows command prompt gives no errors but then I cannot run scripts, so I suspect the same will happen in NppExec
Sample: I did a process to write the following to the console (make sure your encoding is utf-8):
TEST امتحان
What I get on the console (both windows and nppexec) is
TEST +º+à+¬+¡+º+å
This is not even ansi, let alone utf-8, ansi utf-8 would look like this:
TEST امتØان
I guess Scite's rendering is independent of the windows subsystem that controls console behavior. Is there anything we can do get utf-8 output in NppExec or even the windows console?
Best wishes
Idris
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
SciTE may use Scintilla's window for the console output. The Scintilla control handles UTF-8 internally. I did not use Scintilla control for the console - I have used Rich Edit control (native Windows control) instead. Now I see that it was not very clever, but currently there is A LOT OF code which directly works with the Rich Edit control in the plugin, so it's too late to use Scintilla control instead.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
yes there are things you can do to get utf-8 output in NppExec : Join the development team and help us ;-)... More seriously I'm puzzled because I thought that DV had taken care of that (at least I see in his code a clear attempt to use TCHAR, and associated functions. look in the NppExec.cpp in the ConsoleOutput function).
if you can investigate further please feel free.
w.k.r
Nico
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I think that NppExec must support MustDie9x while Notepad++ supports it. Thus, we can not have real Unicode controls (created with CreateWindowW instead of CreateWindowA) in this plugin. Therefore each Unicode string must be converted to ANSI to be shown correctly in non-Unicode window.
I can add one more menu item - "UTF-8 Console Output". This menu item will mean the following: a UTF-8 text is read from the console process'es output, then this text is converted to UCS2-LE (Unicode in Windows) and, finally, from UCS2-LE to ANSI.
However, the latest sources of NppExec can be built as Unicode version. But the Unicode version:
1) will not work under MustDie9x
2) will not be compatible with current implementation of Notepad++
3) is not completely tested.
Regarding the points 1 and 2: if nobody cares about MustDie9x anymore, then I could modify the plugin to have a pure Unicode user interface (windows, dialogs, controls) and the ANSI internal interface with Notepad++. I.e. the Unicode inside, but will work only under Windows 2000/XP/2003/hasta la Vista/etc.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
> But the Unicode version:
> 1) will not work under MustDie9x
This I understand
> 2) will not be compatible with current implementation of Notepad++
How so? Is it only in the sense that Npp supports W98 and that's all?
> 3) is not completely tested.
I'll be happy to test it ;-)
> if nobody cares about MustDie9x anymore
I certainly don't, but in any case is it possible to freeze the present plugin and make it available for people who need the W98 compatibility?
Just a thought.
Thanks so much for your quick response!
Idris
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
By the way, this conversion: from UCS2-LE to ANSI - will not allow you to see correct characters which are not supported by your current locale. So, as far as I know, there is no way to see, for example, hieroglyphics properly without Unicode user's interface when your locale does not support the hieroglyphics.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
>> 2) will not be compatible with current implementation of Notepad++
> How so? Is it only in the sense that Npp supports W98 and that's all?
To support W98, Notepad++ has non-Unicode windows and controls. Therefore Notepad++ is mostly non-Unicode application inside. Thus, if NppExec will be a complete Unicode plugin, then it will have to transform a text from Unicode to ANSI when sending something to Notepad++, and transform ANSI to Unicode when receiving something from Notepad++. So there is a lot of work to do.
The best solution is to create some communication object (class in C++) between Notepad++ and NppExec instead of direct communicating. In this case this class could allow direct communicating for non-Unicode version of the plugin and ANSI-to-Unicode & Unicode-to-ANSI communicating for Unicode version. But, as I said, it needs a lot of work.
Also, as I said, I think that NppExec must support MustDie9x while Notepad++ supports it.
If Don will decide to make Notepad++ to be completely Unicode application, then the plugins will also be completely Unicode (well, at least those plugins where Unicode is needed).
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
> To support W98, Notepad++ has non-Unicode windows and controls.
> Therefore Notepad++ is mostly non-Unicode application inside.
I see. From the outside, I always thought Npp was a fully unicode application (since I do so much of my utf-8 work within it).
> The best solution is to create ....
If there is anything I can do to help, let me know. A utf-8 compliant console in Npp will be HUGE blessing to many of us, especially in the TeX community (and the greater il8n world). This is especially so since the Windows shell is worthless for utf-8. I greatly appreciate your efforts :-)
Best wishes
PS I basically live in the NppExec console and hardly use the Run menu anymore: Great job!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
After the 0.2 RC1 will be released, I'll add the code for "UTF-8 console output" menu item inside an "#ifdef"-block and send the test version to you. If it will not work, I'll simply de-activate this option.
Or maybe I'll add a lot of "#ifdef" elements to have the ability to build both Unicode and non-Unicode version of the plugin.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi,
How do I activate utf-8 output for the console? In Scite I just uncomment
code.page=65001
output.code.page=65001
which works. I can't get it to work in Npp. Is it Windows-related?:
a) Choosing a unicode font for the console does not work;
b) opening a console and typing "chcp 65001" gives "Process failed";
c) creating chcp.bat with "chcp 65001" creates an infinite loop and I have to forcefully close NppExec (thankfully Npp does not crash)
For the Windows command prompt I went into the registry and added support for courier new, but still no satisfaction there....
Doing "chcp 65001" at a windows command prompt gives no errors but then I cannot run scripts, so I suspect the same will happen in NppExec
Sample: I did a process to write the following to the console (make sure your encoding is utf-8):
TEST امتحان
What I get on the console (both windows and nppexec) is
TEST +º+à+¬+¡+º+å
This is not even ansi, let alone utf-8, ansi utf-8 would look like this:
TEST امتØان
I guess Scite's rendering is independent of the windows subsystem that controls console behavior. Is there anything we can do get utf-8 output in NppExec or even the windows console?
Best wishes
Idris
SciTE may use Scintilla's window for the console output. The Scintilla control handles UTF-8 internally. I did not use Scintilla control for the console - I have used Rich Edit control (native Windows control) instead. Now I see that it was not very clever, but currently there is A LOT OF code which directly works with the Rich Edit control in the plugin, so it's too late to use Scintilla control instead.
coucou Idris,
yes there are things you can do to get utf-8 output in NppExec : Join the development team and help us ;-)... More seriously I'm puzzled because I thought that DV had taken care of that (at least I see in his code a clear attempt to use TCHAR, and associated functions. look in the NppExec.cpp in the ConsoleOutput function).
if you can investigate further please feel free.
w.k.r
Nico
Hi Nico,
Since I'm not a programmer, not sure how much direct help I can be to the NppExec team ;-)
OTOH, I have made an Npp package for the ConTeXt typesetting system (including some nppexec scripts)
http://wiki.contextgarden.net/Image:Npp_ConTeXt.zip
so I do give back the best that I can.
Back to the issue at hand:
> if you can investigate further please feel free.
For you guys:
http://mail.python.org/pipermail/python-list/2003-April/199538.html
http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=162041&SiteID=1
If I can be of any other help in getting this solved please let me know.
Best
Idris
> if you can investigate further please feel free.
See also
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/base/writeconsole.asp
Idris
I think that NppExec must support MustDie9x while Notepad++ supports it. Thus, we can not have real Unicode controls (created with CreateWindowW instead of CreateWindowA) in this plugin. Therefore each Unicode string must be converted to ANSI to be shown correctly in non-Unicode window.
I can add one more menu item - "UTF-8 Console Output". This menu item will mean the following: a UTF-8 text is read from the console process'es output, then this text is converted to UCS2-LE (Unicode in Windows) and, finally, from UCS2-LE to ANSI.
However, the latest sources of NppExec can be built as Unicode version. But the Unicode version:
1) will not work under MustDie9x
2) will not be compatible with current implementation of Notepad++
3) is not completely tested.
Regarding the points 1 and 2: if nobody cares about MustDie9x anymore, then I could modify the plugin to have a pure Unicode user interface (windows, dialogs, controls) and the ANSI internal interface with Notepad++. I.e. the Unicode inside, but will work only under Windows 2000/XP/2003/hasta la Vista/etc.
> But the Unicode version:
> 1) will not work under MustDie9x
This I understand
> 2) will not be compatible with current implementation of Notepad++
How so? Is it only in the sense that Npp supports W98 and that's all?
> 3) is not completely tested.
I'll be happy to test it ;-)
> if nobody cares about MustDie9x anymore
I certainly don't, but in any case is it possible to freeze the present plugin and make it available for people who need the W98 compatibility?
Just a thought.
Thanks so much for your quick response!
Idris
By the way, this conversion: from UCS2-LE to ANSI - will not allow you to see correct characters which are not supported by your current locale. So, as far as I know, there is no way to see, for example, hieroglyphics properly without Unicode user's interface when your locale does not support the hieroglyphics.
Idris,
>> 2) will not be compatible with current implementation of Notepad++
> How so? Is it only in the sense that Npp supports W98 and that's all?
To support W98, Notepad++ has non-Unicode windows and controls. Therefore Notepad++ is mostly non-Unicode application inside. Thus, if NppExec will be a complete Unicode plugin, then it will have to transform a text from Unicode to ANSI when sending something to Notepad++, and transform ANSI to Unicode when receiving something from Notepad++. So there is a lot of work to do.
The best solution is to create some communication object (class in C++) between Notepad++ and NppExec instead of direct communicating. In this case this class could allow direct communicating for non-Unicode version of the plugin and ANSI-to-Unicode & Unicode-to-ANSI communicating for Unicode version. But, as I said, it needs a lot of work.
Also, as I said, I think that NppExec must support MustDie9x while Notepad++ supports it.
If Don will decide to make Notepad++ to be completely Unicode application, then the plugins will also be completely Unicode (well, at least those plugins where Unicode is needed).
> To support W98, Notepad++ has non-Unicode windows and controls.
> Therefore Notepad++ is mostly non-Unicode application inside.
I see. From the outside, I always thought Npp was a fully unicode application (since I do so much of my utf-8 work within it).
> The best solution is to create ....
If there is anything I can do to help, let me know. A utf-8 compliant console in Npp will be HUGE blessing to many of us, especially in the TeX community (and the greater il8n world). This is especially so since the Windows shell is worthless for utf-8. I greatly appreciate your efforts :-)
Best wishes
PS I basically live in the NppExec console and hardly use the Run menu anymore: Great job!
After the 0.2 RC1 will be released, I'll add the code for "UTF-8 console output" menu item inside an "#ifdef"-block and send the test version to you. If it will not work, I'll simply de-activate this option.
Or maybe I'll add a lot of "#ifdef" elements to have the ability to build both Unicode and non-Unicode version of the plugin.
Ok, thanks a lot; I look forward to it! BTW, here are a couple of popular utf-8 test files
http://www.w3.org/2001/06/utf-8-test/UTF-8-demo.html
http://www.unics.uni-hannover.de/nhtcapri/multilingual1.html
Some of this depends of course on whether the relavant font has the glyphs.
I also noticed that Scite claims to work under W98; I wonder if its console-unicode works on that platform as well....
Best wishes
Idris