Re: UTF-16 to UTF-8 algorithm

[Markus S. <mar...@us...>]

What George means is "When you want to convert the whole *file*...":

For process-internal strings, the macros work well and fast. We will use 
them for new convenience functions in the next release that allow to 
transform process-internal strings from UTF-8/32 to and from UTF-16.

For (potentially) large files that are read into byte buffers, you will 
not want to figure out the character boundaries yourself. For that, you 
want the buffer handling and exception handling capabilities of the 
converter framework. Also, if you use UTF-16 in files, then the byte order 
may not match the one of your current platform. We have UTF-16BE and 
UTF-16LE converters for that (one of them will do a memcpy()...).

markus

Markus Scherer  IBM GCoC-Unicode/ICU  Cupertino, CA 
mar...@us... (also for SameTime)





"George Rhoten" <gr...@us...>
2001-06-04 12:25

 
        To:     "Chip Tokar" <po...@us...>
        cc:     icu...@dw...
        Subject:        Re: UTF-16 to UTF-8 algorithm

 

There are two ways to deal with UTF-16 and UTF-8.  In some cases the UTF_* 

macros contained in include/unicode/utf.h are what you really need.  You 
can use those macros when you don't want to convert the whole Unicode 
string.

When you want to convert the whole Unicode string, you should use a 
converter.  Please see the User's Guide at: 
http://www-124.ibm.com/icu/userguide/conversion.html for more 
information on this subject.  The converter API is located in 
include/ucnv.h

I hope that helps.

George Rhoten
IBM Globalization Center of Competency/ICU  Cupertino, CA, USA

Sent by:        icu...@dw...
To:     icu...@dw...
cc: 
Subject:        UTF-16 to UTF-8 algorithm



Is there some place that describes what is needed to convert UTF-16
characters to UTF-8 characters?  The ICU tool only uses UTF-16.  However,
I need to convert the UTF-16 characters to UTF-8 characters.  Any help
would be appreciated.........Chipper

I/T SPCL-AD/M APPL INTEGRAT
IBM Global Services - Lex, Ky.
Phone - (859)234-3992   t/l 766-9852
Fax      - (859)243-4999
Internet mail - po...@us...
VM  - POLACK@LEXVM2

_______________________________________________
icu-charsets mailing list
icu...@ww...
http://www-124.ibm.com/developerworks/opensource/mailman/listinfo/icu-charsets


_______________________________________________
icu-charsets mailing list
icu...@ww...
http://www-124.ibm.com/developerworks/opensource/mailman/listinfo/icu-charsets