Re: [Cppcms-users] Questions about decode UTF-8 to Unicode code point

[陈抒 <csf...@gm...>]

Hello Artyom:
  I know the code_point type is uint32_t. In my app, I need to convert it
into uint16_t. Which one of the following ways is portable and don't need
to worry about the endianness(big-endian or little-endian), also can be
used in both 32bit and 64bit platform. I am using GCC4.x above.
Assume x is a code_point value
1. static_cast<uint16_t>(x)
2. x & 0xffff

Correct me if I am wrong, thanks.


Dean Chen
Best regards
http://blog.csdn.net/sheismylife


On Wed, Feb 6, 2013 at 12:20 AM, Artyom Beilis <art...@ya...> wrote:

> First of all there is utf_to_utf functions and if sizeof(wchar_t) == 4
> (which is on Linux/Unix)
> than all you need is to use them.
>
> About the error
>
> code_point c = utf_traits<char, sizeof(char)>::decode(str.begin(),
> str.end());
>
> the first parameter is reference that is moved forward and std.begin()
> can't be changed
> so what you need to do is:
>
> std::string::const_iterator pos = std.begin(),end =str.end();
> code_point c = utf_traits<char, sizeof(char)>::decode(pos,end );
>
> See:
> http://www.boost.org/doc/libs/1_51_0/libs/locale/doc/html/structboost_1_1locale_1_1utf_1_1utf__traits.html#a65f0b0e1075dd000d2c2c15af30be372
>
> Artyom Beilis
> --------------
> CppCMS - C++ Web Framework:   http://cppcms.com/
> CppDB - C++ SQL Connectivity: http://cppcms.com/sql/cppdb/
>
>   ------------------------------
> *From:* 陈抒 <csf...@gm...>
> *To:* cpp...@li...
> *Sent:* Tuesday, February 5, 2013 5:32 PM
> *Subject:* [Cppcms-users] Questions about decode UTF-8 to Unicode code
> point
>
> Hello Artyom:
>   I am using the boost.locale library contributed by you to decode UTF-8.
> The following codes works fine:
> char const * p = "一"; // one Chinese character here
> code_point c = utf_traits<char, sizeof(char)>::decode(p, p + 3);
> cout << "code point: 0x" << std::hex << c << " binary format:B" <<
> PrintIntAsBinaryString(c) << endl;
>
> My Print.. function prints it as what I expect.
> code point: 0x4e00 binary format:B00000000000000000100111000000000
>
> But don't know how to apply the decode function to std::string. when using
> iterator of string object, got some compilation error:
>
> string str = "一";
> code_point c = utf_traits<char, sizeof(char)>::decode(str.begin(),
> str.end());
>
> /home/chenshu/work/asio_echo/codes/main/test/main.cc:18:65: error: no
> matching function for call to
> ‘boost::locale::utf::utf_traits<char>::decode(std::basic_string<char>::iterator,
> std::basic_string<char>::iterator)’
> /home/chenshu/work/asio_echo/codes/main/test/main.cc:18:65: note:
> candidate is:
> /usr/include/boost/locale/utf.hpp:193:27: note: static
> boost::locale::utf::code_point boost::locale::utf::utf_traits<CharType,
> 1>::decode(Iterator&, Iterator) [with Iterator =
> __gnu_cxx::__normal_iterator<char*, std::basic_string<char> >, CharType =
> char, boost::locale::utf::code_point = unsigned int]
> /usr/include/boost/locale/utf.hpp:193:27: note:   no known conversion for
> argument 1 from ‘std::basic_string<char>::iterator {aka
> __gnu_cxx::__normal_iterator<char*, std::basic_string<char> >}’ to
> ‘__gnu_cxx::__normal_iterator<char*, std::basic_string<char> >&’
> make[2]: *** [test_bin/CMakeFiles/util_test.dir/main.cc.o] Error 1
> make[1]: *** [test_bin/CMakeFiles/util_test.dir/all] Error 2
> make: *** [all] Error 2
>
> The above is my first question, that's about converting one single Unicode
> character.
> Another question is how to decode all Unicode code points from string
> object?
>
>
> 陈抒
> Best regards
> http://blog.csdn.net/sheismylife
>
>
> ------------------------------------------------------------------------------
> Free Next-Gen Firewall Hardware Offer
> Buy your Sophos next-gen firewall before the end March 2013
> and get the hardware for free! Learn more.
> http://p.sf.net/sfu/sophos-d2d-feb
> _______________________________________________
> Cppcms-users mailing list
> Cpp...@li...
> https://lists.sourceforge.net/lists/listinfo/cppcms-users
>
>
>
>
> ------------------------------------------------------------------------------
> Free Next-Gen Firewall Hardware Offer
> Buy your Sophos next-gen firewall before the end March 2013
> and get the hardware for free! Learn more.
> http://p.sf.net/sfu/sophos-d2d-feb
> _______________________________________________
> Cppcms-users mailing list
> Cpp...@li...
> https://lists.sourceforge.net/lists/listinfo/cppcms-users
>
>