Is there a way to convert webpages in cyrillic(windows-1251)into utf-8?
I want to spider 4 different domains and 2 of them är are made in cyrillic and 2 in eng/latin1 and I would like tom save them in the same db by using utf-8.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
No it doesnt convert as it should, get strange characters not cyrillic. Im not used to c++ but when I do these kind of operations with php I have to put in SET NAMES UTF8 when connecting to the db. Im running the spider and mysql 5 on a windows server 2003 eng ver.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
It is true that UTF-8 should be default for eevery server but from my experience I know that it is not as obvious. However this change doesn't affect cache data. Still investigating it
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The spider looks great!
Is there a way to convert webpages in cyrillic(windows-1251)into utf-8?
I want to spider 4 different domains and 2 of them är are made in cyrillic and 2 in eng/latin1 and I would like tom save them in the same db by using utf-8.
openwebspider should automatically convert (if possible) each page to UTF-8! Doesn't it?
Encodings support under openwebspider isn't completed yet!
tnx for reply,
No it doesnt convert as it should, get strange characters not cyrillic. Im not used to c++ but when I do these kind of operations with php I have to put in SET NAMES UTF8 when connecting to the db. Im running the spider and mysql 5 on a windows server 2003 eng ver.
if sqlConnect if think there should be following line added:
mysql_options(rMysql, MYSQL_SET_CHARSET_NAME, "utf8");
It is true that UTF-8 should be default for eevery server but from my experience I know that it is not as obvious. However this change doesn't affect cache data. Still investigating it
how do i add utf8 to code ?
which one ?