From: Zoltan B. <zb...@du...> - 2005-04-11 11:36:32
Attachments:
rlib-1.3.2-odbc-fix.patch
|
Hi, I am experimenting with this report maker software to finally deploy it in a multiplatform environment. I make my (GPLed) program under Fedora Core 3/x86-64 and will compile it for Windows with MingW. My software is special purpose software for business planning and invoicing for a subsidiary of the Hungarian national oil company. I use GTK2 and ODBC as both work under both Linux and Windows. I have first compiled rlib-1.3.2 with the --enable-utf8 option because under Fedora Core everything is UTF-8 encoded and GTK2 also has a strong dependence on UTF-8 even on Windows. The report XML file is UTF-8 encoded, as I typed in some literals into it with Hungarian accented characters. My PostgreSQL database is created with UTF-8 encoding. But UTF-8 in RLIB causes problems. All the literals and the strings that come from the data source are displayed incorrectly. And on the terminal, I got "Encoder was NULL" messages. Here's my code that tries to report something: ------------------------------------------- char *query =3D "some sqlquery..."; r =3D rlib_init(); rlib_add_datasource_odbc(r, "mainsource", "ODBCDSN", "uid", "pwd"); rlib_add_query_as(r, "mainsource", query, "mainquery"); rlib_add_report(r, "rlibtest.xml"); rlib_set_output_format(r, RLIB_FORMAT_PDF); rlib_execute(r); output_type =3D rlib_get_content_type_as_text(r); outbuf =3D rlib_get_output(r); output_len =3D rlib_get_output_length(r); fn =3D open("out.pdf", O_RDWR|O_CREAT|O_TRUNC); if (fn) { write(fn, outbuf, output_len); close(fn); } else printf("open() error: %s\n", strerror(errno)); rlib_free(r); ------------------------------------------- After I recompiled rlib without UTF-8 support and set the connection string in the ODBC DSN to "set client_encoding=3D'ISO-8859-2'" all the string are displayed correctly. I wasn't surprised by the strings it fetched from the query, but the literals in the XML file are still UTF-8. (?) The XML encoding is explicitely UTF-8, as the first line is: <?xml version=3D"1.0" encoding=3D"UTF-8"?> OK then, I will have to set up two ODBC DSNs, one for the GTK2 application, one for the report purposes. Not a big problem, but I would like to use only one. When will the UTF-8 usage be fixed in RLIB? Nudge, nudge. :-) The other, more serious problem is that if I use the ODBC input method in RLIB, the last record is repeated 4 times, somehow overwriting the first 3 records. And if the query gave less than 3 records (i.e. 0, 1 or 2) then RLIB crashes. I.e. my query gave 54 records and for easy comparison, I put an ORDER BY clause into it. Running the query from psql shows that the first three records are missing and the last record is repeated four times in the PDF report produced by RLIB. The same happened with the report in text format. It did not happen when I tried using the native PostgreSQL call in RLIB, so I took a look into the ODBC input method and I found a strange next/previous state and row copying logic. I modified the inputs/odbc/odbc.c and it now uses SQLFetchScroll(hStmt, SQL_FETCH_{FIRST|NEXT|PRIOR|LAST}, 0) instead of just SQLFetch(hStmt). It fixed the problem for me. My ODBC 3.5 Developers Guide says that SQLFetchScroll() is defined in X/OPEN 95 CLI or under ODBC 3.0+. But it's CORE Conformance functionality, so it's kind of expected to work anywhere. Well, except ODBC 2.0 and lower, a.k.a in very old Win16 drivers. Hm, I just checked the www.unixodbc.org site and the MySQL and PostgreSQL drivers' ODBC versions are 2.5 and SQLFetchScroll() definitely works in the unixODBC PostgreSQL driver. I expect the same from the MySQL ODBC driver. When the Windows port of RLIB is ready, ODBC will be the first input method that will work and it must work properly. So please, consider adding my patch to RLIB-1.3.3 or whatever the next version is. I read the policy about patches at http://rlib.sicompos.com I will print, sign and send the copyright assignment but please allow me some days to actually do it. I don't have a printer at present and I am off-work to finish the above mentioned software, for my second work. I will definitely send more patches towards you, to fix issues and warnings that pop up. I already found a solution for the "langinfo.h is missing under MingW", so one compilation error is fixed under Windows. The attached ODBC fix at least allows me to create my reports and test it under Linux. And I have some warning fixes that follows the GTK2 development guideline, e.g. replacing (gpointer *)x with GINT_TO_POINTER(x). The macro works correctly under x86-64, where the explicit pointer conversion prints warnings. Best regards, Zolt=E1n B=F6sz=F6rm=E9nyi |
From: Bob D. <bd...@si...> - 2005-04-11 14:13:09
|
Hello, On Mon, 2005-04-11 at 13:47 +0200, Zoltan Boszormenyi wrote: > Hi, > > I am experimenting with this report maker software to finally > deploy it in a multiplatform environment. I make my (GPLed) program > under Fedora Core 3/x86-64 and will compile it for Windows with MingW. Cool! > > My software is special purpose software for business planning and > invoicing for a subsidiary of the Hungarian national oil company. > > I use GTK2 and ODBC as both work under both Linux and Windows. > > I have first compiled rlib-1.3.2 with the --enable-utf8 option > because under Fedora Core everything is UTF-8 encoded and GTK2 > also has a strong dependence on UTF-8 even on Windows. > The report XML file is UTF-8 encoded, as I typed in some literals > into it with Hungarian accented characters. My PostgreSQL database > is created with UTF-8 encoding. One of the reasons UTF-8 support is half ass in RLIB is because UTF-8 PDF Support is not there at all. > > But UTF-8 in RLIB causes problems. All the literals and the strings > that come from the data source are displayed incorrectly. And on the > terminal, I got "Encoder was NULL" messages. > Here's my code that tries to report something: > > ------------------------------------------- > char *query = "some sqlquery..."; > > r = rlib_init(); > rlib_add_datasource_odbc(r, "mainsource", "ODBCDSN", "uid", "pwd"); > rlib_add_query_as(r, "mainsource", query, "mainquery"); > rlib_add_report(r, "rlibtest.xml"); > rlib_set_output_format(r, RLIB_FORMAT_PDF); > rlib_execute(r); > output_type = rlib_get_content_type_as_text(r); > outbuf = rlib_get_output(r); > output_len = rlib_get_output_length(r); > > fn = open("out.pdf", O_RDWR|O_CREAT|O_TRUNC); > if (fn) { > write(fn, outbuf, output_len); > close(fn); > } else > printf("open() error: %s\n", strerror(errno)); > > rlib_free(r); > ------------------------------------------- > > After I recompiled rlib without UTF-8 support and set > the connection string in the ODBC DSN to > "set client_encoding='ISO-8859-2'" all the string are displayed > correctly. I wasn't surprised by the strings it fetched from the > query, but the literals in the XML file are still UTF-8. (?) > The XML encoding is explicitely UTF-8, as the first line is: You should probably make your encoding "ISO-8859-2" in the xml here in order to make the characters come out correctly. > > <?xml version="1.0" encoding="UTF-8"?> > > OK then, I will have to set up two ODBC DSNs, one for the GTK2 > application, one for the report purposes. Not a big problem, > but I would like to use only one. When will the UTF-8 usage > be fixed in RLIB? Nudge, nudge. :-) If you make RLIB compile in windows with MingGW I'll make sure UTF-8 Support works for you :). Cygwin/mingw and -mno-cygwin have been driving me nuts trying to make windows DLL's > > The other, more serious problem is that if I use the ODBC > input method in RLIB, the last record is repeated 4 times, > somehow overwriting the first 3 records. And if the query > gave less than 3 records (i.e. 0, 1 or 2) then RLIB crashes. > > I.e. my query gave 54 records and for easy comparison, > I put an ORDER BY clause into it. Running the query from psql > shows that the first three records are missing and the > last record is repeated four times in the PDF report produced > by RLIB. The same happened with the report in text format. > > It did not happen when I tried using the native PostgreSQL > call in RLIB, so I took a look into the ODBC input method > and I found a strange next/previous state and row copying > logic. I modified the inputs/odbc/odbc.c and it now uses > SQLFetchScroll(hStmt, SQL_FETCH_{FIRST|NEXT|PRIOR|LAST}, 0) > instead of just SQLFetch(hStmt). It fixed the problem for me. > My ODBC 3.5 Developers Guide says that SQLFetchScroll() is > defined in X/OPEN 95 CLI or under ODBC 3.0+. But it's > CORE Conformance functionality, so it's kind of expected to work > anywhere. Well, except ODBC 2.0 and lower, a.k.a in very old > Win16 drivers. Hm, I just checked the www.unixodbc.org site and > the MySQL and PostgreSQL drivers' ODBC versions are 2.5 and > SQLFetchScroll() definitely works in the unixODBC PostgreSQL > driver. I expect the same from the MySQL ODBC driver. > > When the Windows port of RLIB is ready, ODBC will be the first > input method that will work and it must work properly. Yes. I never use the ODBC stuff and I did it a year ago for some dude as a favor and I never maintained it or really checked that it worked properly ;) > > So please, consider adding my patch to RLIB-1.3.3 or whatever the > next version is. > > I read the policy about patches at http://rlib.sicompos.com > I will print, sign and send the copyright assignment but please > allow me some days to actually do it. I don't have a printer > at present and I am off-work to finish the above mentioned > software, for my second work. Cool! Thanks > > I will definitely send more patches towards you, to fix issues > and warnings that pop up. I already found a solution for the > "langinfo.h is missing under MingW", so one compilation error > is fixed under Windows. The attached ODBC fix at least allows > me to create my reports and test it under Linux. Great! > > And I have some warning fixes that follows the GTK2 development > guideline, e.g. replacing (gpointer *)x with GINT_TO_POINTER(x). > The macro works correctly under x86-64, where the explicit > pointer conversion prints warnings. Thanks! Patch looks nice. I'll apply it as soon as you sign the copyright assignment. - Bob |
From: Zoltan B. <zb...@du...> - 2005-04-12 09:15:23
Attachments:
rlib-1.3.2-fullpatch.patch
|
Bob Doan =EDrta: > Hello, >=20 > On Mon, 2005-04-11 at 13:47 +0200, Zoltan Boszormenyi wrote: >=20 >>Hi, >> >>I am experimenting with this report maker software to finally >>deploy it in a multiplatform environment. I make my (GPLed) program >>under Fedora Core 3/x86-64 and will compile it for Windows with MingW. >=20 >=20 > Cool! Thanks. :-) > One of the reasons UTF-8 support is half ass in RLIB is because UTF-8 > PDF Support is not there at all. I see. > You should probably make your encoding "ISO-8859-2" in the xml here in > order to make the characters come out correctly. Thinking about a bit more, the explicit encoding is correct, as the XML file itself IS UTF-8. libxml uses iconv and converts the strings to the encoding you feed it. I'll leave it as it is. > If you make RLIB compile in windows with MingGW I'll make sure UTF-8 > Support works for you :). Cygwin/mingw and -mno-cygwin have been Thanks. :-) > driving me nuts trying to make windows DLL's :-) > Yes. I never use the ODBC stuff and I did it a year ago for some dude > as a favor and I never maintained it or really checked that it worked > properly ;) ODBC is truly cross-platform, that's why I use it whenever I can. > Patch looks nice. I'll apply it as soon as you sign the copyright > assignment. Thanks. Here's a much bigger patch, that also contains the previous one. What it does: - RLIB now uses the Glib2 GINT_TO_POINTER() and GPOINTER_TO_INT() macros, as they compile without warnings and work everywhere. The explicit casts "(type)var" do not. - Detects the host system at compilation time, so Windows/Cygwin/MingW is distinguished. - Detects ODBC on Windows, includes <windows.h> before <sql.h> and uses -lodbc32 instead of -lodbc. - Some Makefile.am magic, to help RPM packaging. It's not polite to override ${libdir} and ${includedir} in subdirectories... - Detects presence of <langinfo.h> and uses internal implementation of nl_langinfo() if missing. - Detects presence of <sys/resource.h> and disable crash detection if missing. - RLIB already detects presence of <monetary.h> but does not follow that if it's missing then strfmon() will also be missing. I just replaced it with an sprintf(buf, "%.2lf", d) in this case. You may have a better idea, though. :-) - On Cygwin/MingW, it detects older or newer compilers and uses the proper -fnative-struct or -mms-bitfields compiler option. Bitfields in structs are different on Windows. - Compiles everything with -D_GNU_SOURCE. It's mainly because of llabs() declaration is missing if -D_ISO99_SOURCE is not defined but this isn't enough: e.g. strcasecmp() is a GNU extension and -D_GNU_SOURCE is needed to compile cleanly. - gettimeofday() is missing from MingW, the unique filename creation in GD support needs this. I used tempnam() for this purpose if gettimeofday() is not found. I haven't actually tested it, yet. - The previously sent ODBC fix and since <sql.h> is a system header under Windows, its inclusion needs #define WINDOWS_LEAN_AND_MEAN 1 and also #include <windows.h> - Replaced bzero(x,y) with memset(x, 0, y) everywhere. - Deleted #include <sys/resource.h> where not needed. - Defined LC_MESSAGES if it's not already defined. Apparently, it's missing under MingW. - rlib.pc do not list -lcpdf as linkage dependence. OK, so this megapatch enabled to compile RLIB under MingW-3.1. The compilation went without warnings on MingW, I still get some [u]int64 <-> long long size difference warnings under Fedora Core 3/x86-64. long long is 128 bits on 64 bit systems. Unfortunately, under MingW, only static libraries get built, even if I ru= n ./configure --enable-shared I get this whenever the final libs are to be built: libtool: link: warning: undefined symbols not allowed in i686-pc-mingw32=20 shared libraries So if you want to use RLIB under MingW, you have to explicitly link to -lr -lrpdf -lr-odbc for now. And collecting the stuff for RLIB to actually link was quite tricky. I will make a short HOWTO to describe the procedure in some days. So, where's the UTF-8 support in RLIB? ;-) Best regards, Zolt=E1n B=F6sz=F6rm=E9nyi |
From: Zoltan B. <zb...@du...> - 2005-04-12 09:51:49
|
Zoltan Boszormenyi =EDrta: > So if you want to use RLIB under MingW, you have to explicitly link to > -lr -lrpdf -lr-odbc for now. It does not work, since the datasource libs are dlopen()ed. I have to work out how to create the DLLs under MingW... Best regards, Zolt=E1n B=F6sz=F6rm=E9nyi |
From: Bob D. <bd...@si...> - 2005-04-12 13:41:27
|
Nice! Great work so far. Did you have any luck getting a rpdf.dll? That is going to be much easier then getting a rlib.dll. rpdf only depends on GLIB2. Is there any chance you can work with RLIB's CVS? Some stuff has changed since the 1.3.2 release (Especially in output performance) - bob On Tue, 2005-04-12 at 11:26 +0200, Zoltan Boszormenyi wrote: > Bob Doan =C3=ADrta: > > Hello, > >=20 > > On Mon, 2005-04-11 at 13:47 +0200, Zoltan Boszormenyi wrote: > >=20 > >>Hi, > >> > >>I am experimenting with this report maker software to finally > >>deploy it in a multiplatform environment. I make my (GPLed) program > >>under Fedora Core 3/x86-64 and will compile it for Windows with MingW. > >=20 > >=20 > > Cool! >=20 > Thanks. :-) >=20 > > One of the reasons UTF-8 support is half ass in RLIB is because UTF-8 > > PDF Support is not there at all. >=20 > I see. >=20 > > You should probably make your encoding "ISO-8859-2" in the xml here i= n > > order to make the characters come out correctly. >=20 > Thinking about a bit more, the explicit encoding is correct, as > the XML file itself IS UTF-8. libxml uses iconv and converts > the strings to the encoding you feed it. I'll leave it as it is. >=20 > > If you make RLIB compile in windows with MingGW I'll make sure UTF-8 > > Support works for you :). Cygwin/mingw and -mno-cygwin have been >=20 > Thanks. :-) >=20 > > driving me nuts trying to make windows DLL's >=20 > :-) >=20 > > Yes. I never use the ODBC stuff and I did it a year ago for some dud= e > > as a favor and I never maintained it or really checked that it worked > > properly ;) >=20 > ODBC is truly cross-platform, that's why I use it whenever I can. >=20 > > Patch looks nice. I'll apply it as soon as you sign the copyright > > assignment. >=20 > Thanks. Here's a much bigger patch, that also contains the previous one. > What it does: >=20 > - RLIB now uses the Glib2 GINT_TO_POINTER() and GPOINTER_TO_INT() > macros, as they compile without warnings and work everywhere. > The explicit casts "(type)var" do not. > - Detects the host system at compilation time, so Windows/Cygwin/MingW > is distinguished. > - Detects ODBC on Windows, includes <windows.h> before <sql.h> > and uses -lodbc32 instead of -lodbc. > - Some Makefile.am magic, to help RPM packaging. It's not polite to > override ${libdir} and ${includedir} in subdirectories... > - Detects presence of <langinfo.h> and uses internal implementation of > nl_langinfo() if missing. > - Detects presence of <sys/resource.h> and disable crash detection > if missing. > - RLIB already detects presence of <monetary.h> but does not follow > that if it's missing then strfmon() will also be missing. I just > replaced it with an sprintf(buf, "%.2lf", d) in this case. You may > have a better idea, though. :-) > - On Cygwin/MingW, it detects older or newer compilers and uses > the proper -fnative-struct or -mms-bitfields compiler option. > Bitfields in structs are different on Windows. > - Compiles everything with -D_GNU_SOURCE. It's mainly because of > llabs() declaration is missing if -D_ISO99_SOURCE is not defined > but this isn't enough: e.g. strcasecmp() is a GNU extension and > -D_GNU_SOURCE is needed to compile cleanly. > - gettimeofday() is missing from MingW, the unique filename creation > in GD support needs this. I used tempnam() for this purpose if > gettimeofday() is not found. I haven't actually tested it, yet. > - The previously sent ODBC fix and since <sql.h> is a system header > under Windows, its inclusion needs #define WINDOWS_LEAN_AND_MEAN 1 > and also #include <windows.h> > - Replaced bzero(x,y) with memset(x, 0, y) everywhere. > - Deleted #include <sys/resource.h> where not needed. > - Defined LC_MESSAGES if it's not already defined. Apparently, it's > missing under MingW. > - rlib.pc do not list -lcpdf as linkage dependence. >=20 > OK, so this megapatch enabled to compile RLIB under MingW-3.1. > The compilation went without warnings on MingW, I still get some > [u]int64 <-> long long size difference warnings under > Fedora Core 3/x86-64. long long is 128 bits on 64 bit systems. >=20 > Unfortunately, under MingW, only static libraries get built, even if I = run > ./configure --enable-shared >=20 > I get this whenever the final libs are to be built: >=20 > libtool: link: warning: undefined symbols not allowed in i686-pc-mingw3= 2=20 > shared libraries >=20 > So if you want to use RLIB under MingW, you have to explicitly link to > -lr -lrpdf -lr-odbc for now. >=20 > And collecting the stuff for RLIB to actually link was quite tricky. > I will make a short HOWTO to describe the procedure in some days. >=20 > So, where's the UTF-8 support in RLIB? ;-) >=20 > Best regards, > Zolt=C3=A1n B=C3=B6sz=C3=B6rm=C3=A9nyi |
From: Zoltan B. <zb...@du...> - 2005-04-13 04:27:19
|
Bob Doan =C3=ADrta: > Nice! >=20 > Great work so far. >=20 > Did you have any luck getting a rpdf.dll? That is going to be much Unfortunately not. I tried to link it to everything glib2 and gmodule2 are linked to: -lwsock32 -lkernel32 -lmsvcrt -lshlwapi, etc. No success. > easier then getting a rlib.dll. rpdf only depends on GLIB2. >=20 > Is there any chance you can work with RLIB's CVS? Some stuff has > changed since the 1.3.2 release (Especially in output performance) OK, I will do a cvs checkout. Best regards, Zolt=C3=A1n B=C3=B6sz=C3=B6rm=C3=A9nyi |
From: Bob D. <bd...@si...> - 2005-04-13 13:11:38
|
Hym... Back when I was trying the port RLIB to windows I recall libtool not wanting to make a DLL. Dunno if you are having these problems. Did you try to manually create the dll w/ gcc? I also tried to manually make a dll and I don't think I could ever get it to work. I didn't have symbol problems, but I think it kept making a static library. I assume because of your work in windows you have more experience w/ this FYI, I'm working on UTF8 for you. But I have a question. Do you know of any way to put UTF8 in PDF documents? I kinda think its not going to work. We can only go as far as Adobe's fonts support things. RPDF could be made utf8 friendly but it might be pointless unless it can be done inside a PDF document. - bob On Wed, 2005-04-13 at 06:38 +0200, Zoltan Boszormenyi wrote: > Bob Doan =C3=ADrta: > > Nice! > >=20 > > Great work so far. > >=20 > > Did you have any luck getting a rpdf.dll? That is going to be much >=20 > Unfortunately not. I tried to link it to everything glib2 and gmodule2 > are linked to: -lwsock32 -lkernel32 -lmsvcrt -lshlwapi, etc. No success. >=20 > > easier then getting a rlib.dll. rpdf only depends on GLIB2. > >=20 > > Is there any chance you can work with RLIB's CVS? Some stuff has > > changed since the 1.3.2 release (Especially in output performance) >=20 > OK, I will do a cvs checkout. >=20 > Best regards, > Zolt=C3=A1n B=C3=B6sz=C3=B6rm=C3=A9nyi |
From: Zoltan B. <zb...@du...> - 2005-04-13 16:02:59
|
Bob Doan =C3=ADrta: > Hym... Back when I was trying the port RLIB to windows I recall libtool > not wanting to make a DLL. >=20 > Dunno if you are having these problems. >=20 > Did you try to manually create the dll w/ gcc? >=20 > I also tried to manually make a dll and I don't think I could ever get > it to work. I didn't have symbol problems, but I think it kept making = a > static library. I assume because of your work in windows you have more > experience w/ this I am looking at how Xine does all this. Xine has a working Win32 port and it uses dlopen very heavily, too, to open all of its plugins. > FYI, I'm working on UTF8 for you. But I have a question. Do you know > of any way to put UTF8 in PDF documents? I kinda think its not going t= o > work. We can only go as far as Adobe's fonts support things. RPDF > could be made utf8 friendly but it might be pointless unless it can be > done inside a PDF document. I don't know the PDF internals enough to answer that. I think for my purpose it would be enough to accept the strings in UTF-8 and output them in ISO-8859-2. That would require only iconv. Best regards, Zolt=C3=A1n B=C3=B6sz=C3=B6rm=C3=A9nyi |
From: Bob D. <bd...@si...> - 2005-04-13 16:19:24
|
On Wed, 2005-04-13 at 18:14 +0200, Zoltan Boszormenyi wrote: > Bob Doan =C3=ADrta: > > Hym... Back when I was trying the port RLIB to windows I recall libto= ol > > not wanting to make a DLL. > >=20 > > Dunno if you are having these problems. > >=20 > > Did you try to manually create the dll w/ gcc? > >=20 > > I also tried to manually make a dll and I don't think I could ever ge= t > > it to work. I didn't have symbol problems, but I think it kept makin= g a > > static library. I assume because of your work in windows you have mo= re > > experience w/ this >=20 > I am looking at how Xine does all this. Xine has a working Win32 port > and it uses dlopen very heavily, too, to open all of its plugins. RLIB uses g_module stuff so it should not be doing any dlopen's.....????? >=20 > > FYI, I'm working on UTF8 for you. But I have a question. Do you kno= w > > of any way to put UTF8 in PDF documents? I kinda think its not going= to > > work. We can only go as far as Adobe's fonts support things. RPDF > > could be made utf8 friendly but it might be pointless unless it can b= e > > done inside a PDF document. >=20 > I don't know the PDF internals enough to answer that. > I think for my purpose it would be enough to accept the strings > in UTF-8 and output them in ISO-8859-2. That would require only iconv. >=20 > Best regards, > Zolt=C3=A1n B=C3=B6sz=C3=B6rm=C3=A9nyi |
From: Bob D. <bd...@si...> - 2005-04-13 18:08:01
|
> I don't know the PDF internals enough to answer that. > I think for my purpose it would be enough to accept the strings > in UTF-8 and output them in ISO-8859-2. That would require only iconv. Ok.. I'm changing things around a bit from how it worked in RLIB 1.2.0 How does this sound: 1) LIBXML is always utf8 and no work is needed there 2) DataSources could really be in any encoding. I'll add a new api rlib_datasource_set_encoder(r, "data_source", "iconv decoding..."); 3) Paramaters passed in from rlib_add_parameter don't need to be touched because the user can do this on their own (it will be assumed they are in UTF8) In the actual engine RLIB will use g_utf8 functions to do all the string stuff (We have wrappers in util.h.. I'll preform an audit to make sure nothing has fallen out) Outputs: We will need to typically encode the output in order to make things work. so rlib_set_output_encoding(r, "iconv encoding"); In PDF for sure we probably need to do this. In HTML we need to do this **I think** and in the <HEAD> do something like content="text/html;charset=iso-8859-2" I'm using your name as my test bed to make sure this all works. :) - bob |
From: Zoltan B. <zb...@du...> - 2005-04-13 19:29:35
|
Bob Doan =EDrta: >>I don't know the PDF internals enough to answer that. >>I think for my purpose it would be enough to accept the strings >>in UTF-8 and output them in ISO-8859-2. That would require only iconv. >=20 >=20 >=20 > Ok.. I'm changing things around a bit from how it worked in RLIB 1.2.0 >=20 > How does this sound: >=20 > 1) LIBXML is always utf8 and no work is needed there >=20 > 2) DataSources could really be in any encoding. I'll add a new api > rlib_datasource_set_encoder(r, "data_source", "iconv decoding..."); >=20 > 3) Paramaters passed in from rlib_add_parameter don't need to be touche= d > because the user can do this on their own (it will be assumed they are > in UTF8) >=20 > In the actual engine RLIB will use g_utf8 functions to do all the strin= g > stuff (We have wrappers in util.h.. I'll preform an audit to make sure > nothing has fallen out) >=20 > Outputs: >=20 > We will need to typically encode the output in order to make things > work. =20 >=20 > so rlib_set_output_encoding(r, "iconv encoding"); >=20 > In PDF for sure we probably need to do this. In HTML we need to do thi= s > **I think** and in the <HEAD> do something like > content=3D"text/html;charset=3Diso-8859-2" It's fine for me. Just yell when it works, I will test it. In the meantime I think I figured out what's the difference between how xine and rlib does. Black magic, thy name is automake! :-) > I'm using your name as my test bed to make sure this all works. :) OK. :-) Best regards, Zolt=E1n B=F6sz=F6rm=E9nyi |
From: Zoltan B. <zb...@du...> - 2005-04-13 23:03:24
Attachments:
rlib-fullpatch.patch.bz2
|
> Bob Doan =EDrta: >> I'm using your name as my test bed to make sure this all works. :) Here's another two test strings: =E1rv=EDzt=FBr=F5 t=FCk=F6rf=FAr=F3g=E9p =C1RV=CDZT=DBR=D5 T=DCK=D6RF=DAR=D3G=C9P They are the standard test strings in Hungary to see whether a charset conversion or a printer font really contains the lower- and uppercase Hungarian accented characters. And now something completely the same. Ladies and gentlemen! <drumroll> Allow me to introduce RLIB Windows port! </drumroll> Please review the Makefile.am changes. I made librpdf and libr to compile as normal DLLs and libr-odbc and the bindings/* as loadable modules. There may still be rough edges, but that shouldn't be too hard to fix now. This time I bzipped the patch so the mailing list filter lets it through the 40KB limit. It's yet another megapatch but now it's against the current CVS. Cheers! Best regards, Zolt=E1n B=F6sz=F6rm=E9nyi |
From: Bob D. <bd...@si...> - 2005-04-14 00:48:31
|
Thank you! Thank you! Thank you! Have you actually produced a report in windows? ;) I'm going to assume DEF_FILE=librpdf.def if WIN32 def_ldflags=-Wl,--output-def,$(DEF_FILE) -export-dynamic -avoid-version endif Is the magic required to make the windows dll's. What is the license on the nl_langinfo stuff? We should do something nice about strfmon. One would think it's lgpl'd code and if it is we might want to include it. Out of curiousity do you have any experiences packaging WIN32 programs? I think it would be in RLIB's best interest to have a nice installable package (Most windows users probably won't want to go through what you went through in order to get this going ;)) I see you fixed my automake flaws. Were you planning on making a RLIB RPM? Thanks for the strings. I'll test w/ them tomorrow. The bottom line is the patch looks very nice. When do you think you will find yourself near a printer to print/sign the (c) thingie??? :) - bob On Thu, 2005-04-14 at 01:14 +0200, Zoltan Boszormenyi wrote: > > Bob Doan írta: > >> I'm using your name as my test bed to make sure this all works. :) > > Here's another two test strings: > > árvíztűrő tükörfúrógép > ÁRVÍZTŰRŐ TÜKÖRFÚRÓGÉP > > They are the standard test strings in Hungary to see > whether a charset conversion or a printer font really > contains the lower- and uppercase Hungarian accented > characters. > > And now something completely the same. > > Ladies and gentlemen! > > <drumroll> > > Allow me to introduce RLIB Windows port! > > </drumroll> > > Please review the Makefile.am changes. > I made librpdf and libr to compile as normal DLLs > and libr-odbc and the bindings/* as loadable modules. > > There may still be rough edges, but that shouldn't be > too hard to fix now. > > This time I bzipped the patch so the mailing list filter > lets it through the 40KB limit. It's yet another megapatch > but now it's against the current CVS. > > Cheers! > > Best regards, > Zoltán Böszörményi -- Bob Doan <bd...@si...> |
From: Zoltan B. <zb...@du...> - 2005-04-14 04:52:27
|
Bob Doan =C3=ADrta: > Thank you! Thank you! Thank you! >=20 > Have you actually produced a report in windows? ;) Not yet. And there is a bug still in my RLIB work. When "make" is finished, "ldd libr.dll" produces the library list I expect. When I run "make install", libr.dll is relinked and after that, the DLL file is not what ldd expects. I am trying to do something with it. > I'm going to assume=20 >=20 > DEF_FILE=3Dlibrpdf.def > if WIN32 > def_ldflags=3D-Wl,--output-def,$(DEF_FILE) -export-dynamic -avoid-versi= on > endif >=20 > Is the magic required to make the windows dll's. Almost. The "-no-undefined" linker option and the occurence of AC_LIBTOOL_WIN32_DLL before AC_PROG_LIBTOOL does most of the work. The -export-dynamic is something I am not sure is needed but the -avoid-version is because I wanted to too libr.dll instead of libr-1.dll or libr-1-3-3.dll. > What is the license on the nl_langinfo stuff?=20 GPL > We should do something nice about strfmon. One would think it's lgpl'd > code and if it is we might want to include it. >=20 > Out of curiousity do you have any experiences packaging WIN32 programs? Very little. > I think it would be in RLIB's best interest to have a nice installable > package (Most windows users probably won't want to go through what you > went through in order to get this going ;)) >=20 > I see you fixed my automake flaws. Were you planning on making a RLIB > RPM? That was the first thing I did on Linux. :-) I will send my RPM specfile, too. > Thanks for the strings. I'll test w/ them tomorrow. You're welcome. > The bottom line is the patch looks very nice. When do you think you > will find yourself near a printer to print/sign the (c) thingie??? :) Today. I am going back to my primary work. Best regards, Zolt=C3=A1n B=C3=B6sz=C3=B6rm=C3=A9nyi |
From: Bob D. <bd...@si...> - 2005-04-14 13:33:07
|
> > What is the license on the nl_langinfo stuff? > > GPL Are you sure it's GPL?? nl_langinfo stuff in glibc is LGPL. Did you write this or did you get it from somewhere else? - bob |
From: Zoltan B. <zb...@du...> - 2005-04-14 17:45:03
Attachments:
rlib-mingw-3.2-fix.patch.bz2
|
Bob Doan =EDrta: >>>What is the license on the nl_langinfo stuff?=20 >> >>GPL >=20 >=20 > Are you sure it's GPL?? nl_langinfo stuff in glibc is LGPL. Did you > write this or did you get it from somewhere else? Yes, I am certain. This code is part of the development version of libextractor. http://gnunet.org/libextractor/ Here's the patch that went into libextractor after the latest release: http://www.mail-archive.com/gnu...@gn.../msg00184.html Both the header and code implementation files are covered by GPL. Today I mailed the signed copyright asignment via airmail. I hope you receive it in a few days. The attached patch fixes compile errors on MingW 3.2-rc3. In Windows, the directory names must not end with directory separators. And the relink problem on "make install" is fixed if I modify ltmain.sh so the only "need_relink=3Dyes" line is modified to "need_relink=3Dno" after running autogen.sh. Finally I was able to compile my rlibtest.c into rlibtest.exe. It produces a very bad PDF. I set up the same data source on Windows so it accesses the same dataset. Here's the output of my program on Linux: ------------------------------------- type=3D'Content-Type: application/pdf Content-Length: 28575 ' buf=3D0x5a5080 length=3D28575 ------------------------------------- As expected, the file length is 28575 on Linux. And here's the output on Windows: ------------------------------------- type=3D'Content-Type: application/pdf Content-Length: 15998 ' buf=3D01603810 length=3D15998 ------------------------------------- But the file length is not 15998, it's 16240 on Windows. ??? And I tested the generated PDFs, too. Upon opening them in Adobe Acrobat Reader, I get this for the PDF generated on Linux in a messagebox: "Insufficient data for an image" The image is not shown but the report header and records are shown correctly. And for the PDF generated in Windows: "There was an error opening this document." "The file is damaged and could not be repaired." BTW, I get this when I open the Linux PDF in GV: ------------------------------------- **** This file has a corrupted %%EOF marker, or garbage after the %%E= OF. GNU Ghostscript 7.07: Unrecoverable error, exit code 1 Error: /undefined in /BXlevel Operand stack: 28562 9 0 --nostringval-- Creator (RPDF By SICOM Systems)=20 CreationDate (D:20050220143402) Producer (RPDF 1.0) Author=20 (RPDF) Title 4 --dict:9/9(ro)(G)-- RLIB Execution stack: %interp_exit .runexec2 --nostringval-- --nostringval--=20 --nostringval-- 2 %stopped_push --nostringval-- --nostringval--=20 --nostringval-- false 1 %stopped_push 1 3 %oparray_pop=20 1 3 %oparray_pop 1 3 %oparray_pop --nostringval--=20 --nostringval-- --nostringval-- --nostringval-- --nostringval--=20 --nostringval-- --nostringval-- false 1 %stopped_push=20 --nostringval-- %loop_continue --nostringval-- --nostringval--=20 --nostringval-- Dictionary stack: --dict:1061/1123(ro)(G)-- --dict:0/20(G)-- --dict:93/200(L)--=20 --dict:93/200(L)-- --dict:97/127(ro)(G)-- --dict:229/230(ro)(G)--=20 --dict:16/24(L)-- Current allocation mode is local ------------------------------------- XPDF opens the file correctly with no warnings on the console. But for the Windows PDF I get this with XPDF: ------------------------------------- Error (0): PDF file is damaged - attempting to reconstruct xref table... Error (1836): Bad Huffman code in DCT stream ------------------------------------- I got this error from GV for the Windows PDF: ------------------------------------- **** This file has a corrupted %%EOF marker, or garbage after the %%E= OF. GNU Ghostscript 7.07: Unrecoverable error, exit code 1 Error: /syntaxerror in readxref Operand stack: Execution stack: %interp_exit .runexec2 --nostringval-- --nostringval--=20 --nostringval-- 2 %stopped_push --nostringval-- --nostringval--=20 --nostringval-- false 1 %stopped_push 1 3 %oparray_pop=20 1 3 %oparray_pop 1 3 %oparray_pop --nostringval--=20 --nostringval-- --nostringval-- --nostringval-- --nostringval--=20 --nostringval-- Dictionary stack: --dict:1061/1123(ro)(G)-- --dict:0/20(G)-- --dict:93/200(L)-- ------------------------------------- If you need the output files, I send them in separate e-mails. Best regards, Zolt=E1n B=F6sz=F6rm=E9nyi |
From: Bob D. <bd...@si...> - 2005-04-14 18:25:51
|
On Thu, 2005-04-14 at 19:56 +0200, Zoltan Boszormenyi wrote: > Bob Doan =C3=ADrta: > >>>What is the license on the nl_langinfo stuff?=20 > >> > >>GPL > >=20 > >=20 > > Are you sure it's GPL?? nl_langinfo stuff in glibc is LGPL. Did you > > write this or did you get it from somewhere else? >=20 > Yes, I am certain. This code is part of the development version of > libextractor. http://gnunet.org/libextractor/ >=20 > Here's the patch that went into libextractor after the latest release: > http://www.mail-archive.com/gnu...@gn.../msg00184.html Hymmm.. Ok. But something is kinda odd here. GLIBC's nl_langinfo is LGPL. How can extractor relicense it under the GPL? This seems wrong to me. And I hate to have to go into all these details but the (c) stuff is really important to the company. I would rather be coding, trust me ;) > Today I mailed the signed copyright asignment via airmail. > I hope you receive it in a few days. Great >=20 > The attached patch fixes compile errors on MingW 3.2-rc3. > In Windows, the directory names must not end with directory separators. Ok >=20 > And the relink problem on "make install" is fixed if I modify ltmain.sh > so the only "need_relink=3Dyes" line is modified to "need_relink=3Dno" > after running autogen.sh. Perhaps we can automate this???? >=20 > Finally I was able to compile my rlibtest.c into rlibtest.exe. > It produces a very bad PDF. I set up the same data source on Windows > so it accesses the same dataset. >=20 Hym.. Sounds like a RPDF on windows problem. I'll try to build this whole thing tonight on my windows box and see how far I get. Could you send me both PDF's please thanks - bob |
From: Zoltan B. <zb...@du...> - 2005-04-14 18:39:01
|
Bob Doan =C3=ADrta: > On Thu, 2005-04-14 at 19:56 +0200, Zoltan Boszormenyi wrote: > >>Bob Doan =C3=ADrta: >> >>>>>What is the license on the nl_langinfo stuff? >>>> >>>>GPL >>> >>> >>>Are you sure it's GPL?? nl_langinfo stuff in glibc is LGPL. Did you >>>write this or did you get it from somewhere else? >> >>Yes, I am certain. This code is part of the development version of >>libextractor. http://gnunet.org/libextractor/ >> >>Here's the patch that went into libextractor after the latest release: >>http://www.mail-archive.com/gnu...@gn.../msg00184.html > > > Hymmm.. Ok. But something is kinda odd here. GLIBC's nl_langinfo is > LGPL. How can extractor relicense it under the GPL? > > This seems wrong to me. And I hate to have to go into all these detai= ls > but the (c) stuff is really important to the company. I would rather = be > coding, trust me ;) AFAIK you can include LGPL code into GPL, it becomes GPLed. But IANAL. I am good at acronyms. :-) >>Today I mailed the signed copyright asignment via airmail. >>I hope you receive it in a few days. > > > Great > > >>The attached patch fixes compile errors on MingW 3.2-rc3. >>In Windows, the directory names must not end with directory separators. > > > Ok But I don't understand how it compiled on MingW 3.1. >>And the relink problem on "make install" is fixed if I modify ltmain.s= h >>so the only "need_relink=3Dyes" line is modified to "need_relink=3Dno" >>after running autogen.sh. > > > Perhaps we can automate this???? Put this line perl -pi.bak -e "s#need_relink=3Dyes#need_relink=3Dno#" ltmain.sh into autogen.sh after libtoolize. >>Finally I was able to compile my rlibtest.c into rlibtest.exe. >>It produces a very bad PDF. I set up the same data source on Windows >>so it accesses the same dataset. >> > > Hym.. Sounds like a RPDF on windows problem. I'll try to build this And remember, my Linux build is 64-bit, there may be bugs there, too. sizeof(int) =3D=3D 4 && sizeof(long) =3D=3D 8 on 64 bit systems. The PDF I produced on my Fedora Core 3/x86-64 is slightly buggy, too. It seems that XPDF's checks are not as strict as AcroRead's or GV's. > whole thing tonight on my windows box and see how far I get. > > Could you send me both PDF's please I am sending them in private. Best regards, Zolt=C3=A1n B=C3=B6sz=C3=B6rm=C3=A9nyi |
From: Bob D. <bd...@si...> - 2005-04-14 19:16:27
|
> > This seems wrong to me. And I hate to have to go into all these details > > but the (c) stuff is really important to the company. I would rather be > > coding, trust me ;) > > AFAIK you can include LGPL code into GPL, it becomes GPLed. > But IANAL. I am good at acronyms. :-) > My problem is that SICOM could no longer sell the commercial version of RLIB if I put the nl_langinfo stuff in since we won't be able to dual copy right it. I take something was was LGPL but turned to GPL and then turn it back to LGPL. Legally I would have the ask the guy who made the nl_langinfo patch to sign the RLIB (c) form. I feel kinda silly doing this but I could always ask. Other then that I think we are in good shape > perl -pi.bak -e "s#need_relink=yes#need_relink=no#" ltmain.sh > > into autogen.sh after libtoolize. Thanks ;) |
From: Zoltan B. <zb...@du...> - 2005-04-22 20:47:02
Attachments:
rlib-fullpatch-4.patch.bz2
|
Hi, here's my last work against the latest CVS. Two main changes since the last one: - ODBC detection on POSIX systems considers $libdir before the hardcoded check for /usr/lib so on my x86-64 /usr/lib64 is used for ODBC libs. rlib.pc no longer contains -L/usr/lib on my system so 64-bit compiled apps linked against rlib doesn't complain about incompatible libs in /usr/lib - Warning fixes so compiling with "gcc4 -Wall -Werror" succeeds. GCC 4.0 produces many more warnings than previous versions. And guess what? It fixes the crashes on my 64-bit system finally. This was the last thing I could try as valgrind doesn't run on AMD64 yet. So 1.3.3 will mark both Windows and at least Linux/AMD64 ports. Cheers! :-) One thing, though. Are you sure about your change in rlib_set_datasource_encoding()? As it stands in the CVS, line 407 of libsrc/api.c is tif->info.encoder =3D rlib_charencoder_new(encoding, "UTF-8"); rlib_charencoder_new() takes the parameters in (to, from) order. I changed it in my patch to the order I originally proposed: tif->info.encoder =3D rlib_charencoder_new("UTF-8", encoding); So when the datasource is read, the strings are converted from the set encoding to UTF-8 and it's paired with rlib_set_output_encoding() so when outputting the internally UTF-8 strings are converted from UTF-8 to the set output encoding. Best regards, Zolt=E1n B=F6sz=F6rm=E9nyi |
From: Bob D. <bd...@si...> - 2005-04-22 20:59:12
|
Thanks!!! > One thing, though. Are you sure about your change in > rlib_set_datasource_encoding()? > As it stands in the CVS, line 407 of libsrc/api.c is > > tif->info.encoder = rlib_charencoder_new(encoding, "UTF-8"); > > rlib_charencoder_new() takes the parameters in (to, from) order. > > I changed it in my patch to the order I originally proposed: > > tif->info.encoder = rlib_charencoder_new("UTF-8", encoding); > > So when the datasource is read, the strings are converted from the set > encoding to UTF-8 and it's paired with rlib_set_output_encoding() so > when outputting the internally UTF-8 strings are converted from UTF-8 > to the set output encoding. Thats probably correct :) It's Friday ;) I'll review the patch over the weekend. - bob |
From: Zoltan B. <zb...@du...> - 2005-04-23 05:11:37
|
Bob Doan =EDrta: > Thanks!!! BTW most of the GCC4 warnings were harmless, i.e. variable assignments between (pointers to/array of) chars of different signedness. Three of the fixes may have cured my crash. Look at the changes of layout.c:rlib_encode_text() , formatstring.c:rlib_format_string_default() and pcode_op_functions.c:rlib_pcode_operator_str(). These fixed int/long size differences on my 64-bit machine. Any of these may have caused silent (most probably) stack or (less likely) heap corruption that made RLIB crash at very distant and unrelated functions. >>One thing, though. Are you sure about your change in >>rlib_set_datasource_encoding()? >>As it stands in the CVS, line 407 of libsrc/api.c is >> >> tif->info.encoder =3D rlib_charencoder_new(encoding, "UTF-8"); >> >>rlib_charencoder_new() takes the parameters in (to, from) order. >> >>I changed it in my patch to the order I originally proposed: >> >> tif->info.encoder =3D rlib_charencoder_new("UTF-8", encoding); >> >>So when the datasource is read, the strings are converted from the set >>encoding to UTF-8 and it's paired with rlib_set_output_encoding() so >>when outputting the internally UTF-8 strings are converted from UTF-8 >>to the set output encoding. >=20 >=20 > Thats probably correct :) >=20 > It's Friday ;) I understand that. :-) > I'll review the patch over the weekend. Thanks. Best regards, Zolt=E1n B=F6sz=F6rm=E9nyi |
From: Zoltan B. <zb...@du...> - 2005-04-23 20:00:13
Attachments:
rlib-no-gd-fix.patch
|
Hi, I tried to compile on Windows again, this time with no GD installed. This small patch is need to compile without GD. Best regards, Zolt=E1n B=F6sz=F6rm=E9nyi |
From: Zoltan B. <zb...@du...> - 2005-04-14 19:00:54
Attachments:
libxml-2.0.pc
|
Bob Doan =C3=ADrta: > Hym.. Sounds like a RPDF on windows problem. I'll try to build this > whole thing tonight on my windows box and see how far I get. You will need these: GLIB2/GTK2 runtime, Windows installer: http://prdownloads.sourceforge.net/gimp-win/gtk%2B-2.6.4-setup.zip?downlo= ad You will need these below from this page: http://www.gimp.org/~tml/gimp/win32/downloads.html Just extract them under C:\MingW. http://www.gimp.org/~tml/gimp/win32/glib-dev-2.4.7.zip http://www.gimp.org/~tml/gimp/win32/pkgconfig-0.15.zip http://www.gimp.org/~tml/gimp/win32/libiconv-1.9.1.bin.woe32.zip But don't use the zlib from the above page, use these instead: ZLIB DLL(s): http://prdownloads.sourceforge.net/gnuwin32/zlib-1.2.2-bin.zip?download ZLIB headers, static lib and DLL stub: http://prdownloads.sourceforge.net/gnuwin32/zlib-1.2.2-lib.zip?download You will also need libxml2 (this link listed on the Gimp/Win32 page) http://www.zlatkovic.com/projects/libxml/binaries.html The libxml2 ZIP has a root directory, libxml2-2.6.19.win32. Move the bin, include and lib directories from there to C:\MingW, too. You will also need the attached pkgconfig file so RLIB finds libxml2. The libxml2 ZIP does not contain the corresponding .pc file. Put it into C:\MingW\lib\pkgconfig You may try GD, too: http://gnuwin32.sourceforge.net/packages/gd.htm I successfully linked RLIB to GD under MingW 3.1 on my machine, but I haven't installed GD on the other machine with MingW 3.2-rc3. Could it be the source of the PDF problem? If you have a problem just tell me. Best regards, Zolt=C3=A1n B=C3=B6sz=C3=B6rm=C3=A9nyi |
From: Zoltan B. <zb...@du...> - 2005-04-13 16:42:51
|
Bob Doan =C3=ADrta: > RLIB uses g_module stuff so it should not be doing any > dlopen's.....????? I was talking about how xine compiles its' shared libs and plugins, because under Windows even rlib/rpdf/test does not link because librpdf.a is empty. Somehow the non-static functions aren't exported. Best regards, Zolt=C3=A1n B=C3=B6sz=C3=B6rm=C3=A9nyi |