mdbtools-dev

[mdb-dev] More patches soon ...

[Nirgal V. <con...@ni...>]

Hello MDB folks

Attached are a bunch of new patches for the github repository. These are work in progress, and they assume you first merged "schema-indexes".

"ole" provide a new function called mdb_ole_read_full to easily read a full ole field, with all the chunks.
I already described it yesterday.
IMHO, the whole LVAL system should be rewrote to provide and use a unique interface. This would change the API.
How bad would that be?
Do you know any real usage for libmdb other than gmdb2 and command line tools provided?

"props" finaly works:
catalog.c read the props, table.c assign them.
prtable have been change to display them.
I want to make sure I have some feed back about ole patch before proposing it for merge...
And that code still need some cleaning.
\o/

You'll not want to merge "schema2" yet for sure, as it assumes you use postgres backend.
I include it nevertheless so you can see what we're about to get:
- NOT NULL constrainst
- Not empty constraints
- Columns comments
And soon, hopefully:
- Short date/Long date behaviour
- Default values

Attached is also an example of output...

Cheers!

[mdb-dev] More patches soon ...

[Dmitry N. <luc...@ma...>]

Hi, Nirgal Vourgère!

You job is greate. Thanks.
I found your git-repository and compiled worked and new functionality mdbtools.
But when I would build the RPM package for my distr (Mandriva 2010), the build CFLAGS of:
CFLAGS="-O2 -g -pipe -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -DHAVE_ICONV -DSQL"
give the errors in sources:

gcc -DHAVE_CONFIG_H -I. -I../../include  -I../../include -DORBIT2=1 -pthread -I/usr/include/gtk-2.0 -I/usr/lib64/gtk-2.0/include -I/usr/include/atk-1.0 -I/usr/include/cairo -I/usr/include/pango-1.0 -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include/pixman-1 -I/usr/include/freetype2 -I/usr/include/libpng12 -I/usr/include/libglade-2.0 -I/usr/include/libxml2 -I/usr/include/libgnomeui-2.0 -I/usr/include/libart-2.0 -I/usr/include/gconf/2 -I/usr/include/gnome-keyring-1 -I/usr/include/libgnome-2.0 -I/usr/include/libbonoboui-2.0 -I/usr/include/libgnomecanvas-2.0 -I/usr/include/gnome-vfs-2.0 -I/usr/lib64/gnome-vfs-2.0/include -I/usr/include/orbit-2.0 -I/usr/include/dbus-1.0 -I/usr/lib64/dbus-1.0/include -I/usr/include/libbonobo-2.0 -I/usr/include/bonobo-activation-2.0 -I/usr/include/gail-1.0   -DDATADIR="\"/usr/local/share\"" -DGMDB_ICONDIR="\"/usr/local/share/gmdb/glade/\"" -DGMDB_GLADEDIR="\"/usr/local/share/gmdb/glade/\""    -O2 -g -pipe -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -DHAVE_ICONV -DSQL -DHAVE_ICONV -DSQL -MT main2.o -MD -MP -MF .deps/main2.Tpo -c -o main2.o main2.c
main2.c: In function ‘gmdb_about_cb’:
main2.c:75: warning: passing argument 1 of ‘gtk_show_about_dialog’ from incompatible pointer type
/usr/include/gtk-2.0/gtk/gtkaboutdialog.h:67: note: expected ‘struct GtkWindow *’ but argument is of type ‘struct GtkWidget *’
main2.c: In function ‘gmdb_help_cb’:
main2.c:105: error: format not a string literal and no format arguments
make[3]: *** [main2.o] Error 1

This errors take the CFLAGS="-Werror=format-security", without this flags compiled as is.

Also give errors in build mdbtools with unixODBC:
Making all in odbc
make[2]: Вхожу у каталог "/home/luckas/rpm/BUILD/mdbtools-0.6pre1.patched/src/odbc"
/bin/sh ../../libtool --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I../../include  -I../../include -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include   -I/usr/include   -O2 -g -pipe -Wformat -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -DHAVE_ICONV -DSQL -DHAVE_ICONV -DSQL -DUNIXODBC -MT odbc.lo -MD -MP -MF .deps/odbc.Tpo -c -o odbc.lo odbc.c
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I../../include -I../../include -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include -O2 -g -pipe -Wformat -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -DHAVE_ICONV -DSQL -DHAVE_ICONV -DSQL -DUNIXODBC -MT odbc.lo -MD -MP -MF .deps/odbc.Tpo -c odbc.c  -fPIC -DPIC -o .libs/odbc.o
odbc.c:223: error: expected declaration specifiers or ‘...’ before ‘SQLROWOFFSET’
odbc.c:224: error: expected declaration specifiers or ‘...’ before ‘SQLROWSETSIZE’
odbc.c:220: error: conflicting types for ‘SQLExtendedFetch’
/usr/include/sqlext.h:1857: note: previous declaration of ‘SQLExtendedFetch’ was here
odbc.c: In function ‘SQLExtendedFetch’:
odbc.c:236: error: ‘pcrow’ undeclared (first use in this function)
odbc.c:236: error: (Each undeclared identifier is reported only once
odbc.c:236: error: for each function it appears in.)
odbc.c: At top level:
odbc.c:1485: error: expected declaration specifiers or ‘...’ before ‘SQLROWCOUNT’
odbc.c:1482: error: conflicting types for ‘SQLSetStmtOption’
/usr/include/sql.h:796: note: previous declaration of ‘SQLSetStmtOption’ was here
make[2]: *** [odbc.lo] Error 1

My configurations:
Mandriva 2010.0, x86_64, mdbtools-git-2011-02-28, lib64gtk+2.0_0-devel-2.18.9-1.2mdv2010.0.x86_64, lib64unixODBC-devel-2.2.14-8mdv2010.0.x86_64

Also I would like propouse a some thinks:
1) need to fix maximum varchar (255) for mysql
2) need to fix the procedure of sanitize (-S) the dump of MDB files. You already changes dates to utf-8 codepage, and the more "dummy"
MS windows users and "admins" used the non ASCII symbols for names of tables and columns in the MDB-files, like as cyrilic :(
May be the procedure of sanitize can replace the symbols in names as transliterate (need check the match of names and length?)
Now, the replaced only spaces in names as "_", and .... the other non-english symbols taking "____________________________" as name of table.

Good Luck, Dmitry

[mdb-dev] (no subject)

[CCOR58 <cat...@gm...>]

A bit off topic, but can anyone point me to a decent tutorial type site 
for linux sound architecture/setup as well as one for
finding comparable coding for converting msaccess/mdb to OpenOffice Base 
(hsql).

I installed a streaming audio automation control package and sound quit. 
Setup to use alsa and jack. 'notifications' grab sound
and prevent jackd from starting and without jackd I am unable to run the 
streaming app or patch otrher media players to outputs.
OS is openSuSE 11.2 with 1.0 GB RAM and 120 GB HD on HP Pavilion a520n.

I want to be able to use Open Office Base to generate summary reports 
from the streaming app's log data, that I know I can do in access, hence 
the need for the qqq on mdb info.

Thanks, any tips or pointers to decent tutorial site would be 
appreciated. Most forums are debian/ubuntu based and don't really relate 
easily
to SuSE syntax or command ref sets.

Andy

[mdb-dev] [patch] Re: More patches soon ...

[Nirgal V. <con...@ni...>]

On Friday 04 March 2011 18:29:22 Dmitry Nikitin wrote:
> Hi, Nirgal Vourgère!
> You job is greate. Thanks.
> I found your git-repository and compiled worked and new functionality mdbtools.

I don't have a git repository. Brian Burns is the author.
The latest repository is located at https://github.com/brianb/mdbtools.git
I occasionaly update some unoffical debian packages at http://www.nirgal.com/mdbtools/

> But when I would build the RPM package for my distr (Mandriva 2010), the build CFLAGS of:
> CFLAGS="-O2 -g -pipe -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -DHAVE_ICONV -DSQL"
> give the errors in sources:
> 
> gcc -DHAVE_CONFIG_H -I. -I../../include  -I../../include -DORBIT2=1 -pthread -I/usr/include/gtk-2.0 -I/usr/lib64/gtk-2.0/include -I/usr/include/atk-1.0 -I/usr/include/cairo -I/usr/include/pango-1.0 -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include/pixman-1 -I/usr/include/freetype2 -I/usr/include/libpng12 -I/usr/include/libglade-2.0 -I/usr/include/libxml2 -I/usr/include/libgnomeui-2.0 -I/usr/include/libart-2.0 -I/usr/include/gconf/2 -I/usr/include/gnome-keyring-1 -I/usr/include/libgnome-2.0 -I/usr/include/libbonoboui-2.0 -I/usr/include/libgnomecanvas-2.0 -I/usr/include/gnome-vfs-2.0 -I/usr/lib64/gnome-vfs-2.0/include -I/usr/include/orbit-2.0 -I/usr/include/dbus-1.0 -I/usr/lib64/dbus-1.0/include -I/usr/include/libbonobo-2.0 -I/usr/include/bonobo-activation-2.0 -I/usr/include/gail-1.0   -DDATADIR="\"/usr/local/share\"" -DGMDB_ICONDIR="\"/usr/local/share/gmdb/glade/\"" -DGMDB_GLADEDIR="\"/usr/local/share/gmdb/glade/\""    -O2 -g -pipe -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -DHAVE_ICONV -DSQL -DHAVE_ICONV -DSQL -MT main2.o -MD -MP -MF .deps/main2.Tpo -c -o main2.o main2.c
> main2.c: In function ‘gmdb_about_cb’:
> main2.c:75: warning: passing argument 1 of ‘gtk_show_about_dialog’ from incompatible pointer type
> /usr/include/gtk-2.0/gtk/gtkaboutdialog.h:67: note: expected ‘struct GtkWindow *’ but argument is of type ‘struct GtkWidget *’
> main2.c: In function ‘gmdb_help_cb’:
> main2.c:105: error: format not a string literal and no format arguments
> make[3]: *** [main2.o] Error 1
> 
> This errors take the CFLAGS="-Werror=format-security", without this flags compiled as is.

I think it's not the proper way to fix that. It's a very interresting warning/error message my compiler doesn't show. A pity.

I think it's the same problem than using "printf(message)" when you have no idea whether the message might contain "%".

You should use puts(message) or printf("%s", message) in these cases.

Attached is a patch. Does it help?


> Also give errors in build mdbtools with unixODBC:
> Making all in odbc
> make[2]: Вхожу у каталог "/home/luckas/rpm/BUILD/mdbtools-0.6pre1.patched/src/odbc"
> /bin/sh ../../libtool --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I../../include  -I../../include -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include   -I/usr/include   -O2 -g -pipe -Wformat -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -DHAVE_ICONV -DSQL -DHAVE_ICONV -DSQL -DUNIXODBC -MT odbc.lo -MD -MP -MF .deps/odbc.Tpo -c -o odbc.lo odbc.c
> libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I../../include -I../../include -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include -O2 -g -pipe -Wformat -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -DHAVE_ICONV -DSQL -DHAVE_ICONV -DSQL -DUNIXODBC -MT odbc.lo -MD -MP -MF .deps/odbc.Tpo -c odbc.c  -fPIC -DPIC -o .libs/odbc.o
> odbc.c:223: error: expected declaration specifiers or ‘...’ before ‘SQLROWOFFSET’
> odbc.c:224: error: expected declaration specifiers or ‘...’ before ‘SQLROWSETSIZE’
> odbc.c:220: error: conflicting types for ‘SQLExtendedFetch’
> /usr/include/sqlext.h:1857: note: previous declaration of ‘SQLExtendedFetch’ was here
> odbc.c: In function ‘SQLExtendedFetch’:
> odbc.c:236: error: ‘pcrow’ undeclared (first use in this function)
> odbc.c:236: error: (Each undeclared identifier is reported only once
> odbc.c:236: error: for each function it appears in.)
> odbc.c: At top level:
> odbc.c:1485: error: expected declaration specifiers or ‘...’ before ‘SQLROWCOUNT’
> odbc.c:1482: error: conflicting types for ‘SQLSetStmtOption’
> /usr/include/sql.h:796: note: previous declaration of ‘SQLSetStmtOption’ was here
> make[2]: *** [odbc.lo] Error 1
> 
> My configurations:
> Mandriva 2010.0, x86_64, mdbtools-git-2011-02-28, lib64gtk+2.0_0-devel-2.18.9-1.2mdv2010.0.x86_64, lib64unixODBC-devel-2.2.14-8mdv2010.0.x86_64

Attached is another fix for that one.

> Also I would like propouse a some thinks:
> 1) need to fix maximum varchar (255) for mysql

I read limit is 65535 since mysql 5.0.3.
http://dev.mysql.com/doc/refman/5.0/en/char.html

> 2) need to fix the procedure of sanitize (-S) the dump of MDB files. You already changes dates to utf-8 codepage, and the more "dummy"
> MS windows users and "admins" used the non ASCII symbols for names of tables and columns in the MDB-files, like as cyrilic :(
> May be the procedure of sanitize can replace the symbols in names as transliterate (need check the match of names and length?)
> Now, the replaced only spaces in names as "_", and .... the other non-english symbols taking "____________________________" as name of table.

Sanitize is really buggy.
I would recommand removing it from the code.
All backends should now handle the non ascii characters in table/colum names correctly, quoting them.
Do you really need -S?
If using Jet3, don't forget to set MDB_JET3_CHARSET as described in mdb-schema manual and HACKING file.

Re: [mdb-dev] [patch] Re: More patches soon ...

[Dmitry N. <luc...@ma...>]

Hi!

> Attached is a patch. Does it help?
> 

Probe the patch "g_warning.diff", but have'nt success, give next errors:  
$ make
....
gcc -DHAVE_CONFIG_H -I. -I../../include  -I../../include -pthread -DORBIT2=1 -I/usr/include/gtk-2.0 -I/usr/lib/gtk-2.0/include -I/usr/include/atk-1.0 -I/usr/include/cairo -I/usr/include/pango-1.0 -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/include/pixman-1 -I/usr/include/freetype2 -I/usr/include/libpng12 -I/usr/include/libglade-2.0 -I/usr/include/libxml2 -I/usr/include/libgnomeui-2.0 -I/usr/include/libart-2.0 -I/usr/include/gconf/2 -I/usr/include/gnome-keyring-1 -I/usr/include/libgnome-2.0 -I/usr/include/libbonoboui-2.0 -I/usr/include/libgnomecanvas-2.0 -I/usr/include/gnome-vfs-2.0 -I/usr/lib/gnome-vfs-2.0/include -I/usr/include/orbit-2.0 -I/usr/include/dbus-1.0 -I/usr/lib/dbus-1.0/include -I/usr/include/libbonobo-2.0 -I/usr/include/bonobo-activation-2.0 -I/usr/include/gail-1.0   -DDATADIR="\"/usr/local/share\"" -DGMDB_ICONDIR="\"/usr/local/share/gmdb/glade/\"" -DGMDB_GLADEDIR="\"/usr/local/share/gmdb/glade/\""    -O2 -g -pipe -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -DHAVE_ICONV -DSQL -DHAVE_ICONV -DSQL -MT table_export.o -MD -MP -MF .deps/table_export.Tpo -c -o table_export.o table_export.c
table_export.c: In function ‘gmdb_table_export_button_cb’:
table_export.c:198: error: format not a string literal and no format arguments
table_export.c:204: error: format not a string literal and no format arguments
table_export.c:209: error: format not a string literal and no format arguments
table_export.c:214: error: format not a string literal and no format arguments
make[3]: *** [table_export.o] Error 1
make[3]: Залишаю каталог "/home/luckas/SVN/MDB/mdbtools.git/src/gmdb2"

list of errors lines:
[luckas@wind gmdb2]$ cat -n table_export.c|grep 198
   198                          if (i>0) fprintf(outfile,delimiter);
[luckas@wind gmdb2]$ cat -n table_export.c|grep 204
   204          if (need_headers) fprintf(outfile,lineterm);
[luckas@wind gmdb2]$ cat -n table_export.c|grep 209
   209                          if (i>0) fprintf(outfile,delimiter);
[luckas@wind gmdb2]$ cat -n table_export.c|grep 214
   214                  fprintf(outfile,lineterm);

Previous warnings also presented.

The odbc patch worked also without  CFLAGS="-Werror=format-security"
But with in - give error:
gcc -DHAVE_CONFIG_H -DLOCALEDIR=\"/usr/local/share/locale\" -I. -I../../include  -I../../include -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include     -O2 -g -pipe -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -DHAVE_ICONV -DSQL -DHAVE_ICONV -DSQL -DUNIXODBC -MT mdb-export.o -MD -MP -MF .deps/mdb-export.Tpo -c -o mdb-export.o mdb-export.c
mdb-export.c: In function ‘main’:
mdb-export.c:208: error: format not a string literal and no format arguments
make[2]: *** [mdb-export.o] Error 1

Others warnings add in the attched file...

> I don't have a git repository. Brian Burns is the author.
> The latest repository is located at https://github.com/brianb/mdbtools.git

Sorry :) - for Brian. 

> > But when I would build the RPM package for my distr (Mandriva 2010), the
> build CFLAGS of:
> > CFLAGS="-O2 -g -pipe -Wformat -Werror=format-security
> -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
> --param=ssp-buffer-size=4 -DHAVE_ICONV -DSQL"
> > give the errors in sources:
> > main2.c: In function ‘gmdb_about_cb’:
> > main2.c:75: warning: passing argument 1 of ‘gtk_show_about_dialog’ from
> incompatible pointer type
> > /usr/include/gtk-2.0/gtk/gtkaboutdialog.h:67: note: expected ‘struct
> GtkWindow *’ but argument is of type ‘struct GtkWidget *’
> > main2.c: In function ‘gmdb_help_cb’:
> > main2.c:105: error: format not a string literal and no format arguments
> > make[3]: *** [main2.o] Error 1
> > 
> > This errors take the CFLAGS="-Werror=format-security", without this flags
> compiled as is.
> 
> I think it's not the proper way to fix that. It's a very interresting
> warning/error message my compiler doesn't show. A pity.
> 
> I think it's the same problem than using "printf(message)" when you have no
> idea whether the message might contain "%".
> 
> You should use puts(message) or printf("%s", message) in these cases.
> 
> > Also give errors in build mdbtools with unixODBC:
> > My configurations:
> > Mandriva 2010.0, x86_64, mdbtools-git-2011-02-28,
> lib64gtk+2.0_0-devel-2.18.9-1.2mdv2010.0.x86_64,
> lib64unixODBC-devel-2.2.14-8mdv2010.0.x86_64

> > Also I would like propouse a some thinks:
> > 1) need to fix maximum varchar (255) for mysql
> 
> I read limit is 65535 since mysql 5.0.3.
> http://dev.mysql.com/doc/refman/5.0/en/char.html

Oh, yes!  But I think it will be better for compatibillity (max 510 for MSAccess- max 255 utf8 for mysql), 
And somebody have old comp's whit old mysql.  

> > 2) need to fix the procedure of sanitize (-S) the dump of MDB files. You
> already changes dates to utf-8 codepage, and the more "dummy"
> > MS windows users and "admins" used the non ASCII symbols for names of tables
> and columns in the MDB-files, like as cyrilic :(
> > May be the procedure of sanitize can replace the symbols in names as
> transliterate (need check the match of names and length?)
> > Now, the replaced only spaces in names as "_", and .... the other
> non-english symbols taking "____________________________" as name of table.

> Sanitize is really buggy.
> I would recommand removing it from the code.
> All backends should now handle the non ascii characters in table/colum names
> correctly, quoting them.
> Do you really need -S?

I think- Yes, "-S" its good idea for package!!! (non-ASCII symbols in variables, names of tables and other is bad practise in developing.
  
> If using Jet3, don't forget to set MDB_JET3_CHARSET as described in mdb-schema
> manual and HACKING file.

I tested JET4 file with ciryllic symbols, spaces, " ' " and others :((( in names of tables, fields.

Good Luck, Dmitry

[mdb-dev] [patch] Re: -Werror=format-security

[Nirgal V. <con...@ni...>]

On Monday 07 March 2011 16:57:09 Dmitry Nikitin wrote:
> > Attached is a patch. Does it help?
> Probe the patch "g_warning.diff", but have'nt success, give next errors:
> (...)

Hello Dimitry

Attached is another more general patch about printf(string_that_may_contain_percent) problems.
It replaces g_warning.diff
Project now compiles OK here with gcc option -Werror=format-security
Can you test it?

Re: [mdb-dev] [patch] Re: -Werror=format-security

[Dmitry N. <luc...@ma...>]

Hi! Nirgal, 

> > > Attached is a patch. Does it help?
> > Probe the patch "g_warning.diff", but have'nt success, give next errors:
> > (...)
> 
> Hello Dimitry
> 
> Attached is another more general patch about printf(string_that_may_contain_percent) problems.
> It replaces g_warning.diff
> Project now compiles OK here with gcc option -Werror=format-security
> Can you test it?

Yes, tested and process of compilation passed without errors.
Thanks for quickly respond.

[mdb-dev] Re: Sanitizing table/column names

[Nirgal V. <con...@ni...>]

On Monday 07 March 2011 16:57:09 Dmitry Nikitin wrote:
> (...)
> > > 2) need to fix the procedure of sanitize (-S) the dump of MDB files. You
> > already changes dates to utf-8 codepage, and the more "dummy"
> > > MS windows users and "admins" used the non ASCII symbols for names of tables
> > and columns in the MDB-files, like as cyrilic :(
> > > May be the procedure of sanitize can replace the symbols in names as
> > transliterate (need check the match of names and length?)
> > > Now, the replaced only spaces in names as "_", and .... the other
> > non-english symbols taking "____________________________" as name of table.
> 
> > Sanitize is really buggy.
> > I would recommand removing it from the code.
> > All backends should now handle the non ascii characters in table/colum names
> > correctly, quoting them.
> > Do you really need -S?
> 
> I think- Yes, "-S" its good idea for package!!! (non-ASCII symbols in variables, names of tables and other is bad practise in developing.
>   
> > If using Jet3, don't forget to set MDB_JET3_CHARSET as described in mdb-schema
> > manual and HACKING file.
> 
> I tested JET4 file with ciryllic symbols, spaces, " ' " and others :((( in names of tables, fields.

As you pointed out, sanitizing is really tricky.
I tested a BIG5 encoded jet3 database, and just like you, ended up with conflicts because I had a lot of columns with the same name "__"!
What do you suggest we should do? Rename every thing into table1, table2, table3...?? Remember we now export relationships. I suspect we might export relationships on linked tables in the future. Renaming these will really be messy.

I do agree having ideograms as column names, or even spaces is not very fun.
But in my experience, export does work with these -- I tested postgres backend only I'm afraid.

*I think fixing database schemas is beyond the scope of mdbtools.*

mdbtools will convert your poorly design access schema into a poorly design postgres schema for example. It will keep spaces, cyrilic, ideogram or whatever. It will produce a working unicode database. You just need to properly quote your 
names in your queries, that's all. I've read mysql uses ` character.

IMHO, you should use another tool if you want to rename your columns.

Re: [mdb-dev] Sanitizing table/column names

[Dmitry N. <luc...@ma...>]

Hi, Nirgal!

Some others difficults has in the process to import scheme in MySQL DB.
This errors produce situation to ignore first command of mysql like as "DROP
DATABASE IF EXIST" and as result INSERT dates into old table.
The 2 lines of comment in header of scheme formed in src/libmdb/backend.c
has "-------------------------------------------------------------------------------", wich not ignored
by mysql. I propose replace there one symbol to space, as "-- ---------------------------"
This lines will not reproduce the warnings in import scheme to MySQL
Attach the small patch do this: mdbtools-mysql-warning.patch

My conf: Mandriva 2010 x86_64, mysql-5.1.42-0.5mdv2010.0 mdbtools-0.6pre1-git-2011-02-21 with
format-security and odbctypes patches.

Nirgal Vourgère <con...@ni...> wrote:
> On Monday 07 March 2011 16:57:09 Dmitry Nikitin wrote:
> > > > 2) need to fix the procedure of sanitize (-S) the dump of MDB files. You
> > > already changes dates to utf-8 codepage, and the more "dummy"
> > > > MS windows users and "admins" used the non ASCII symbols for names of tables
> > > and columns in the MDB-files, like as cyrilic :(
> > > > May be the procedure of sanitize can replace the symbols in names as
> > > transliterate (need check the match of names and length?)
> > > > Now, the replaced only spaces in names as "_", and .... the other
> > > non-english symbols taking "____________________________" as name of table.
> > 
> > > Sanitize is really buggy.
> > > I would recommand removing it from the code.
> > > All backends should now handle the non ascii characters in table/colum names
> > > correctly, quoting them.
> > > Do you really need -S?
> > 
> > I think- Yes, "-S" its good idea for package!!! (non-ASCII symbols in variables, names of tables and other is bad practise in developing.
> > I tested JET4 file with ciryllic symbols, spaces, " ' " and others :((( in names of tables, fields.
> 
> As you pointed out, sanitizing is really tricky.
> I tested a BIG5 encoded jet3 database, and just like you, ended up with conflicts because I had a lot of columns with the same name "__"!
> What do you suggest we should do? Rename every thing into table1, table2, table3...?? Remember we now export relationships. I 
> suspect we might export relationships on linked tables in the future. Renaming these will really be messy.
> I do agree having ideograms as column names, or even spaces is not very fun.
> But in my experience, export does work with these -- I tested postgres backend only I'm afraid.
> *I think fixing database schemas is beyond the scope of mdbtools.*
> mdbtools will convert your poorly design access schema into a poorly design postgres schema for example. It will keep spaces, cyrilic, ideogram or whatever. It will produce a working unicode database. You just need to properly quote your 
> names in your queries, that's all. I've read mysql uses ` character.
> IMHO, you should use another tool if you want to rename your columns.

Yes, I use sed in shell script to automatic rename the badlly names in scheme and dump. But it's need more time for testing and make
good scheme to rename.
I propouse (as temporary) leave in the code sanitize for replace the scaces by underscore symbol and should comment the code, wich replace others (non literal-numerical) symbols. So, in future will continued develop the procedure to replace non-ASCII symbols in the transliterated. May be, create two arrays of letters ASCII and matched for character transliteration, such as in Ukrainian:
"Ніргал Вургере" => "nirgal_vourgere"

Good Luck, Dmitry