mdbtools-dev

[mdb-dev] Segmentation fault in 0.7_rc1

[Vitali <ml...@vr...>]

Hello,
 
with the current version I get this error:
server:~/test # mdb-sql Museum.mdb
1 => SELECT Name as museum_name FROM Museum
2 => go
Error at Line : syntax error near as
syntax error near asSegmentation fault
 
 I compiled mdbtools on Suse Linux Enterprise Server 11 SP1 X64
How can I debug this problem?
 
 
Regards
Vitali

[mdb-dev] 0.7_rc1 release

[Brian B. <br...@br...>]

Hi all,

I've tagged 0.7_rc1 in the repository.

You can get the tarball version here:

https://github.com/brianb/mdbtools/zipball/0.7_rc1

Let me know if there are any problems!

Brian

Re: [mdb-dev] mdb-export encoding issue

[Jakob E. <jab...@gm...>]

In Jet4, all text is stored using the UCS2 encoding. However, Access uses a special trick to reduce storage requirements: ASCII characters are stored as single byte characters, and all others are stored as two byte characters. A null byte is used to switch between the two encoding methods. In your field, such a NULL byte will appear between the ASCII text and the hebrew characters. Apparently, this NULL byte causes mdb-export to believe it has reached the end of the string. This shouldn't happen.

Which version of mdb-export are you using? There are a lot of old versions "in the wild". It is best if you compile the most current version from github yourself so you can ensure you are using a recent version.

I'd be glad to try reading your file with the newest version, if you are interested just send me a copy of the database to my private email address.

Best regards,
Jakob


On 23.09.2011, at 08:03, אריאל קלגסבלד Ariel Klagsbald wrote:

>  
>  
> 2011/9/21 Nirgal <con...@ni...>:
> > Jet4 always use unicode (UCS2) internally.
> >
> > Output should be utf-8, unless you set env var MDBICONV (there is no underscore).
> >
>   I tried again and again, and MDBICONV seems to have no effect (a strange fact by itself). Any more ideas please? Maybe I'm wrong, and it isn't an encoding problem. Can something else cause emd-export to ignore half of the field?
>  
>  
> [ak@ch ~/]$ setenv MDBICONV UTF-8
> [ak@ch ~/]$ mdb-export -QHd^ WebStructure.mdb FilePaths | grep '^130419\^'
> 130419^9817^0113-20000101-010645-45_^Hebrew|HWomen|HinuchYeladimShlomBayit|HinuchYeladim|R0113-5|R0113-2^01/01/00 00:00:00^84^20223203^0113^1^0^45 ��ז� �ט��י ��ט��, ��' ��ך, ךי'ב^0^0^0^0
> [ar@ch ~/]$ setenv MDBICONV iso-8859-1
> [ar@ch ~/]$ mdb-export -QHd^ WebStructure.mdb FilePaths | grep '^130419\^'
> 130419^9817^0113-20000101-010645-45_^Hebrew|HWomen|HinuchYeladimShlomBayit|HinuchYeladim|R0113-5|R0113-2^01/01/00 00:00:00^84^20223203^0113^1^0^45 ��ז� �ט��י ��ט��, ��' ��ך, ךי'ב^0^0^0^0
> [ar@ch ~/]$ setenv MDBICONV nothingatall
> [ar@ch ~/]$ mdb-export -QHd^ WebStructure.mdb FilePaths | grep '^130419\^'
> 130419^9817^0113-20000101-010645-45_^Hebrew|HWomen|HinuchYeladimShlomBayit|HinuchYeladim|R0113-5|R0113-2^01/01/00 00:00:00^84^20223203^0113^1^0^45 ��ז� �ט��י ��ט��, ��' ��ך, ךי'ב^0^0^0^0
> [ar@ch ~/]$
>  
>  
>  See? MDBICONV has no effect. The 10th field (it's hebrew) seems the same (even if your terminal doesn't show hebrew, you can see there's no difference), and the 3rd field is still truncated. Only the numbers appear.
>  
>  
>  
>   Any help please?!?
>  
> >
> > On Wednesday 21 September 2011 09:55:12 אריאל קלגסבלד Ariel Klagsbald wrote:
> >> I hope this is the place to post such a problem. And I also hope my
> >> diagnosys is correct (that it's really is an encoding problem. I'm not
> >> sure).
> >>
> >> Well, I have a large mdb file, in which one of the fields contains strings like
> >>
> >> 0007-20101223-214033-שמות-בגדר_שם.mp3
> >>
> >> or
> >>
> >> 0007-20110714-213442-יום_טוב_שני_של_גלויות.mp3
> >>
> >> That is, part english, part numbers and part Hebrew (yes, that's
> >> hebrew, in case you can't see it in your browser).
> >>
> >> When I use mdb-export to extract data from this file, I get the
> >> numbers correctly, but only them. The hebrew and english parts are
> >> simply missing (even the '3' in the 'mp3' suffix). That is, when I
> >> extract the latter example I get only
> >>
> >> 0007-20110714-213442
> >>
> >> I'll add that other fields contain only hebrew (e.g.
> >>  יום טוב שני של גלויות, יב' תמוז, תשע'א
> >> in the example ebove), and they seem to be extracted correctly. That
> >> is, I get some gibberish which I guess is the correct data, only my
> >> terminal can't present it.
> >>
> >> I though it might be an encoding problem, so I've played a bit with
> >> MDB_ICONV, MDB_JET_CHARSET, MDB_JET3_CHARSET and MDB_JET4_CHARSET but
> >> it showed no difference.
> >> The file seems to be JET4 (so mdb-ver claims). I've no idea what
> >> encoding does it use (I don't know how to find out. Any ideas?), but I
> >> guess it's utf-8 (only a guess).
> >>
> >>   I'll be grateful for any help!
> >> Ariel.
> >>
> >> ------------------------------------------------------------------------------
> >> All the data continuously generated in your IT infrastructure contains a
> >> definitive record of customers, application performance, security
> >> threats, fraudulent activity and more. Splunk takes this data and makes
> >> sense of it. Business sense. IT sense. Common sense.
> >> http://p.sf.net/sfu/splunk-d2dcopy1
> >> _______________________________________________
> >> mdbtools-dev mailing list
> >> mdb...@li...
> >> https://lists.sourceforge.net/lists/listinfo/mdbtools-dev
> >>
> >
>  
>  
> ------------------------------------------------------------------------------
> All of the data generated in your IT infrastructure is seriously valuable.
> Why? It contains a definitive record of application performance, security
> threats, fraudulent activity, and more. Splunk takes this data and makes
> sense of it. IT sense. And common sense.
> http://p.sf.net/sfu/splunk-d2dcopy2_______________________________________________
> mdbtools-dev mailing list
> mdb...@li...
> https://lists.sourceforge.net/lists/listinfo/mdbtools-dev

Re: [mdb-dev] mdb-export encoding issue

[אריאל ק. A. K. <kid...@gm...>]

2011/9/21 Nirgal <con...@ni...>:
> Jet4 always use unicode (UCS2) internally.
>
> Output should be utf-8, unless you set env var MDBICONV (there is no
underscore).
>
  I tried again and again, and MDBICONV seems to have no effect (a strange
fact by itself). Any more ideas please? Maybe I'm wrong, and it isn't an
encoding problem. Can something else cause emd-export to ignore half of the
field?


[ak@ch ~/]$ setenv MDBICONV UTF-8
[ak@ch ~/]$ mdb-export -QHd^ WebStructure.mdb FilePaths | grep '^130419\^'
130419^9817^0113-20000101-010645-45_^Hebrew|HWomen|HinuchYeladimShlomBayit|HinuchYeladim|R0113-5|R0113-2^01/01/00
00:00:00^84^20223203^0113^1^0^45 ��ז� �ט��י ��ט��, ��' ��ך, ךי'ב^0^0^0^0
[ar@ch ~/]$ setenv MDBICONV iso-8859-1
[ar@ch ~/]$ mdb-export -QHd^ WebStructure.mdb FilePaths | grep '^130419\^'
130419^9817^0113-20000101-010645-45_^Hebrew|HWomen|HinuchYeladimShlomBayit|HinuchYeladim|R0113-5|R0113-2^01/01/00
00:00:00^84^20223203^0113^1^0^45 ��ז� �ט��י ��ט��, ��' ��ך, ךי'ב^0^0^0^0
[ar@ch ~/]$ setenv MDBICONV nothingatall
[ar@ch ~/]$ mdb-export -QHd^ WebStructure.mdb FilePaths | grep '^130419\^'
130419^9817^0113-20000101-010645-45_^Hebrew|HWomen|HinuchYeladimShlomBayit|HinuchYeladim|R0113-5|R0113-2^01/01/00
00:00:00^84^20223203^0113^1^0^45 ��ז� �ט��י ��ט��, ��' ��ך, ךי'ב^0^0^0^0
[ar@ch ~/]$


 See? MDBICONV has no effect. The 10th field (it's hebrew) seems the same
(even if your terminal doesn't show hebrew, you can see there's no
difference), and the 3rd field is still truncated. Only the numbers appear.



  Any help please?!?

>
> On Wednesday 21 September 2011 09:55:12 אריאל קלגסבלד Ariel Klagsbald
wrote:
>> I hope this is the place to post such a problem. And I also hope my
>> diagnosys is correct (that it's really is an encoding problem. I'm not
>> sure).
>>
>> Well, I have a large mdb file, in which one of the fields contains
strings like
>>
>> 0007-20101223-214033-שמות-בגדר_שם.mp3
>>
>> or
>>
>> 0007-20110714-213442-יום_טוב_שני_של_גלויות.mp3
>>
>> That is, part english, part numbers and part Hebrew (yes, that's
>> hebrew, in case you can't see it in your browser).
>>
>> When I use mdb-export to extract data from this file, I get the
>> numbers correctly, but only them. The hebrew and english parts are
>> simply missing (even the '3' in the 'mp3' suffix). That is, when I
>> extract the latter example I get only
>>
>> 0007-20110714-213442
>>
>> I'll add that other fields contain only hebrew (e.g.
>>  יום טוב שני של גלויות, יב' תמוז, תשע'א
>> in the example ebove), and they seem to be extracted correctly. That
>> is, I get some gibberish which I guess is the correct data, only my
>> terminal can't present it.
>>
>> I though it might be an encoding problem, so I've played a bit with
>> MDB_ICONV, MDB_JET_CHARSET, MDB_JET3_CHARSET and MDB_JET4_CHARSET but
>> it showed no difference.
>> The file seems to be JET4 (so mdb-ver claims). I've no idea what
>> encoding does it use (I don't know how to find out. Any ideas?), but I
>> guess it's utf-8 (only a guess).
>>
>>   I'll be grateful for any help!
>> Ariel.
>>
>>
------------------------------------------------------------------------------
>> All the data continuously generated in your IT infrastructure contains a
>> definitive record of customers, application performance, security
>> threats, fraudulent activity and more. Splunk takes this data and makes
>> sense of it. Business sense. IT sense. Common sense.
>> http://p.sf.net/sfu/splunk-d2dcopy1
>> _______________________________________________
>> mdbtools-dev mailing list
>> mdb...@li...
>> https://lists.sourceforge.net/lists/listinfo/mdbtools-dev
>>
>

Re: [mdb-dev] mdb-export encoding issue

[Nirgal <con...@ni...>]

Jet4 always use unicode (UCS2) internally.

Output should be utf-8, unless you set env var MDBICONV (there is no underscore).


On Wednesday 21 September 2011 09:55:12 אריאל קלגסבלד Ariel Klagsbald wrote:
> I hope this is the place to post such a problem. And I also hope my
> diagnosys is correct (that it's really is an encoding problem. I'm not
> sure).
> 
> Well, I have a large mdb file, in which one of the fields contains strings like
> 
> 0007-20101223-214033-שמות-בגדר_שם.mp3
> 
> or
> 
> 0007-20110714-213442-יום_טוב_שני_של_גלויות.mp3
> 
> That is, part english, part numbers and part Hebrew (yes, that's
> hebrew, in case you can't see it in your browser).
> 
> When I use mdb-export to extract data from this file, I get the
> numbers correctly, but only them. The hebrew and english parts are
> simply missing (even the '3' in the 'mp3' suffix). That is, when I
> extract the latter example I get only
> 
> 0007-20110714-213442
> 
> I'll add that other fields contain only hebrew (e.g.
>  יום טוב שני של גלויות, יב' תמוז, תשע'א
> in the example ebove), and they seem to be extracted correctly. That
> is, I get some gibberish which I guess is the correct data, only my
> terminal can't present it.
> 
> I though it might be an encoding problem, so I've played a bit with
> MDB_ICONV, MDB_JET_CHARSET, MDB_JET3_CHARSET and MDB_JET4_CHARSET but
> it showed no difference.
> The file seems to be JET4 (so mdb-ver claims). I've no idea what
> encoding does it use (I don't know how to find out. Any ideas?), but I
> guess it's utf-8 (only a guess).
> 
>   I'll be grateful for any help!
> Ariel.
> 
> ------------------------------------------------------------------------------
> All the data continuously generated in your IT infrastructure contains a
> definitive record of customers, application performance, security
> threats, fraudulent activity and more. Splunk takes this data and makes
> sense of it. Business sense. IT sense. Common sense.
> http://p.sf.net/sfu/splunk-d2dcopy1
> _______________________________________________
> mdbtools-dev mailing list
> mdb...@li...
> https://lists.sourceforge.net/lists/listinfo/mdbtools-dev
> 

[mdb-dev] mdb-export encoding issue

[אריאל ק. A. K. <kid...@gm...>]

I hope this is the place to post such a problem. And I also hope my
diagnosys is correct (that it's really is an encoding problem. I'm not
sure).

Well, I have a large mdb file, in which one of the fields contains strings like

0007-20101223-214033-שמות-בגדר_שם.mp3

or

0007-20110714-213442-יום_טוב_שני_של_גלויות.mp3

That is, part english, part numbers and part Hebrew (yes, that's
hebrew, in case you can't see it in your browser).

When I use mdb-export to extract data from this file, I get the
numbers correctly, but only them. The hebrew and english parts are
simply missing (even the '3' in the 'mp3' suffix). That is, when I
extract the latter example I get only

0007-20110714-213442

I'll add that other fields contain only hebrew (e.g.
 יום טוב שני של גלויות, יב' תמוז, תשע'א
in the example ebove), and they seem to be extracted correctly. That
is, I get some gibberish which I guess is the correct data, only my
terminal can't present it.

I though it might be an encoding problem, so I've played a bit with
MDB_ICONV, MDB_JET_CHARSET, MDB_JET3_CHARSET and MDB_JET4_CHARSET but
it showed no difference.
The file seems to be JET4 (so mdb-ver claims). I've no idea what
encoding does it use (I don't know how to find out. Any ideas?), but I
guess it's utf-8 (only a guess).

  I'll be grateful for any help!
Ariel.

[mdb-dev] [patch] 64 bits

[Nirgal <con...@ni...>]

Attached patch fixes compilation of odbc driver, and odbc unittest on 64bits platform.

http://www.nirgal.com/mdbtools/debian.squeeze/debian/patches/
http://www.nirgal.com/debian/pool/main/m/

[mdb-dev] [patches] odbc

[Nirgal <con...@ni...>]

04_odbcgetcol.diff fixes fill of pcbColName even when szColName is null. This was crashing my pyodbc tests.

05_odbctypes.diff is based on a patch by Mark Williams:
https://github.com/markrwilliams/mdbtools/commit/4c847948fcd035f3ebafcc0cab3be4de6a90771b so that odbc returns the correct types.
short dates support (without time).
I did not get the opportunity to tests long string reading, despite mangling the original patch. Sorry.

Full patch series against brian git master is at http://www.nirgal.com/mdbtools/debian.squeeze/debian/patches/series
test binaries available at http://www.nirgal.com/mdbtools/debian.squeeze/

[mdb-dev] [patches] clean doc, warnings & abi

[Nirgal <con...@ni...>]

Attached are 3 patches:

01_cleandoc.diff
make clean in doc/ removes .1 files

02_warning4.diff
put back container initialisation from previous warning patch

03_abi.diff
More simple map files without revisions.
Add ODBCINST*.

Re: [mdb-dev] [patch] Warnings, third try

[Brian B. <br...@br...>]

So much for smooth.  Applied.

On Tue, Aug 30, 2011 at 4:46 PM, Nirgal <con...@ni...> wrote:
> On Tuesday 30 August 2011 20:14:19 Brian Bruns wrote:
>> All done.  New process is much smoother.
>
> Somehow, Hunk #7 of previous 01_warnings2.diff did not make it to git, so that now project does not compile.
> Attached is the trivial fix.
>

[mdb-dev] [patch] Warnings, third try

[Nirgal <con...@ni...>]

On Tuesday 30 August 2011 20:14:19 Brian Bruns wrote:
> All done.  New process is much smoother.

Somehow, Hunk #7 of previous 01_warnings2.diff did not make it to git, so that now project does not compile.
Attached is the trivial fix.

Re: [mdb-dev] [patches] Warnings & debug again

[Brian B. <br...@br...>]

All done.  New process is much smoother.

On Tue, Aug 30, 2011 at 4:04 PM, Nirgal <con...@ni...> wrote:
> On Monday 29 August 2011 00:05:52 Brian Bruns wrote:
>> I applied all the patches, except (...)
>
> Here's another try with warnings and debug patches.
>
> ------------------------------------------------------------------------------
> Special Offer -- Download ArcSight Logger for FREE!
> Finally, a world-class log management solution at an even better
> price-free! And you'll get a free "Love Thy Logs" t-shirt when you
> download Logger. Secure your free ArcSight Logger TODAY!
> http://p.sf.net/sfu/arcsisghtdev2dev
> _______________________________________________
> mdbtools-dev mailing list
> mdb...@li...
> https://lists.sourceforge.net/lists/listinfo/mdbtools-dev
>
>

[mdb-dev] [patches] Warnings & debug again

[Nirgal <con...@ni...>]

On Monday 29 August 2011 00:05:52 Brian Bruns wrote:
> I applied all the patches, except (...)

Here's another try with warnings and debug patches.

[mdb-dev] [patch] mdbsql.h typo

[Nirgal <con...@ni...>]

Desccription:
 * Removes declaration of inexistant mdbsql_bind_all
   We are using mdb_sql_bind_all declared bellow

Re: [mdb-dev] [patch] debug

[Brian B. <br...@br...>]

Hi Nirgal,

I'll be applying these patches this evening.

Brian
On Aug 27, 2011 6:25 PM, "Nirgal" <con...@ni...> wrote:
> Description:
> * Fix unsigned/signed bug in display
> * Fix end of loop detection in gmdb_val_to_str
> * Support debug of encrypted files
>
> (see http://www.nirgal.com/mdbtools/debian.squeeze/debian/patches/seriesfor full list)

Re: [mdb-dev] Memo field with binary data

[Vasilis V. <Vas...@ce...>]

I've inherited the database, and I don't have Office to modify it.
Is there a way to change to avoid the iconv conversion in mdb to access the data as such.

Vasilis

________________________________________
From: Nirgal [con...@ni...]
Sent: 27 August 2011 08:04
To: Vasilis Vlachoudis
Cc: mdb...@li...
Subject: Re: [mdb-dev] Memo field with binary data

mdbtools does not support \0 in memo fields.

mdbtools is using iconv to convert charsets in memo fields, so it will probably destroy your data first anyways.

I wonder how you could put some binary in there by the way. :)

Can you try convert these into Ole/Binary fields first? These should work.


On Thursday 25 August 2011 14:27:10 Vasilis Vlachoudis wrote:
> Hi all,
>
> I have a database that I want to read from linux, in which the Memo field contains variable length binary data actually a list of double precision numbers.
> The problems are:
> 1. The memo is reported with max length of 255, however it contains more than 255 characters.  MS-Access it claims that the Memo can be up to 64k
> 2. Since the memo field contains binary data, it contains also '00'x characters which forces the mdb-export to stop on the first null char, rather dumping the whole string.
>
> How can I extract the binary data (length and actual data) unprocessed so I can access the numbers
>
> Best Regards
> Vasilis
>

[mdb-dev] [patch] debug

[Nirgal <con...@ni...>]

Description:
  * Fix unsigned/signed bug in display
  * Fix end of loop detection in gmdb_val_to_str
  * Support debug of encrypted files

(see http://www.nirgal.com/mdbtools/debian.squeeze/debian/patches/series for full list)

[mdb-dev] [patch] gmdb table_def

[Nirgal <con...@ni...>]

Description: Changes in gmdb2 table_def
 * Show column "Null allowed" that was hidden
 * Add column "Description" thanks to props
 * Auto resize columns

Re: [mdb-dev] Memo field with binary data

[Nirgal <con...@ni...>]

mdbtools does not support \0 in memo fields.

mdbtools is using iconv to convert charsets in memo fields, so it will probably destroy your data first anyways.

I wonder how you could put some binary in there by the way. :)

Can you try convert these into Ole/Binary fields first? These should work.


On Thursday 25 August 2011 14:27:10 Vasilis Vlachoudis wrote:
> Hi all,
> 
> I have a database that I want to read from linux, in which the Memo field contains variable length binary data actually a list of double precision numbers.
> The problems are:
> 1. The memo is reported with max length of 255, however it contains more than 255 characters.  MS-Access it claims that the Memo can be up to 64k
> 2. Since the memo field contains binary data, it contains also '00'x characters which forces the mdb-export to stop on the first null char, rather dumping the whole string.
> 
> How can I extract the binary data (length and actual data) unprocessed so I can access the numbers
> 
> Best Regards
> Vasilis
> 

[mdb-dev] [patches] link

[Nirgal <con...@ni...>]

Attached are 2 patches:

asneeded.diff add -as-need option to linker.
This prevents the libraries to try to load uneeded libraries.
Example warning before: libbonobo-activation.so.4 could be avoided if "/usr/bin/gmdb2" were not uselessly linked against it (they use none of its symbols).

odbclink.diff stops embeding libmdb and libmdbsql inside libmdbodbc
I have the feeling mdb and mdbsql libraries run time code should not be included in mdbodbc. mdbodbc should just call the existing functions in the correct libraries. This patch does that.

[mdb-dev] [patch] documentation

[Nirgal <con...@ni...>]

Description: Fix documentation
  * Changed some references about last source from sourceforge to github
  * Document environement vars in man
  * Change FSF address everywhere
  * Changed licence in some source headers to match README rules GPL/LGPL

[mdb-dev] [patch] Support for utf8 in sql

[Nirgal <con...@ni...>]

Attached is a patch that fix utf8 usage in sql (flex) and makes a few cosmetic changes in mdb-sql

[mdb-dev] [patch] warnings

[Nirgal <con...@ni...>]

The attached patch fixes many compilation warnings.
Remaining warnings are real problems not trivial to fix...
That's all for today :)

[mdb-dev] [patch] limited sql date support

[Nirgal <con...@ni...>]

The attached patch derived from the one at https://github.com/markrwilliams/mdbtools/commit/6e9b71af1ff194721928181dc9e694b65c999b98
provide limited sql support for date type.
Exemple usage:
select * from table where dt > 12345678.75

Here 12345678.75 is the number of second since epoch.
Not perfect, but still usefull.

[mdb-dev] [patch] odbc without W

[Nirgal <con...@ni...>]

I had a look at the recent changes in odbc, and could finally test it.
I encountered a batch of problems with the unicode functions:
Some clients like pyodbc really don't like the fact that some wide functions are missing: It wants all of them or none of them.
Also, the code assume we are a little endian platform, with a lot casts of pointer to numbers of different sizes.
The patch attached enclose the W functions in a #ifdef ENABLE_ODBC_W that is *disabled* by default. I'm affraid we'll get too many problems otherwise. It also fixed some of the problems if you enable the macro.

Does anyone know how to add some nice parameters to libtools?
Right now, you need to add a -D to compile flags if you want wide odbc. Utf8 is usually enough IMHO.