Re: [Mingw-users] Reading binary binary files originated from Linux in XP

[Tor L. <tm...@ik...>]

> The binary files I get are saved in a Linux environment and I copied them to
> my XP.

Yes, but *how* did you copy them? That was I wanted to know.

> Then I run my decoder on a msys shell and got an exception back from
> XP pointing to an error from ntdll.dll.

Sounds a bit random then. You say "exception" but your code is in C?
If you run your program under gdb, do you get any sensible backtrace?

Have you checked that your program doesn't have any heap corruption
bugs? Try running it under valgrind on Linux. Note that the Microsoft
C runtime (that MinGW-compiled code uses) often is more sensible to
slight buffer overruns (of a dynamically allocated buffer), for
instance, than glibc on Linux. And such a heap corruption can then
cause an error at some random point later.

> I do use the "rb" flag in fopen, written in c, and compiled using the mingw
> gcc compiler version 3.4.5.  I  thought it was the size of the file

You said 1.4 GB, that is not over any magic limit. 2 GB and 4 GB are
the interesting limits. (2 GB as that is 2^31, i.e. one more than the
maximum value of a signed int, and 4 GB is 2^32, one more than the
maximum value of an unsigned int.)

 > so I attempted to use head to get whatever amount of data I could
to smaller size
> and surprise that it worked.

You didn't answer my question why you use "head", a command explcitly
intended for *text* files, on what you say are *binary* files?
Actually, what is it that makes you consider the files binary?

> One of the binary files has about 6.3mm lines per head,

Sorry, I don't understand what you mean at all. 6.3 millimeters per "head"?

> I then did a head -6500000 binary.file > new.binary.file(to get all lines) and my decoder can
> read and decode it fine.

One possible explanation is that copying the file has corrupted it in
a way related to text vs. binary mode, and then running the head
command "uncorrupts" it. But without knowing more details, it is
impossible to know.

> That got me to wonder if head and cat in msys append or strip away certain
> characters invalid to MinGW C.

head might do all kinds of odd stuff to non-text files indeed, like
splitting "lines" at zero bytes and mangling line endings. cat
presumably should work fine on non-text files. But I would trust "cp"
more anyway.

I don't know what you mean with "characters invalid to MinGW C". If
the files are *binary*, as you say, they don't contain "characters"
(in the logical sense, i.e. lines of characters, in some specific
codeset /encoding like ASCII, ISO8859-1 or UTF-8), but *bytes*. (Note
that the type "char" in C does not mean "character" but *byte*. Only
for some character sets / encodings does one character fit into one
byte.) And trust me, if you open a file in binary mode no bytes are in
any way invalid.

--tml