From: Daniel R. <da...@pc...> - 2007-04-27 15:30:50
|
>On 2007-04-26 23:01Z, Daniel Raymond wrote: >> I have a very large text file (21GB) and when I run "tail file.txt" I get the >> following error: >> >> tail: file.txt: Bad address >> >> Since 'tail' is such a simple command I tried writing my own. But when I call >> fseek(fin, -1024, SEEK_END) it fails with the following error: >> >> errno = 22 (EINVAL) > >http://www.google.com/search?q=msdn+fseek+einval > >Wouldn't fseek's 'long int' argument be inadequate for >your file, because 2^32 < 21G ? 1) There should be no problem fitting -1024 into a long int. 2) I tried _fseeki64 but I get an undefined reference linker error. 3) Is it a known issue that 'tail' is broken for large files? It seems that large files are exactly what 'tail' is useful for. |
From: Daniel R. <da...@pc...> - 2007-04-30 15:19:24
|
>> >Wouldn't fseek's 'long int' argument be inadequate for >> >your file, because 2^32 < 21G ? >> >> 1) There should be no problem fitting -1024 into a long int. > >No, but what does fseek() return? It's the resultant offset from >*start* of file, and 21G - 1024 cannot be computed in a long int, >so fseek() *must* fail. I've gotten several responses defending fseek()'s misbehavior but nobody has offered me a solution. Surely I am not the first person to try to read large files in MinGW. |
From: Roger W. <ROG...@sa...> - 2007-04-30 15:30:51
|
Daniel Raymond wrote: >>>> Wouldn't fseek's 'long int' argument be inadequate for >>>> your file, because 2^32 < 21G ? >>>> >>> 1) There should be no problem fitting -1024 into a long int. >>> >> No, but what does fseek() return? It's the resultant offset from >> *start* of file, and 21G - 1024 cannot be computed in a long int, >> so fseek() *must* fail. >> > > I've gotten several responses defending fseek()'s misbehavior but nobody has > offered me a solution. Surely I am not the first person to try to read large > files in MinGW. > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > MinGW-users mailing list > Min...@li... > > You may change your MinGW Account Options or unsubscribe at: > https://lists.sourceforge.net/lists/listinfo/mingw-users > > i think __CRT_INLINE off64_t lseek64 (int fd, off64_t offset, int whence) is what you are looking for. Its in io.h -- Roger Wells, P.E. SAIC 221 Third St Newport, RI 02840 401-847-4210 (voice) 401-849-1585 (fax) rog...@sa... |
From: Daniel R. <da...@pc...> - 2007-04-30 15:35:02
|
>i think >__CRT_INLINE off64_t lseek64 (int fd, off64_t offset, int whence) >is what you are looking for. Its in io.h I believe lseek() must be used with open() not fopen(). I've never gotten open() to work under MinGW. It always fails with errno == 2 (ENOENT). Besides, isn't fopen() preferable under most cases because it is more portable and buffers reads and writes? |
From: Michael G. <mg...@te...> - 2007-04-30 15:51:32
|
> >i think > >__CRT_INLINE off64_t lseek64 (int fd, off64_t offset, int whence) > >is what you are looking for. Its in io.h >=20 > I believe lseek() must be used with open() not fopen(). I've never gotte= n=20 > open() to work under MinGW. Surely you know about fileno(FILE*), don't you ? It is in stdio.h and provides the int filedescriptor corresponding to the FILE* you get from fopen. HTH, Michael =2D-=20 Technosis GmbH, Gesch=E4ftsf=FChrer: Michael Gerdau, Tobias Dittmar Sitz Hamburg; HRB 89145 Amtsgericht Hamburg Vote against SPAM - see http://www.politik-digital.de/spam/ Michael Gerdau email: mg...@te... GPG-keys available on request or at public keyserver |
From: Keith M. <kei...@to...> - 2007-04-30 15:59:45
|
Daniel Raymond wrote: > I believe lseek() must be used with open() not fopen(). Huh? fseek() expects a FILE; lseek() expects a file descriptor. You can get the file descriptor for an open FILE, using fileno(). > I've never gotten open() to work under MinGW. I use it regularly, and I've never had the slightest problem. > It always fails with errno == 2 (ENOENT). Perhaps you could post an example, to illustrate this claimed defect; it's likely to be a fault in your usage. > Besides, isn't fopen() preferable under most cases because it > is more portable and buffers reads and writes? Huh? Both are completely portable. open() is lower level, but you can get just the same level of buffering, for files opened either way. Regards, Keith. |
From: Roger W. <ROG...@sa...> - 2007-04-30 18:31:31
|
Daniel Raymond wrote: >> i think >> __CRT_INLINE off64_t lseek64 (int fd, off64_t offset, int whence) >> is what you are looking for. Its in io.h >> > > I believe lseek() must be used with open() not fopen(). I've never gotten > open() to work under MinGW. It always fails with errno == 2 (ENOENT). Besides, > isn't fopen() preferable under most cases because it is more portable and > buffers reads and writes? > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > MinGW-users mailing list > Min...@li... > > You may change your MinGW Account Options or unsubscribe at: > https://lists.sourceforge.net/lists/listinfo/mingw-users > > then how about: int __cdecl fseeko64 (FILE*, off64_t, int); in stdio.h? -- Roger Wells, P.E. SAIC 221 Third St Newport, RI 02840 401-847-4210 (voice) 401-849-1585 (fax) rog...@sa... |
From: Daniel R. <da...@pc...> - 2007-04-30 23:40:50
|
>then how about: >int __cdecl fseeko64 (FILE*, off64_t, int); >in stdio.h? That still doesn't work. Here is my program and output: wldr10@wldr10-00 /d/la_trace $ cat end.c; gcc end.c; a.exe #include <errno.h> #include <stdio.h> #include <stdlib.h> int main(int argc, char *argv[]) { FILE *fin; int result; unsigned long cnt = 1024; unsigned char *buf = malloc(cnt+1); strcpy(buf, "Empty\n"); fin = fopen("temp.txt", "rb"); printf("fopen result = %p, errno = %d\n", fin, errno); result = fseeko64(fin, -cnt, SEEK_END); printf("fseek result = %d, errno = %d\n", result, errno); result = fread(buf, sizeof(unsigned char), cnt, fin); printf("fread result = %d, errno = %d\n", result, errno); buf[cnt] = 0; // NULL terminate printf("%s", buf); fclose(fin); } fopen result = 77C5FCE0, errno = 0 fseek result = 0, errno = 0 fread result = 0, errno = 0 Empty wldr10@wldr10-00 /d/la_trace $ |
From: Greg C. <chi...@co...> - 2007-05-01 00:07:42
|
On 2007-04-30 23:40Z, Daniel Raymond wrote: >> then how about: >> int __cdecl fseeko64 (FILE*, off64_t, int); >> in stdio.h? > > That still doesn't work. Here is my program and output: [...] > unsigned long cnt = 1024; [...] > result = fseeko64(fin, -cnt, SEEK_END); Just a guess: is off64_t(-1024) the same as off64_t(-unsigned long(1024))? |
From: James S. <jam...@op...> - 2007-05-01 00:22:24
|
On Mon, 2007-04-30 at 18:40 -0500, Daniel Raymond wrote: > unsigned long cnt = 1024; > result = fseeko64(fin, -cnt, SEEK_END); > result = fread(buf, sizeof(unsigned char), cnt, fin); mm, you declare an "unsigned long cnt" that I think is really still 32 bit and should be signed. Needs to be "long long" _or_ #include <inttypes.h> and use int64_t". Also I think fread takes a size_t for cnt, so a cast might be nice. Cheers, James. |
From: Daniel R. <da...@pc...> - 2007-05-01 15:27:20
|
>On Mon, 2007-04-30 at 18:40 -0500, Daniel Raymond wrote: > >> unsigned long cnt = 1024; > >> result = fseeko64(fin, -cnt, SEEK_END); > >> result = fread(buf, sizeof(unsigned char), cnt, fin); > >mm, you declare an "unsigned long cnt" that I think is really still 32 >bit and should be signed. Needs to be "long long" _or_ #include ><inttypes.h> and use int64_t". > >Also I think fread takes a size_t for cnt, so a cast might be nice. That was it. I changed cnt to "long long" and it worked. Thanks for your help guys. Now what about 'tail'? It should be easy enough to fix. I'll do it myself if someone can point me to the source. |
From: Greg C. <chi...@co...> - 2007-05-01 19:51:08
|
On 2007-05-01 15:27Z, Daniel Raymond wrote: > > [...] Now what about 'tail'? It should be easy enough to fix. I'll do it > myself if someone can point me to the source. http://mingw.org/MinGWiki/index.php/official%20CVS MSYS cvs...look in the 'coreutils' package. |
From: Keith M. <kei...@us...> - 2007-04-27 19:51:41
|
On Friday 27 April 2007 16:30, Daniel Raymond wrote: > >Wouldn't fseek's 'long int' argument be inadequate for > >your file, because 2^32 < 21G ? > > 1) There should be no problem fitting -1024 into a long int. No, but what does fseek() return? It's the resultant offset from *start* of file, and 21G - 1024 cannot be computed in a long int, so fseek() *must* fail. Regards, Keith. |