From: Timothy M. <ter...@gm...> - 2013-08-28 23:23:20
|
Hello I just compiled and tried a program that opens files with std::fstream, like (minimal complete example): #include <iostream> #include <fstream> int main(int argc, char *argv[]) try { std::fstream f; if (argc != 2) return 1; f.exceptions(f.badbit | f.failbit | f.eofbit); f.open(argv[1], f.in | f.out | f.binary | f.ate); return 0; } catch (std::ios_base::failure const &ex) { std::cerr << "File manipulation error: " << ex.what() << ".\n"; return 1; } Than I give the name of a file that is exactly 4GB, as the value pointed by argv[1]. The f.open(); line immediately throws an class std::ios_base::failure exception with message: "basic_ios::clear". Even the `ls` command in the MinGW shell window displays the file as having size 0: Crina@Adrian-PC ~ $ make large-file # no Makefile present g++ large-file.cc -o large-file Crina@Adrian-PC ~ $ large-file.exe datafile.bin File manipulation error: basic_ios::clear. Crina@Adrian-PC ~ $ ll datafile.bin -rw-r--r-- 1 Crina Administrators 0 Aug 28 05:59 datafile.bin Crina@Adrian-PC ~ $ Despite the output from ll, the file actually has a size of 4GB. Is there a way to make the file stream class std::fstream work with large files in MinGW ? (cygwin version of the program works as expected) Trying to open a smaller file works fine: Crina@Adrian-PC ~ $ large-file.exe large-file.cc Crina@Adrian-PC ~ $ echo $? 0 Crina@Adrian-PC ~ $ I have tested sizeof(std::fstream::off_type) in gdb, the retuned value is 8, indicating a 64-bit integer. Note that the large input file used has been created from an empty file with a similar program compiled with g++ 4.7.2 under MinGW, that repeatedly fills a 1 MB buffer with random numbers and writes the buffer with f.write(); Thank you, Timothy Madden |
From: John E. / T. <td...@td...> - 2013-08-29 00:03:25
|
On 8/28/2013 5:23 PM, Timothy Madden wrote: > Is there a way to make the file stream class std::fstream work with > large files in MinGW ? (cygwin version of the program works as expected) Hi Timothy, There is unfortunately some missing functionality in recent official MinGW GCC releases: the libstdc++ library is compiled without "Large File Support" due to an incomplete implementation on Windows targets. See <https://sourceforge.net/p/mingw/bugs/1422/> for details -- the bug has been closed but is still present in the MinGW GCC 4.7.2 release. TDM-GCC[1], an alternate GCC toolchain distribution, includes a fix for this bug, and I believe MinGW-w64-based[2] GCC distributions may not be affected by it. -John E. / TDM [1] - <http://tdm-gcc.tdragon.net/> [2] - <http://mingw-w64.sourceforge.net/> |
From: Timothy M. <ter...@gm...> - 2013-08-29 20:19:09
|
Pe 29.08.2013 03:03, John E. / TDM a scris: > On 8/28/2013 5:23 PM, Timothy Madden wrote: >> Is there a way to make the file stream class std::fstream work with >> large files in MinGW ? (cygwin version of the program works as expected) > > Hi Timothy, > > There is unfortunately some missing functionality in recent official > MinGW GCC releases: the libstdc++ library is compiled without "Large > File Support" due to an incomplete implementation on Windows targets. > See <https://sourceforge.net/p/mingw/bugs/1422/> for details -- the bug > has been closed but is still present in the MinGW GCC 4.7.2 release. > > TDM-GCC[1], an alternate GCC toolchain distribution, includes a fix for > this bug, and I believe MinGW-w64-based[2] GCC distributions may not be > affected by it. Thank you, Does this mean that some previous gcc releases on MinGW used to support large files ? Can I expect the bug to be fixed in the next gcc release to be included in MinGW ? Is there some work or plans for the next gcc version in MinGW ? Thank you, Timothy Madden |
From: John E. / T. <td...@td...> - 2013-08-29 23:49:04
|
On 8/29/2013 2:18 PM, Timothy Madden wrote: > Pe 29.08.2013 03:03, John E. / TDM a scris: *snip* >> There is unfortunately some missing functionality in recent official >> MinGW GCC releases: the libstdc++ library is compiled without "Large >> File Support" due to an incomplete implementation on Windows targets. >> See <https://sourceforge.net/p/mingw/bugs/1422/> for details -- the bug >> has been closed but is still present in the MinGW GCC 4.7.2 release. >> >> TDM-GCC[1], an alternate GCC toolchain distribution, includes a fix for >> this bug, and I believe MinGW-w64-based[2] GCC distributions may not be >> affected by it. > Thank you, My pleasure. > Does this mean that some previous gcc releases on MinGW used to support > large files ? I suspect not. > Can I expect the bug to be fixed in the next gcc release to be included > in MinGW ? Is there some work or plans for the next gcc version in MinGW ? Hopefully Earnie Boyd will answer this question definitively, as I believe he's planning to do an official 4.8.x release in the next few months. The functionality is enabled in 32-bit TDM-GCC with a fairly straightforward patch, but said patch (nor any variant of it) has never been applied to upstream GCC. -John E. / TDM |
From: Pau G. i Q. <pgq...@el...> - 2013-08-29 21:37:35
|
On Thu, Aug 29, 2013 at 10:18 PM, Timothy Madden <ter...@gm...>wrote: > Does this mean that some previous gcc releases on MinGW used to support > large files ? > > AFAIK MinGW32-based variants (mingw.org, nuwen, etc) never included Large File Support and nothing makes me think this is going to change. More details on _FILE_OFFSET_BITS=64 here: http://users.suse.com/~aj/linux_lfs.html MinGW variants based on MinGW-w64 do support LFS. E. g. mingw-w64.sf.net, mingwbuilds, TDM and others. -- Pau Garcia i Quiles http://www.elpauer.org (Due to my workload, I may need 10 days to answer) |
From: Keith M. <kei...@us...> - 2013-08-30 09:33:10
|
On 29 August 2013 01:03, John E. / TDM <td...@td...> wrote: > On 8/28/2013 5:23 PM, Timothy Madden wrote: > > Is there a way to make the file stream class std::fstream work with > > large files in MinGW ? (cygwin version of the program works as expected) > > There is unfortunately some missing functionality in recent official > MinGW GCC releases: the libstdc++ library is compiled without "Large > File Support" due to an incomplete implementation on Windows targets. > See <https://sourceforge.net/p/mingw/bugs/1422/> for details -- the bug > has been closed but is still present in the MinGW GCC 4.7.2 release. > It was closed when we migrated to the new SF Allura paltform, because it related to a no-longer-current release of GCC. It was expected that, if it continues to affect current GCC, you would either reopen it yourself, or at least request that it be reopened; I see no reason why we can't still do that now, so I've gone ahead and done so. -- Regards, Keith. |
From: Keith M. <kei...@us...> - 2013-08-30 09:57:46
|
On 29 August 2013 22:06, Pau Garcia i Quiles wrote: > AFAIK MinGW32-based variants (mingw.org, nuwen, etc) never included Large > File Support > I believe this to be so, but... > and nothing makes me think this is going to change. > ...there's absolutely no reason why it shouldn't, given the will to make it happen, but the GCC folks upstream need to be on board; (we have no desire to be burdened with maintaining local patches in perpetuity). > More details on _FILE_OFFSET_BITS=64 here: > > http://users.suse.com/~aj/linux_lfs.html > That's specific to Linux, and therefore completely irrelevant for MinGW. Until it's ratified by the GCC folks upstream, it remains uncertain how LFS will be activated for the Win32 platform; _FILE_OFFSET_BITS may, or may not be applicable. -- Regards, Keith. |
From: Pau G. i Q. <pgq...@el...> - 2013-08-30 10:54:19
|
On Fri, Aug 30, 2013 at 11:57 AM, Keith Marshall < kei...@us...> wrote: > More details on _FILE_OFFSET_BITS=64 here: >> >> http://users.suse.com/~aj/linux_lfs.html >> > > That's specific to Linux, and therefore completely irrelevant for MinGW. > Until it's ratified by the GCC folks upstream, it remains uncertain how > LFS will be activated for the Win32 platform; _FILE_OFFSET_BITS may, or > may not be applicable. > > _FILE_OFFSET_BITS=64 is already implemented and working in mingw-w64-based compilers, I've been using this feature for a year and a half. -- Pau Garcia i Quiles http://www.elpauer.org (Due to my workload, I may need 10 days to answer) |
From: Keith M. <kei...@us...> - 2013-08-30 11:11:07
|
On 30 August 2013 11:53, Pau Garcia i Quiles wrote: > _FILE_OFFSET_BITS=64 is already implemented and working in mingw-w64-based > compilers, I've been using this feature for a year and a half. > Still irrelevant. Until it's ratified by upstream, it remains no more than a de-facto quasi-standard. |
From: マーズ <ma...@th...> - 2013-08-29 01:58:29
|
Greetings, Can you tell me the filesystem, is it NTFS? VFAT? FAT32? My hunch is that filesystem support is needed for large files greater than 2 Gbytes. Timothy Madden wrote: > Hello > > I just compiled and tried a program that opens files with std::fstream, > like (minimal complete example): > > #include <iostream> > #include <fstream> > > int main(int argc, char *argv[]) > try > { > std::fstream f; > > if (argc != 2) > return 1; > > f.exceptions(f.badbit | f.failbit | f.eofbit); > f.open(argv[1], f.in | f.out | f.binary | f.ate); > > return 0; > } > catch (std::ios_base::failure const &ex) > { > std::cerr << "File manipulation error: " << ex.what() << ".\n"; > > return 1; > } > > Than I give the name of a file that is exactly 4GB, as the value pointed > by argv[1]. The f.open(); line immediately throws an class > std::ios_base::failure exception with message: "basic_ios::clear". > > Even the `ls` command in the MinGW shell window displays the file as > having size 0: > > Crina@Adrian-PC ~ > $ make large-file # no Makefile present > g++ large-file.cc -o large-file > > Crina@Adrian-PC ~ > $ large-file.exe datafile.bin > File manipulation error: basic_ios::clear. > > Crina@Adrian-PC ~ > $ ll datafile.bin > -rw-r--r-- 1 Crina Administrators 0 Aug 28 05:59 datafile.bin > > Crina@Adrian-PC ~ > $ > > Despite the output from ll, the file actually has a size of 4GB. > > Is there a way to make the file stream class std::fstream work with > large files in MinGW ? (cygwin version of the program works as expected) > > Trying to open a smaller file works fine: > > Crina@Adrian-PC ~ > $ large-file.exe large-file.cc > > Crina@Adrian-PC ~ > $ echo $? > 0 > > Crina@Adrian-PC ~ > $ > > I have tested sizeof(std::fstream::off_type) in gdb, the retuned value > is 8, indicating a 64-bit integer. > > Note that the large input file used has been created from an empty file > with a similar program compiled with g++ 4.7.2 under MinGW, that > repeatedly fills a 1 MB buffer with random numbers and writes the buffer > with f.write(); > > Thank you, > Timothy Madden > > > ------------------------------------------------------------------------------ > Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more! > Discover the easy way to master current and previous Microsoft technologies > and advance your career. Get an incredible 1,500+ hours of step-by-step > tutorial videos with LearnDevNow. Subscribe today and save! > http://pubads.g.doubleclick.net/gampad/clk?id=58040911&iu=/4140/ostg.clktrk > _______________________________________________ > MinGW-users mailing list > MinGW- us...@li... > > This list observes the Etiquette found at > http://www.mingw.org/Mailing_Lists. > We ask that you be polite and do the same. Disregard for the list etiquette > may cause your account to be moderated. > > _______________________________________________ > You may change your MinGW Account Options or unsubscribe at: > https://lists.sourceforge.net/lists/listinfo/mingw-users > Also: > mailto:mingw-users- re...@li...?subject=unsubscribe |
From: Timothy M. <ter...@gm...> - 2013-08-29 20:25:27
|
Pe 29.08.2013 04:58, マーズ a scris: > Greetings, > Can you tell me the filesystem, is it NTFS? VFAT? FAT32? > My hunch is that filesystem support is needed for large files > greater than 2 Gbytes. NTFS, Windows 7 Ultimate SP1 32-bit. cygwin works fine with the program, and Windows Explorer shows the right file size too. Thank you, Timothy Madden |
From: マーズ <ma...@th...> - 2013-08-30 00:16:15
|
Greetings Timothy Madden, Do you keep a bitcoin wallet in your win7 SP1 32-bit? That app surely uses alot of large files. Timothy Madden wrote: > Pe 29.08.2013 04:58, マーズ a scris: >> Greetings, >> Can you tell me the filesystem, is it NTFS? VFAT? FAT32? >> My hunch is that filesystem support is needed for large files >> greater than 2 Gbytes. > > NTFS, Windows 7 Ultimate SP1 32-bit. > > cygwin works fine with the program, and Windows Explorer shows the right > file size too. > > Thank you, > Timothy Madden > > > > > ------------------------------------------------------------------------------ > Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more! > Discover the easy way to master current and previous Microsoft technologies > and advance your career. Get an incredible 1,500+ hours of step-by-step > tutorial videos with LearnDevNow. Subscribe today and save! > http://pubads.g.doubleclick.net/gampad/clk?id=58040911&iu=/4140/ostg.clktrk > _______________________________________________ > MinGW-users mailing list > Min...@li... > > This list observes the Etiquette found at > http://www.mingw.org/Mailing_Lists. > We ask that you be polite and do the same. Disregard for the list etiquette > may cause your account to be moderated. > > _______________________________________________ > You may change your MinGW Account Options or unsubscribe at: > https://lists.sourceforge.net/lists/listinfo/mingw-users > Also: mailto:mingw-users- re...@li...?subject=unsubscribe |