#4046 (ok 4.0.6) Exporting huge Tables causes memory-Problems

4.0.4
fixed
None
5
2014-12-15
2013-08-06
No

We got a Problem with the 4.0.4.1 Version and exporting of huge Tables oder Databases. Since we upgraded to Major-Release 4 exporting of huge Tables wont work and causes errors like

Out of memory (allocated 1270349824) (tried to allocate 629166081 bytes) in libraries/Response.class.php

or

Allowed memory size of 1073741824 bytes exhausted (tried to allocate 697377364 bytes) in libraries/OutputBuffering.class.php on line 114

A raw-Dump of this Table is about 578MB (very close to that 62916.. bytes), and we already tried to set memory_limit to maximum-value but it didnt help.

I suspect Output-Buffering to be a Problem, the Content of the Output-Buffer is copied on some Parts, for example in OutputBuffering.class.php

        $buffer->_content = ob_get_contents();
        ob_end_clean();

At the moment when ob_get_contents is called the buffer-content is hold twice times and causing memory Problems.

According to my Customer, exporting this huge Table was no Problem in phpMyAdmin 3.

Thanks for help!

Discussion

  • Michal Čihař

    Michal Čihař - 2013-08-11

    I think output buffering should not be used at all for export - it has own logic for sending data to client to reduce memory usage.

     
  • brian m. carlson

    We're also seeing this problem. Usually a database of at least 50 MB is sufficient to trigger this, but sometimes it takes a larger database.

     
  • Stacey Fuller

    Stacey Fuller - 2013-08-26

    I am also having this problem. One of my clients has a table with a large amount of data in it (>4000) entries. I was able to back up this database previously in version 3 releases and now it times out using version 4.

     
  • jimmy silva

    jimmy silva - 2013-08-26

    Same issue here can not export any databases larger then 52 MB

     
  • Mat Connolley

    Mat Connolley - 2013-08-28

    We're also having this problem - in v3 we could export 100 small databases in one go, now we have to export them all separately :(

     
  • J. Nick Koston

    J. Nick Koston - 2013-08-28

    Is there a resolution ETA on this issue?

     
  • Mat Connolley

    Mat Connolley - 2013-08-28

    Hi,

    I'm currently out of the office, returning on 30th August. If you need assistance please call me on 07590 676544.

    Best wishes,

    Mat.

     
  • Marc Delisle

    Marc Delisle - 2013-08-29

    I confirm the problem and it's going to be our priority, targetting a fix for 4.0.6-rc2.

     
  • J. Nick Koston

    J. Nick Koston - 2013-08-29

    Thank you. Please email me where to send beer (or other preferences)

     
    Last edit: J. Nick Koston 2013-08-29
  • Marc Delisle

    Marc Delisle - 2013-08-29
    • assigned_to: Marc Delisle
     
  • Mat Connolley

    Mat Connolley - 2013-08-29

    Hi,

    I'm currently out of the office, returning on 30th August. If you need assistance please call me on 07590 676544.

    Best wishes,

    Mat.

     
  • Marc Delisle

    Marc Delisle - 2013-09-05
    • summary: Exporting huge Tables causes memory-Problems --> (ok 4.0.6) Exporting huge Tables causes memory-Problems
    • status: open --> fixed
     
  • Mat Connolley

    Mat Connolley - 2013-09-11

    Thanks Marc! I can confirm that it's working again for us for an export of 410MB.

     

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks