#4046 (ok 4.0.6) Exporting huge Tables causes memory-Problems

4.0.4
fixed
Marc Delisle
None
5
2014-12-15
2013-08-06
H. Teichmann
No

We got a Problem with the 4.0.4.1 Version and exporting of huge Tables oder Databases. Since we upgraded to Major-Release 4 exporting of huge Tables wont work and causes errors like

Out of memory (allocated 1270349824) (tried to allocate 629166081 bytes) in libraries/Response.class.php

or

Allowed memory size of 1073741824 bytes exhausted (tried to allocate 697377364 bytes) in libraries/OutputBuffering.class.php on line 114

A raw-Dump of this Table is about 578MB (very close to that 62916.. bytes), and we already tried to set memory_limit to maximum-value but it didnt help.

I suspect Output-Buffering to be a Problem, the Content of the Output-Buffer is copied on some Parts, for example in OutputBuffering.class.php

        $buffer->_content = ob_get_contents();
        ob_end_clean();

At the moment when ob_get_contents is called the buffer-content is hold twice times and causing memory Problems.

According to my Customer, exporting this huge Table was no Problem in phpMyAdmin 3.

Thanks for help!

Discussion

  • Michal Čihař
    Michal Čihař
    2013-08-11

    I think output buffering should not be used at all for export - it has own logic for sending data to client to reduce memory usage.

     
  • We're also seeing this problem. Usually a database of at least 50 MB is sufficient to trigger this, but sometimes it takes a larger database.

     
  • Stacey Fuller
    Stacey Fuller
    2013-08-26

    I am also having this problem. One of my clients has a table with a large amount of data in it (>4000) entries. I was able to back up this database previously in version 3 releases and now it times out using version 4.

     
  • jimmy silva
    jimmy silva
    2013-08-26

    Same issue here can not export any databases larger then 52 MB

     
  • Mat Connolley
    Mat Connolley
    2013-08-28

    We're also having this problem - in v3 we could export 100 small databases in one go, now we have to export them all separately :(

     
  • J. Nick Koston
    J. Nick Koston
    2013-08-28

    Is there a resolution ETA on this issue?

     
  • Mat Connolley
    Mat Connolley
    2013-08-28

    Hi,

    I'm currently out of the office, returning on 30th August. If you need assistance please call me on 07590 676544.

    Best wishes,

    Mat.

     
  • Marc Delisle
    Marc Delisle
    2013-08-29

    I confirm the problem and it's going to be our priority, targetting a fix for 4.0.6-rc2.

     
  • J. Nick Koston
    J. Nick Koston
    2013-08-29

    Thank you. Please email me where to send beer (or other preferences)

     
    Last edit: J. Nick Koston 2013-08-29
  • Marc Delisle
    Marc Delisle
    2013-08-29

    • assigned_to: Marc Delisle
     
  • Mat Connolley
    Mat Connolley
    2013-08-29

    Hi,

    I'm currently out of the office, returning on 30th August. If you need assistance please call me on 07590 676544.

    Best wishes,

    Mat.

     
  • Marc Delisle
    Marc Delisle
    2013-09-05

    • summary: Exporting huge Tables causes memory-Problems --> (ok 4.0.6) Exporting huge Tables causes memory-Problems
    • status: open --> fixed
     
  • Mat Connolley
    Mat Connolley
    2013-09-11

    Thanks Marc! I can confirm that it's working again for us for an export of 410MB.