Learn how easy it is to sync an existing GitHub or Google Code repo to a SourceForge project! See Demo

Close

#2760 (ok 4.1.2) Warn about incomplete exports

4.1.1
fixed
Marc Delisle
1
2013-12-23
2008-09-12
Marc Delisle
No

From the forum:

i recently exported a database from phpmyadmin as a backup of my data. i then deleted the original database from the server, safe in the knowledge that i had a backup (or so i thought).

however, i later noticed the following line at the end of the file:

<b>Fatal error</b>: Maximum execution time of 300 seconds exceeded in <b>/usr/local/cpanel/base/3rdparty/phpMyAdmin/libraries/export/sql.php</b> on line <b>904</b><br />

it was then that i realised phpmyadmin had not fully exported the file. even worse - it had not warned me that the export was not complete! i have therefore completely lost some very precious data. this is a major problem!!!

for god's sake, please add a warning message to this software to let people know when an export has not completed. it can't be rocket science and it's an extremely serious problem.

aside from that, phpmyadmin is a great piece of software. but please, sort this out!

Discussion

1 2 > >> (Page 1 of 2)
  • Fantu
    Fantu
    2008-10-31

    there is same problem with import i think that happens when it is near the maximum time but PMA does not stop because running a very long query (or operation) and reaches the limit before finish it and execute che check for stop (and view when stop for restart from it)

     
  • Marc Delisle
    Marc Delisle
    2009-04-05

    • assigned_to: nobody --> lem9
     
  • Marc Delisle
    Marc Delisle
    2009-04-28

    • assigned_to: lem9 --> nobody
     
  • J.M. Roth
    J.M. Roth
    2009-06-21

    On my installation it seems that saving as uncompressed text and "zipped" works fine.
    "gzipped" generates a corrupt file, and "bzipped" seems to cut the exported data.
    v2.9.1.1-Debian-10 (Debian Etch)

     
  • Marc Delisle
    Marc Delisle
    2009-09-19

    • assigned_to: nobody --> tomassrnka
     
  • Marc Delisle
    Marc Delisle
    2010-03-18

    Any progress on this bug fixing?

     
  • Marc Delisle
    Marc Delisle
    2010-06-27

    • assigned_to: tomassrnka --> nobody
     
  • Piyush Mishra
    Piyush Mishra
    2011-03-20

    I'll try to work on it. can't be sure but I'll try as I learn. i have been bitten by this bug and want to kill it :X

     
  • Piyush Mishra
    Piyush Mishra
    2011-03-20

    I am currently learning the Code Base and have no good experience on this but I think this would be a good way to proceed.
    http://pastebin.com/T33AwNcC

    What do you think?

     
  • Marc Delisle
    Marc Delisle
    2011-03-20

    @piyushmishra, please explain how this code would fit inside the current export code. Remember, we don't know where we'll be in the logic when the max execution time happens.

     
  • Piyush Mishra
    Piyush Mishra
    2011-03-21

    as far as I am planning it right now, it'll be in PMA_exportOutputHandler (export.php) as this function gets called with each output.
    we can keep a decent time window of 5 seconds perhaps to make sure we catch most of the issues
    and keep a final fallback for this as an error handler to warn the user and probably work start again with the last known working file. its too early for me right now as I am still very new to the code base. I'll dig deeper and post more about it

     
  • Piyush Mishra
    Piyush Mishra
    2011-03-21

    I think I narrowed the issue down to around line 175 {PMA}/export.php.
    Where PMA sends header('X-pmaPing: Pong'); to buy more time.
    This doesnt work on server's with safe_mode 'On' in php as execution stops on the default max_execution time and set_time_limit() has no effect at all
    so we can atleast warn the user if safe mode is on and may be make a new check for time being close to time near that line to stop executing the script and send out some errors.

     
  • Piyush Mishra
    Piyush Mishra
    2011-03-23

    Scratch the first part of my last comment. the error can be handled from that function. I've sent a mail to the mailing list regarding this. can anyone please help me with that?
    here's a copy of the mail
    I've hit a road block with a bug and would appreciate a lot if some seasoned developers would help me get over the problem.
    Narrowed down the problem to being from php.ini 's safe mode setting which doesn't let set_time_limit override default max_execution_time in php.ini

    After making a rough idea of how the export works in PMA it struck me that we can use the function that basically outputs every line in all types of output. [install dir]/export.php > PMA_exportOutputHandler to check if we are going to run out of time

    http://pastebin.com/8BCSN7es

    I made these changes from around line 110 search for function called PMA_exportOutputHandler and you'll find it.
    Now the final lines on my download give out the line "It's gonna blow!!!"
    last few lines.
    http://pastebin.com/BEbv42Ds

    Now I need to make sure that

    User gets an error. (session or something? $GLOBALS['message'] = PMA_Message::error(); is of no use for files going line by line :( )
    The function passes back the last output line or may be redirects for another file download for lines after the current one being output.

    Can you guys think of any other scenario for an error (except for safe mode) and can you guide me on shooting out the errors?
    Worst case scenario we can use the error handling function for a fallback but I want to make a clean solution if possible.

    return false; will obviously stop the output before an error can be made but that will not warn a user.

    What about a generic warning in case php is running in safe mode?

     
  • Marc Delisle
    Marc Delisle
    2013-01-20

    • milestone: 2.11.9 --> 3.5.6
     
  • Smita Kumari
    Smita Kumari
    2013-12-16

    Hello sir,
    I have tried to fix this bug. Here is what i have done:
    On click of Go button i have called a javascript function that make a get reuest to server after the time equal to script execution time limit. on the server side i have used the register_shutdown_function to track if time limit has been exceeded. if execution stop due to an error it stores in cookie (pma_error) which is checked by ajax GET request sent by client after the specified time interval. If it finds pma_error cookie set then it displays to the user and remove cookie else does nothing. It's working fine for me. Here is the commit i have made https://github.com/smita786/phpmyadmin/commit/fa66b68181799705d60066e842ffd7c86350d0b6 for the same. Please review if it looks good, I'll submit the pull request. OR if you can suggest something to make it work fine.

    Thanks

     
  • Marc Delisle
    Marc Delisle
    2013-12-16

    Hi Smita,
    I have tested your commit under Windows 7 / Firefox 24. Server is running Linux, PHP 5.5.3. I see no message displayed when I reach the maximum execution time of 10 seconds in my case.

     
  • Marc Delisle
    Marc Delisle
    2013-12-16

    Hi Smita,
    with this second version, now I see the timeout message correctly. However, something bothers me. I am testing with the "employees" sample MySQL database, which produces an export of about 177 MB. Without your patch, when I ask an export, it almost immediately starts, I can follow its progress via the browser (Firefox), and the virtual memory for the httpd process reaches a maximum of 160 MB.

    With your patch, I no longer can follow the download progress, so I'm wondering what is happening. Also, the virtual memory of httpd reaches 360 MB, which means that users will use more resources on the web server, and might hit some memory limits.

     
  • Smita Kumari
    Smita Kumari
    2013-12-17

    Hi Sir,

    I needed to set a cookie to access it across the requests. For some reason setting a session variable was not working as expected. In the shutdown function I am setting a cookie pma_error if timeout occurs and using it in an Ajax request to check weather error had occurred. So for being able to set the cookie I needed to start the output buffering (using ob_start function), That's why all the outputs is first getting buffered and then in the end sent to the browser resulting in late download start and more consumption of VM.
    So I think its not a good approach to track the error. If you can suggest me some way within PMA, I can set a variable in one request and can access the same variable in other request ? as I’m not very much familiar with the codebase, I might be missing something.

    Thank You!

     
  • Marc Delisle
    Marc Delisle
    2013-12-17

    Hi Smita,
    here are some (untested) ideas.

    First, you need to use a session variable instead of a cookie, to avoid the output buffer start as you said. To be able to do this, the session must have been started and this is one of the things done by libraries/common.inc.php. So, in export.php, your verification for the timeout should be done after common.inc.php has been loaded (which is under the "if !defined('TESTSUITE')" section anyway, and must be there due to the testing suite requirements).

    Second, the js code calling export.php should have the token parameter (otherwise it will be rejected for security reasons). For an example of $.get that correctly uses the token, see js/server_privileges.js.

    Third (this is minor), the shutdown() function should have a standard name (prefix PMA_ if not located in a class) and be in a .lib.php file. This can wait until the rest works correctly.

    Hope that helps.

     
    Last edit: Marc Delisle 2013-12-17
  • Marc Delisle
    Marc Delisle
    2013-12-17

    • Group: 3.5.6 --> 4.1.1
     
  • Smita Kumari
    Smita Kumari
    2013-12-17

    Hello Marc,

    Here is my latest commit https://github.com/smita786/phpmyadmin/commit/70d272087b4ec85b23eed6207504cecd48dc9b83 , using SESSION variable now as you have suggested. and have removed the output buffering so late download start and more consumption of VM should go.
    Please review and let me know if It's working fine or if something I need to modify.

    Thanks

     
    • Marc Delisle
      Marc Delisle
      2013-12-17

      Smita,
      Works fine for me, this is great and will solve an important bug. Have a look at further comments I wrote on your commit.

       
  • Smita Kumari
    Smita Kumari
    2013-12-17

    I have made another commit with localization of error message as suggested by you. Here is it https://github.com/smita786/phpmyadmin/commit/6faaa78f44721f7b6609b6b05bb23c88b1a32413 . Let me know if it'll work then I'll submit the pull request.
    Also, Now I have put a predefined time out error message that you have suggested in your comment So I think there is no need to show the error message coming from backend.

    Thanks

     
1 2 > >> (Page 1 of 2)