#420 loss of files on full disk

release
open
Program (402)
9
2007-12-19
2004-09-14
Anonymous
No

Today, september 14th 2004, I noticed something
unpleasant. I work on my own share of a large company
server which does not have a very large space. Writing
text in Latex often requires ps (eps) figures which can
run big. So, I ran out of space. Unfortunately, nedit
simply gave the message "can not save, no space on
device" when I wanted to save a tex file created in
nedit. The only option I had was to press ok, which I
did, thinking I would have to re-edit the file after
making space on my share by removing some old files.
However, nedit also decided to delete the tex file
altogether! So, it did not leave the tex file unchanged,
but the tex file was gone completely. Fortunately, we
have a good backup system on our server, but I can
imagine this can be very annoying when working on
your private computer, running out of space (that can
happen) and then losing your tex file you have been
editing on for the last hour or so.

Just wanted to report this, maybe it's not new, but I
think it is something to recon with.

With regards,

victor land
PhD FOM Institute for Plasma Physics Rijhuizen
The Netherlands

reply: vland@rijnh.nl

Discussion

  • Thorsten Haude

    Thorsten Haude - 2004-09-15
    • priority: 5 --> 8
     
  • Nathan Gray

    Nathan Gray - 2004-09-15

    Logged In: YES
    user_id=121553

    The "remove(fullname);" line in this code is the culprit:

    --- code ---
    /* write to the file */
    #ifdef IBM_FWRITE_BUG
    write(fileno(fp), fileString, fileLen);
    #else
    fwrite(fileString, sizeof(char), fileLen, fp);
    #endif
    if (ferror(fp))
    {
    DialogF(DF_ERR, window->shell, 1, "Error saving File",
    "%s not saved:\n%s", "OK", window->filename,
    errorString());
    fclose(fp);
    remove(fullname);
    XtFree(fileString);
    return FALSE;
    }
    --- end code ---

    The problem is that by this point we may have already written to the file,
    probably corrupting it. Rather than leaving a corrupt file, NEdit deletes
    it. This is arguably a bad idea, but it's incorrect to think NEdit deleted
    the original, unaltered copy -- it was already overwritten.

    The safe way to do this is to create a backup of the file before saving
    and then remove it when the save has completed. The first half is
    already possible using the "Preferences->Default Settings->Make Backup
    Copy (*.bck)" option. The only problem is that the backup copy is not
    deleted after the save completes.

    Perhaps we should make it so the default behaviour is "make backup
    copy; save; delete backup copy." If the user selects "Make Backup
    Copy" then the backup copy would not be deleted. It would mean saving
    would take extra disk space for a short time, but the only time it would
    cause a problem is when you have enough space for 1 copy of the file
    but not 2. In this case you're playing with fire anyway! It would also
    mean saves would take a while longer (either that or the file's inode
    number would change).

     
  • Thorsten Haude

    Thorsten Haude - 2006-09-30
    • priority: 8 --> 9
     
  • Andrew Hood

    Andrew Hood - 2006-10-01

    Logged In: YES
    user_id=36856

    If we were to replace all uses of fwrite() with code like
    this we would be notified there was a problem

    ssize_t written;
    while((written=write(fd, ptr, bytesleft))>0) {
    ptr+=written;
    bytesleft-=written;
    }
    if (written<0) {
    DialogF(DF_ERR, window->shell, 1, "Error saving File",
    "your file %s is toast because of %s. you'd better save it
    somewhere else", window->filename, errorString());
    );
    /* cleanup */
    }
    if (close(fd)<0)) {
    DialogF(DF_ERR, window->shell, 1, "Error saving File",
    "your file %s is toast because of %s. you'd better save it
    somewhere else", window->filename, errorString());
    /* cleanup */
    }
    else {
    /* save was OK */
    }

     
  • Nobody/Anonymous

    Logged In: NO

    I suppose we can measure an existing file's size and, if smaller than the require size, try to extend it with fseek() (using fopen(..., "r+b") first). The opposite, writing a shorter text to a pre-existing file without emptying it first, requires file truncation; I don't know a standard way to do that!

    The doSave() function copies file buffers around. If no adjustment is necessary (applying null chars, replacing line endings), it would be good to simply use the original buffer (BufAsString()) as the source for writing.

     
  • Thorsten Haude

    Thorsten Haude - 2007-12-19

    Logged In: YES
    user_id=119143
    Originator: NO

    I looked into this and I think we can avoid the problem itself pretty easily, but there are some deeper issues. My fix simply switches between ENOSPC and everything else and avoids deletion on ENOSPC. The file is butchered but survives.

    I added a rename(file, file.CORRUPT) just to explore an alternative. Also, this is obviously not production-quality. Apart from the TODOs, I also used symbolic error messages etc. Feel free to comment.

    When the error shows the user is really up Shit Creek. He operates with either huge files, very little free space, or both; he wanted no .bck or ignored the error message; and he will shortly lose data. I think the only *good* way NEdit can handle that would be constant file monitoring and preallocation. I think that's not worth the effort for a pathological situation like this.

    Nate's solution ("make backup copy; save; delete backup copy.") would work for most people; most people also would use .bcks (and heed error messages) and wouldn't have the problem in the first place. NEdit must of course be able to edit files on the edge of a full disk.

    What is the role of the culprity remove() anyway? What error better results in no file at all than in a butchered one?
    File Added: fulldisk.2007-12-19.3.diff

     
  • Thorsten Haude

    Thorsten Haude - 2007-12-19
    • milestone: --> release
    • assigned_to: nobody --> yooden
     
  • Tony Balinski

    Tony Balinski - 2007-12-20

    Logged In: YES
    user_id=618141
    Originator: NO

    Maybe we should do this: open the file for update; if the buffer (after "expanding" line end marks appropriately) is longer than the file, start writing this new end; then, if that was successful, write over the front. If the buffer is shorter, overwrite the front and truncate the file at the new file end position. If we hit the ENOSPC in the file-longer case, this will happen when adding the new end, so we can simply abandon the change, truncating the file to its original length. I presume that overwriting the front of the file and making it shorter are doable under these conditions.

    Just my 2 rappen.

     
  • Thorsten Haude

    Thorsten Haude - 2007-12-20

    Logged In: YES
    user_id=119143
    Originator: NO

    Good idea, but nobody mentioned that there would be no way to trunc the file. I looked in my reference for a few minutes yesterday and found nothing either.

    This mode would preserve the content as much as possible even for shorter files, so it would be much nicer for shorter files than truncing from the start. We would also avoid race conditions.

     
  • Tony Balinski

    Tony Balinski - 2007-12-20

    Logged In: YES
    user_id=618141
    Originator: NO

    f?truncate() are available in POSIX (2004 at least), but I wonder if they're too young to be present on the many legacy systems NEdit runs on. Crazy when you think that you can easily grow files, but not cut them short!

    I understand (also after a few minutes) that some file systems won't support growing a file using truncate(). That shouldn't be a problem though. Maybe there's a way of detecting whether truncate exists, and using it on those systems that have it (VMS anybody?).

     

Log in to post a comment.