Documents stored in the 'files' folder

Developers
Daimonji
2012-03-06
2013-03-22
  • Daimonji
    Daimonji
    2012-03-06

    Hello Eric,

    I noticed that files generated in the /files folder tend to grow faster, and after some time, the system complains because it lacks some inodes.

    For test purposes, I removed most content in files/data and files/document, and I noticed that I can still download documents. However, it is not possible now to upload a new document, because it tries to copy it in a folder that doens't exist anymore.

    My questions are:
    - Where are stored uploaded documents? Locally in the files folder or in the database?
    - Is there a way to fix the removed data?

    Thank you,
    Daimonji

     
  • Eric Lawman
    Eric Lawman
    2012-03-06

    Stored uploaded documents go in the files/data folder.  Inline DocMGR "documents" go in the files/documents folder.  You really don't want to remove anything in the files/ directory, except for the tmp/ folder.  You can clean that out regularly, and many users simply create a cron job to delete the contents of the folder every night.

    The "data" and "documents" folders have 16 sub directories in the first level, and 256 in the second.  The files get spread throughout the folder hierarchy to prevent too many files going into one sub-directory.  Back when the project first started, many file systems started complaining once you had 100,000 or more files in a folder.  So, that should keep you from having inode problems, and makes me wonder if there is something wrong w/ your drive/filesystem?  Hard to say, I'm just guessing.

    If you have a backup of the files directory, you just drop the backup back in.  Or if there are files that have been uploaded since your last backup, you should be able to use rsync to merge the backup with what you have in DocMGR now. 

    You are right, though.  We only create the data/ file structure on install because we assume it will always be there.

     
  • Daimonji
    Daimonji
    2012-03-07

    Hi,

    It seems that as this folder was versioned, creating no more than 12000 folders with .svn folders or files included.

    My data/documents folder has, after these .svn files are removed, 9 files and 4,112 folders, so I guess the structure folder is fine (16*256=4096).

    Thank you,
    Daimonji