#55 Designing how empty space will be used

open
nobody
5
2012-09-12
2011-06-11
Anonymous
No

I have a project where i have 1000+ sqlite databases as many small files, and when running my project, hundereds of thousands (eventually millions) of additions are made to these files by insert commands, since files are added to in random, ALL files get fragmented immediately, and continuously! I eventually get hundereds of files with thousands+ of fragments, between only 20-500 mb in size.

one way to avoid this is of course pre allocate the file with software (making a file 50 mb on the first kb write).

but i thought, "why not do this with a defrag program?" And this will also have many more uses for all kinds of applications

the only thing needed to do is:

let user choose WHERE to put the beginnings of certain files, in the empty space on a partition, sort of like a MANUAL defrag..

just by putting the beginnings of my files, 50 MB apart, this will automatically be solved!
Because: windows adds data to the end of files i think.

yes i am aware that freebsd and linux are much better than windows on file system efficiency, and i will move there eventually,
but windows put all my gigabytes of to-be-continuously inflated files in the FIRST 9 GBs of a 780 GB partition (because i moved all files from a different drive to this one), and defrag does not really improve the situation for the future.

Discussion

  • i have now a 3.2 GB database file in 9416 fragments :-) most of it is only at the end...

     
  • Hi friends,

    I recommend all the people using computers primarily for database management to give Linux a chance to handle all their databases. As you have already mentioned, Linux/BSD uses much more efficient file systems which are much more suitable for databases store.


    Dmitri