|
From: Rene <re...@gr...> - 2002-07-18 06:13:24
|
I've been looking at the file index structure. Maybe using flat files wasn't such a Great Idea(tm) in the long run. I've done some research into using db3 (sleepycat software) instead. Results are nice. It's a really simple database and fairly quick as well. I did a test in which I searched for files larger than x bytes, smaller than y bytes and contains the string "ab" in its name. It took less than 4 seconds to parse the database. That was with ~130000 files. Each files has 4 entries "path","name","date" and "size" The db file is 38 MB in size. The index file I used to fill the database is only 13 MB in size. But the search speed is really amazing compared to the way bobs do it. I tested this on a athlon 1800+ SMP computer. (it only uses on cpu for searching) Check this link for more information. http://dk.php.net/manual/en/ref.dba.php Redhat 7.2 and 7.3 ships php with db3 enabled. To make it completely useable there would have to be at least to databases for each share. One for the files and one for the dirs. In the dirs database the should be a reference to which index numbers that dir has in the file database. eg. index 0 = /somedir/at/some/path/ index 1 = 1000-1304 which would indicate that there are 304 files in this dir and they are located between index 1000 and index 1304 in the file database. This is needed for a really fast search which is needed when we display files in the browser. This way a share with 15000 dirs should take less than 1 second to display a file list in the browser. The real feature is that this is a 1000 times easier to handle in php. Let me know what you think. Is it a good idea to use it? -Rene |