After some long time, I again had a chance to work upon my favourite
filesystem after reiserfs,
flickrfs<http://manishrjain.googlepages.com/flickrfs>.
This release doesn't add nice cool features, this one is targeted towards
robustness, resource utilization and efficiency. The previous versions of
flickrfs had various issues, namely:
- Slow internet connection, time-outs or other errors, would grind
flickrfs to a halt; forbidding it to download information about sets, and
stream.
- If the number of images grow, in order of thousands, flickrfs would
start consuming tons of RAM and eventually swap space; and would literally
make my old thinkpad R40 crash.
- Retrieval of full metadata information for each image was time and
network resource consuming, and mostly not required.
- Sometimes flickr response would return error, even if the uploading
of image succeeds, making flickrfs generate false error report; and
confusing the end-user, who would then end up uploading the same image
twice. (confusing huh? ;-))
- For developers, a.k.a. me, flickrfs code was huge, all in one file,
with lot of boilerplate code sections.
This version v1.3.9 of flickrfs tackles all these issues (also see features
list <http://manishrjain.googlepages.com/flickrfs#feature-list>):
1. Robustness - Flickr operations, as in, interactions with flickr
server are now fail-safe. flickrfs handles the URLError exceptions thrown by
the operation, checks its result, and retries the operation multiple times
in case of failure. Thus, even if connection is lost temporarily, flickrfs
would continue its operations unaffected.
2. Efficiency - flickrfs retrieves the sets information in parallel,
background threads. Hence, the directory structure is created quickly,
allowing users to start working. If some sets couldn't be retrieved the
first time, they'd be taken care of when *syncing* kicks in. Secondly, full
metadata information of images would only be retrieved when asked for;
though they'll always be listed in the directory. Its just like the way
flickrfs handles images.
3. Resource Saving - flickrfs would now store the image information in
Berkeley db database, instead of storing it in memory. Let come millions of
images, flickrfs would still consume only negligible amounts of RAM;
regarding hard disk space, it would be in 10s of Megabytes; not an issue for
today's computers.
4. Refactoring of code - Developers, again a.k.a. me, the code is now
divided into 3 different files and... wait! hmm.. you shouldn't care about
it. Just know that its more beautiful :-).
So, just give it a
whirl<https://sourceforge.net/project/showfiles.php?group_id=151995&package_id=220315&release_id=483298>.
I'm sure you'd like it. More about flickrfs on its
homepage<http://manishrjain.googlepages.com/flickrfs>
.
|
