[Ebiness-crawler] B-tree storage on FS and design questions

[Mike D. <md...@3d...>]

Hi,

Sellaro, you were talking about implementing a B-tree solution for
storage... well why don't we use a filesystem that's build with that kind
of thing in mind:
http://www.namesys.com/

ReiserFs uses a balanced tree for the filesystem, so it would, in theory,
work the same way. We would then have to write an interface to it, but
that would eliminate a whole pile of work.

We wouldn't have to write tree checkers/repair/creation/maintenance progs,
for when the tree got nuked or whatever.

What do you think?

Also, I haven't heard much from anyone for a while - are you guys still
there? :-)

I'm hoping to finish off the crawler docs soon and start working on the
browser docs next week. Can you guys tell me who's interested in which
particular area? I know I want to do both the infrastructure side of the
crawler and the 3D bit of the browser.

Obviously, as the browser hasn't even been looked at, it's open for
suggestions on all aspects. So far I've looked at SDL and OpenGl for the
general 3D side of things and as far as I'm concerned, we only need to do
a Linux (and Solaris) version of the browser, but if anyone is a whiz at
MFC/DirectX, maybe we could get something going for WinXX?

cya
Mike