Re: [Relfs-devel] "LOCAL" #define in Common.h
Status: Pre-Alpha
Brought to you by:
applejack
|
From: Vincenzo a. N. N. <vin...@ya...> - 2004-08-01 22:38:33
|
On Sunday 01 August 2004 22:37, Guido Villa wrote:
> I'm giving a quick look at the source code...
>
> Why not write:
>
> #define LOCAL(path) (path +1)
>
> instead of:
>
> #define LOCAL(path) (string(".")+string(path))
>
First of all, by now we are developing a prototype. C++ is not a good
language for rapid prototyping, neither is C, (I usually choose ocaml
when I am in a hurry, and I have many segfault-free programs written in
a couple of weeks :) ) but I had to choose between cross-language
interoperability and rapidity of development. Since I want a stable
interface for plugins, and the ability to write plugins in almost any
language, I choose C at first. But not having a rich standard library
convinced me to switch to C++ and here we are.
Since I am prototyping, I don't care about performance too much [and I
don't care too much about how I code - I don't code that bad
usually :P ].
My idea is that to make a good design you need to have implemented the
thing; in other words there's something good in that "I feel like
rewriting everything". I want to write a _very small_ and modular
program, and then to audit, and eventually rewrite some part, module by
module. But the plugin interface, and the data model, must be stable,
so that people can write plugins while we correct and /or rewrite the
core, being confident that their plugins will keep on working.
Said this, RelFS.cpp is almost plain C, and it will stay that way or
even more C, so feel free to audit and correct it. Even the class relfs
is unuseful and might be removed. In the new code however there is a
single parameter for all plugin operations (for ease of development),
of type "class File", so a bit of memory allocation is required.
The rest of the program will remain in C++ I guess, if not translated
into a more usable language, because of the need for high level
libraries.
In fact I had defined LOCAL exactly as you suggested, but then I needed
strings in the rest of the program and so I opted for using strings
everywhere. I am not sure about why I choose to add an initial "./",
feel free to experiment removing it to discover why :) However the most
important speedups will come from:
1. using the threaded fuse mainloop. This is necessary, since we can't
block the whole filesystem waiting for a cdrom read. By now this would
mean implementing either a thread waiting for SQL queries and sending
it over a protected connection, or making as many SQL connections as
queries in the program, since libpqxx doesn't accept multiple threads
on a single opened connection. I would opt for solution 1.
2. asynchronous plugins. This is not as easy as it might seem, since we
can't just fork a process or a thread for each indexing operation, or
the system will almost freeze with tons of processes reading from disk.
It would be better to have a queue of indexers to run (and to remove
them from the queue if the file is removed in the meantime). Also, a
good idea would be to try to gather more information about what exactly
plugins will read from a file: if three different indexers will read a
huge file from the beginning to the end, it could be better to allow
them to declare it, and to pass the same data to all of them. Caching
mitigates this problem but this doesn't mean we can't improve it.
>
> I like the plugin proxy idea, because plugin calls (which could be
> slow) can be moved out of sync from the main process.
Oh, not only they can, but they _will_ be moved out of sync asap :) The
most important things for now are a stable index plugins interface and
a good data model. I will think about the data model in the next days.
After these two steps, we will focus on getting maximum speed and
stability from relfs.cpp, including a separate thread or process for
plugins. If other people agree, of course! I am no chief here even if I
am the only coder right now :)
I would be glad to see patches to code, but I recommend not to take care
of details at the moment, because it's highly probable for entire files
to be removed in this phase. It's better to focus on how features can
be implemented (and to see IF certain features can be implemented at
all using the current design).
bye
Vincenzo
--
I was dressed for success,
but success it never comes
[Pavement - Here]
|