[Freevo-devel] Re: kaa.vfs benchmarks

[Dirk M. <di...@fr...>]

Jason Tackaberry wrote:
> On Sun, 2005-10-30 at 10:47 +0100, Dirk Meyer wrote:
>> files. It crashed in the ipc code. The code wants to send the whole
>
> Can you show a traceback?

At home. :)

But the trace is simply because the sicket ressource is unavailable
because you send to much data.

>> data on a non blocking socket. Tack: read the help doc: either
>> everything is send or nothing, even for non blocking sockets. As a
>> result, you closed the socket. I changed the code to only send 4096
>> bytes packages each iteration. The result is _very_ bad. It takes 2
>> seconds (!!!) to send the data. I replaced the sending code with afast
>
> Sending only 4kb per each write is indeed going to be a fair bit slower,

My code sends 4k until the ressource is unavailable. It is fast
enough. The 2 seconds is a unix socket problem (see below)

> although 2 seconds seems rather too long.  The proper thing to do here
> is to simply to handle the case where the socket.error exception is
> resource temporarily unavailable and in that case, to return without
> closing the socket.  I've committed such an update to kaa.base -- can
> you see if it works now?
>
>
>> seconds. The server sends no more data. It looks to me that the unix
>> sockets don't report the reading back to the server. It looks like a
>> bad problem with unix sockets to me.
>
> I'm not sure what you mean here.  Can you elaborate?

Server wants to send 200k data. But it can only send 100k until the
socket would block. OK, no problem, return to main and wait until
select tells us that we can write again. The client gets the data,
this results in a select notification that the client can read
data. It reads the 100k. BUT now the server does not receive the
message that 100k are read and that the server can send again. This
happens after 2 seconds both sides doing nothing. I have no idea why.

>> You can test this by the current svn of kaa.vfs, you only need to
>> change the path in test/client.py where to scan for media files. 
>
> I can't reproduce any problem here, but my biggest directory only sends
> 50kb of data over IPC. Maybe that's not enough.

No, you can send 100k without problem.

> This is a bug that needs to get fixed.  

Yes

> We shouldn't not use ipc because it's broken.  We should fix the
> bug.

Yes

> We might decide that ipc has too much overhead for vfs and not use
> for moving db data anyway, but our decision shouldn't be a
> workaround for other brokenness.

Yes, but it looks like we already have a problem with the overhead. 


Dischi

-- 
Don't trust reality. After all, it's only a collective hunch.