Re: [Themis-dev] HTTP/TCP layer changes

[Raymond R. <ra...@ba...>]

On 08/17/2011 01:55 PM, Mark Hellegers wrote:
>> Hi Mark (and anyone else out there) :)!      I've been looking at the
>> network code trying to solve the deadlock issue that was pointed out, and
>> I have it partially solved, with a good idea about what else I'll need to
>> do. The problem seems to be that the BeOS/Haiku BLock&  BAutolock don't
>> quite work as advertised, or at least that I didn't understand them
>> properly. The way I understood them, a thread controlling a given lock
>> can enter code that requests that lock to be in place as much and as
>> often as necessary, but it doesn't seem to be behaving that way. Or
>> perhaps I'm still not grasping some of the nuances.
> Hey Raymond,
>
> as far as I know your understanding is correct.
> The BLocker documentation even gives an example of how this works.
> Are you sure you are balancing the Lock and Unlock calls?
Yeah, I double checked all of them, and in some places I'm using 
BAutolock which should just simply handle it, but I'm still getting 
deadlocks. I'm trying to figure out if I should convert everything over 
to BAutolock or go back through and convert Lock() to LockWithTimeout().
>>      Nonetheless, I've commented out a few of the locks, and Mark's
>> experimental code, more often than not, seems to be flying along,
>> requesting retrieval of images on pages that have been successfully
>> parsed. There are still some deadlock situations arising, but I'll be
>> able to eliminate those with time.
> Themis is finally starting to act like a browser now :)
> Now we just need to render all the downloaded content in the right way.
> Can't be that hard. ;)
Not at all! lol We only have a million things left to do!
>>      On other news, in the TCP layer (in particular in the connection
>> class), I have increased the size of the buffer used to actually receive
>> data from the network (in a recv call if I remember correctly) from 10 KB
>> to 1 MB. As a result, the estimated bytes per second calculation has
>> typically jumped into the hundreds of thousands of bytes rather than a
>> few tens of thousands. Related to that, I have moved code that actually
>> processes that data in the HTTP layer from the layer manager loop into
>> the data is waiting notification callback function.
> Yeah, my benchmark download of 10MB went from 140kB/s to 260kB/s. Nearly
> double.
> Netpositive still manages 400kB/s though. ;)
I made another buffer size change that might be more beneficial, but 
we'll see...
>>      I'm sure I probably introduced new bugs as a result of these
>> changes, but I'm not done yet. I just wanted to put the code into someone
>> else's hands to test.
> I hope you can squash the tcp manager cpu usage and crash I mentioned in
> jabber.
> If those get fixed, we can reliably try and render stuff with Themis.
>
> Thanks for the changes,
>
> Mark
>
Not a problem... I'd commit my more recent changes but I think I've done 
nothing but make a bigger mess of the whole thing... I'm getting a lot 
more crashes with my more recent changes and I'm not entirely sure why. 
I haven't worked with it in a week now, otherwise I'd tell you exactly 
what I'm seeing, but I hope to get back to it this weekend.

Raymond