Re: [Wonder-disc] EOF & Multithreading

[Miguel A. <ar...@gu...>]

Hi!

   Sorry, I followed up to the wrong list. :(

   Yours

Miguel Arroz

On 2008/07/11, at 16:15, Miguel Arroz wrote:

> Hi!
>
>  I suppose you could use multiple EOF stacks, but I don't know if it  
> will perform better or not. It depends on what you call "waiting for  
> data".
>
>  If "waiting for data" is waiting for the data to be read from the  
> hard drive to memory, then you will make it even worse, because you  
> will cause more seek events, reducing the efective speed.
>
>  If "waiting for data" is waiting for Java to convert the raw data  
> to objects, then yes, you may gain a significative speed boost *if*  
> your machine is multi-core or multi-CPU, because you will be able to  
> do that conversion in parallel, using one core/CPU to each EOF stack.
>
>  Also, think is you can transfer your work for the DB itself. Like  
> if you are counting rows with certain characteristics, etc. If you  
> are working with so much data in a read-only fashion, it might be  
> interesting to push much of the work to the DB itself, to avoid all  
> the expensive overhead of fetching the data to WO, converting it to  
> objects, manage the memory usage, etc.
>
>  Yours
>
> Miguel Arroz
>
> On 2008/07/11, at 16:04, Randy Wigginton wrote:
>
>> I am working on an analysis application that reads tens of millions  
>> of
>> rows in order to analyze website behavior.  Because the application  
>> is
>> so read-intensive, it currently spends 90% of its time waiting for
>> data.  To address this, I am considering moving the DB reading into
>> one or more separate threads in order to more fully utilize the DB  
>> and
>> application servers.   A search did not turn up anything that I found
>> useful.
>>
>> Has anyone on this list done this previously?  Are there any kind of
>> guidelines, best practices, etc?
>>
>> Thanks for any guidance!
>>
>> -------------------------------------------------------------------------
>> Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
>> Studies have shown that voting for your favorite open source project,
>> along with a healthy diet, reduces your potential for chronic  
>> lameness
>> and boredom. Vote Now at http://www.sourceforge.net/community/cca08
>> _______________________________________________
>> Wonder-disc mailing list
>> Won...@li...
>> https://lists.sourceforge.net/lists/listinfo/wonder-disc
>
> http://www.survs.com
>
> _______________________________________________
> Do not post admin requests to the list. They will be ignored.
> Webobjects-dev mailing list      (Web...@li...)
> Help/Unsubscribe/Update your Subscription:
> http://lists.apple.com/mailman/options/webobjects-dev/arroz%40guiamac.com
>
> This email sent to ar...@gu...

http://www.survs.com