feedreader-development

[Feedreader-development] About crashes with big databases

[Toomas T. <to...@i-...>]

Hello!

I think I know what is the problem of Feedreader when subscribing to a 
lot of feeds (i have tested with 1000). Problem is not about XML files 
but that headline search function and headline database structures are 
not optimized for a lot of articles.

Let's say that we have 1000 feeds and every feed has 100 articles (i 
think it can be even more, i have some archives set to 500). So now we 
have 100000 articles from where to search. Search must be done every 
time new headline has to be added to headline database and every time a 
feed is selected from left pane. Also this search is runned then 
clicking on listview...

But what does our search function look like? Like this :
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Function GetHeadlineIndex( Owner: real; Const headline, link: string; 
read: Integer): Integer;
var
  i: integer;
begin
//  EnterCriticalSection(gCritSect);
  try
    for i := 0 to length(gHeadlines) - 1 do
    begin

      if (not gHeadlines[i].deleted) and
         ((owner = gHeadlines[i].owner) or (owner = 0)) and
         ((read = gHeadlines[i].IsRead) or (read = -1)) and
         ((headline = '') or (pos(headline, gHeadlines[i].tiitel) > 0)) and
         ((link = '') or (pos(link, gHeadlines[i].link) > 0))
      then begin
        Result := i;
        Exit;
      end;

    end;
  finally
//    LeaveCriticalSection(gCritSect);
  end;

  Result := -1;
end;
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

This is dumbest algorithm to use for this kind on job (it was designed 
by me :)))) if there are a lot off items in array. It just goes through 
entire headline list searching for items.

So, nothing will became better with 500+ subscription databases, if we 
have this kind of function. I have even thought about adding database 
engine to FR (http://sourceforge.net/projects/tpflashfiler/). BTW this 
flashfiler went opensource in jan. 2004. Before it was product of 
TurboPower. This makes me think that it can have some quality.

Greetings
Toomas

Re: [Feedreader-development] About crashes with big databases

[Nicole S. <ni...@gn...>]

Toomas Toots <to...@i-...> wrote:
>have 100000 articles from where to search. Search must be done every 
>time new headline has to be added to headline database and every time a 
>feed is selected from left pane. 

This would fit for my behaviour of speed reading = selecting much headlines
in short times, because I doubleclick them and read them afterwards.

But it would not explain why, even if I leave it alone to get the feeds, it
does mark everything. But I asume it is a piece of the puzzle.

Nicole
	
-- 
http://ideengeberin.de/ - pragmatische Ideen für KMU
http://beissholz.de/ - von bissig bis garstig

Re: [Feedreader-development] About crashes with big databases

[Karsten H. <kh...@kh...>]

Nicole Simon schrieb:
> Toomas Toots <to...@i-...> wrote:
> 
>>have 100000 articles from where to search. Search must be done every 
>>time new headline has to be added to headline database and every time a 
>>feed is selected from left pane. 
> 
> 
> This would fit for my behaviour of speed reading = selecting much headlines
> in short times, because I doubleclick them and read them afterwards.
> 
> But it would not explain why, even if I leave it alone to get the feeds, it
> does mark everything. But I asume it is a piece of the puzzle.
> 

One problem could be that the search algorithm is running while the
content of the headline array is changed (updated) by other threads.

Karsten

Re: [Feedreader-development] About crashes with big databases

[Karsten H. <kh...@kh...>]

I've heard only good about Flashfiler. Only question is, does it need to 
be installed or are some dll's in the application directory enough?

Toomas Toots schrieb:
> Hello!
> 
> I think I know what is the problem of Feedreader when subscribing to a 
> lot of feeds (i have tested with 1000). Problem is not about XML files 
> but that headline search function and headline database structures are 
> not optimized for a lot of articles.
> 
> Let's say that we have 1000 feeds and every feed has 100 articles (i 
> think it can be even more, i have some archives set to 500). So now we 
> have 100000 articles from where to search. Search must be done every 
> time new headline has to be added to headline database and every time a 
> feed is selected from left pane. Also this search is runned then 
> clicking on listview...
> 
> But what does our search function look like? Like this :
> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 
> 
> Function GetHeadlineIndex( Owner: real; Const headline, link: string; 
> read: Integer): Integer;
> var
>  i: integer;
> begin
> //  EnterCriticalSection(gCritSect);
>  try
>    for i := 0 to length(gHeadlines) - 1 do
>    begin
> 
>      if (not gHeadlines[i].deleted) and
>         ((owner = gHeadlines[i].owner) or (owner = 0)) and
>         ((read = gHeadlines[i].IsRead) or (read = -1)) and
>         ((headline = '') or (pos(headline, gHeadlines[i].tiitel) > 0)) and
>         ((link = '') or (pos(link, gHeadlines[i].link) > 0))
>      then begin
>        Result := i;
>        Exit;
>      end;
> 
>    end;
>  finally
> //    LeaveCriticalSection(gCritSect);
>  end;
> 
>  Result := -1;
> end;
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 
> 
> 
> This is dumbest algorithm to use for this kind on job (it was designed 
> by me :)))) if there are a lot off items in array. It just goes through 
> entire headline list searching for items.
> 
> So, nothing will became better with 500+ subscription databases, if we 
> have this kind of function. I have even thought about adding database 
> engine to FR (http://sourceforge.net/projects/tpflashfiler/). BTW this 
> flashfiler went opensource in jan. 2004. Before it was product of 
> TurboPower. This makes me think that it can have some quality.
> 
> Greetings
> Toomas
> 
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by: Oracle 10g
> Get certified on the hottest thing ever to hit the market... Oracle 10g. 
> Take an Oracle 10g class now, and we'll give you the exam FREE.
> http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click
> _______________________________________________
> Feedreader-development mailing list
> Fee...@li...
> https://lists.sourceforge.net/lists/listinfo/feedreader-development
>