Learn how easy it is to sync an existing GitHub or Google Code repo to a SourceForge project! See Demo

Close

#2530 Page Sizes, Nested Mode, Large Discussions

closed-fixed
Rob Malda
Comments (601)
9
2003-07-22
2003-06-11
Rob Malda
No

Pater and I have discussed this quite a bit now, and we
have a ton of stats to back up some pretty substantial
performance issues.

Essentially, there are several bugs working together to
make discussion pages blow up to hugely unweldly
sizes... this gets worse as pages get larger... it is
already evident on discussions with 500 comments...

A discussion with 700 comments at score:1 was
tested.... nested mode resulted in a 450k page,
threaded in 300k.

A story with 4000 comments (2872 at threshold 1) was
tested and was 2.2 *megs* nested, and 700k threaded.

THE BUG Comment Limit is essentially ignored for nested
discussions. We've found pages with hundreds upon
hundreds of comments displayed, in full text, using the
default user preferences. Often exceeding a meg for a
single page. The server issues are many- huge pages,
tons of extra DB calls, to say nothing of the
bandwidth. And these are with all normal user
preferences, save selecting nested mode instead of
threaded. Heaven forbid! What needs to happen is we
need to make a more realistic break in discussions.
We started by dropping comment limit, but it turns out
that this number is largely ignored. Breaks need to be
chosen much more conservatively! We shouldn't be
serving pages that are more than a meg in size /ever/.
I think 500k is a good upper end ceiling.

THE IDEAL SOLUTION: Ultimately I think that COMMENT
LIMIT should be replaced with a maximum discussion byte
count. This shouldn't be all that difficult to do
since our text is all in some variable anyway... of
course, the problem is that we have been avoiding the
transfer of comments_text until the last minute. I
assume we can cache a byte count at least (don't we
already to handle large comments being truncated?) but
we'll also need to account for the size consumed by
HTML for layout and such. We could probably use some
sort of average for reply lines & full comments
displayed since the actual amount of variation between
those is relatively small.

Anonymous users would be given a relatively small
default max discussion size... perhaps a couple hundred
k. We should test a few discussions for byte counts
and see approximately how many comments = how many
bytes so we can make a fair assessement of how small a
page can be before it makes page There is a potential
subscriber plum here as well but we'd want to be
careful to still have some sort of cap. I don't think
we should ever be serving 2 meg files ;)

I think this would be MUCH more clear than the existing
system of comment limits and indexes and splits and
breaks which is extremely confusing.

Discussion

  • Jonathan Pater
    Jonathan Pater
    2003-06-24

    • priority: 7 --> 9
     
  • Jonathan Pater
    Jonathan Pater
    2003-07-01

    Logged In: YES
    user_id=24936

    I've fixed this so that commentlimit finally works, perhaps this
    should be reassigned to me as a feature request, in so far as
    changing commentlimit to bytelimit. We already store the size in
    bytes of each comment, so some of the work is already done.

     
  • Jonathan Pater
    Jonathan Pater
    2003-07-22

    Logged In: YES
    user_id=24936

    Done! I've left the old commentlimit in as well, and page breaks
    are determined now by which limit is hit first. My only worry is
    that linkCommentPages doesn't take bytelimit into account yet,
    but that can be fixed. We also might want to play with how reply
    lines are counted.

     
  • Jonathan Pater
    Jonathan Pater
    2003-07-22

    • assigned_to: cowboyneal --> cmdrtaco
    • status: open --> open-fixed
     
  • Rob Malda
    Rob Malda
    2003-07-22

    • status: open-fixed --> closed-fixed