slashcode-development — This list is for development of slashcode, no install questions please.

Re: [Slashcode-development] Dynamic story content...

[Chris N. <pu...@po...>]

At 9:16 -0800 2000.12.04, <con...@ho...> wrote:
>does anyone here have experince and/or ideas with making story content
>dynamic? for example putting some PHP / perl / db queries / etc into the
>text of a story so when it's outputted to the page the dynamic stuff would
>be parsed first.
>
>this would be really useful for our site.

There are no plans for this right now.  I think it might be reasonable to
add this in the future, maybe for Slash 2.1 (right now version 1.1 is in
progress, which when released will be 2.0).

-- 
Chris Nandor                      pu...@po...    http://pudge.net/
Open Source Development Network    pu...@os...     http://osdn.com/

[Slashcode-development] New bender release

[Brian A. <br...@ta...>]

Second tarball for bender can now be found on ftp.slashdot.org. This
version includes bug fixes
from the last version along with support for PostgreSQL, plugins and
themes. The user journals
have been added along with a CLI template editor.

[Slashcode-development] RESEND: some performance testing results

[Chris N. <pu...@po...>]

Sigh, slashdev is not sending out my mail in a timely manner, so I am
sending to here.

-------------

OK, the setup: I have two machines, exactly the same, in the same place.
One is running the latest Slash 1.0.x from CVS, and the other is running
bender.

I set up a test where the two sites have a story with the same comments,
hundreds of comments for that story.  I craft URLs so that each site will
give me the data in the same manner (nested, oldest first, threshold 1,
etc. all the way through).  50 comments are displayed in each, along with
anothe hundred or so links to other stories.  I download that URL with lwp
from the OSDN network to my Linux box at home.

So the key differences: one is 1.0.x, one is bender.  Also, 1.0.x is using
a different "theme."  That and the fact that the default bender theme is
still not optimized contributes to these size differences for the
downloaded files:

	1.0.x:		152265 bytes
	bender:		248981 bytes

I downloaded each a bunch of times.  So 1.0.x downloads in one second.
bender downloads in 7 seconds.

If I disable all the templates, so they do not print anything and do not
even get processed, so they take up relatively no time, and less than 1K of
data is sent, then the bender download takes 5 seconds.

Now, my methods and tests may be flawed, but they seem to bear out the
other testing I've done informally over the last months.  There is nothing
extremely scientific in them, I just tried to get some reasonable
comparisons.

So the good news is that ALL of the templates required to process a page
with 50 comments (that means the dispComment template executed 50 times,
linkComment over a hundred times, the header, footer, various other
templates, etc.) and send all that data to the client takes about 2-3
seconds.  That ain't too shabby for all that processing (though it would be
nice to speed it up).  You might know that before, I was getting worse
results in a more informal test, but I had broken caching.  Now that
templates are cached, those templates are processing quite quickly.

The bad news is that we need to find more places to optimize, including
templates, but not exclusive to templates.  This includes the HTML in the
templates, because these resulting pages are bigger than their older
conterparts, and they take significantly longer for my browser to render,
too.

The other good news is that the performance -- except for the HTML
optimization -- seems to be pretty good for most sites.  Most sites won't
have to worry about pages this big, and even if they do, the download time
is still pretty reasonable.  We will be able to leave some of the work on
optimizations, in my opinion, for Slash 2.1.

-- 
Chris Nandor                      pu...@po...    http://pudge.net/
Open Source Development Network    pu...@os...     http://osdn.com/

[Slashcode-development] some performance testing results

[Chris N. <pu...@po...>]

OK, the setup: I have two machines, exactly the same, in the same place.
One is running the latest Slash 1.0.x from CVS, and the other is running
bender.

I set up a test where the two sites have a story with the same comments,
hundreds of comments for that story.  I craft URLs so that each site will
give me the data in the same manner (nested, oldest first, threshold 1,
etc. all the way through).  50 comments are displayed in each, along with
anothe hundred or so links to other stories.  I download that URL with lwp
from the OSDN network to my Linux box at home.

So the key differences: one is 1.0.x, one is bender.  Also, 1.0.x is using
a different "theme."  That and the fact that the default bender theme is
still not optimized contributes to these size differences for the
downloaded files:

	1.0.x:		152265 bytes
	bender:		248981 bytes

I downloaded each a bunch of times.  So 1.0.x downloads in one second.
bender downloads in 7 seconds.

If I disable all the templates, so they do not print anything and do not
even get processed, so they take up relatively no time, and less than 1K of
data is sent, then the bender download takes 5 seconds.

Now, my methods and tests may be flawed, but they seem to bear out the
other testing I've done informally over the last months.  There is nothing
extremely scientific in them, I just tried to get some reasonable
comparisons.

So the good news is that ALL of the templates required to process a page
with 50 comments (that means the dispComment template executed 50 times,
linkComment over a hundred times, the header, footer, various other
templates, etc.) and send all that data to the client takes about 2-3
seconds.  That ain't too shabby for all that processing (though it would be
nice to speed it up).  You might know that before, I was getting worse
results in a more informal test, but I had broken caching.  Now that
templates are cached, those templates are processing quite quickly.

The bad news is that we need to find more places to optimize, including
templates, but not exclusive to templates.  This includes the HTML in the
templates, because these resulting pages are bigger than their older
conterparts, and they take significantly longer for my browser to render,
too.

The other good news is that the performance -- except for the HTML
optimization -- seems to be pretty good for most sites.  Most sites won't
have to worry about pages this big, and even if they do, the download time
is still pretty reasonable.  We will be able to leave some of the work on
optimizations, in my opinion, for Slash 2.1.

-- 
Chris Nandor                      pu...@po...    http://pudge.net/
Open Source Development Network    pu...@os...     http://osdn.com/

Re: [Slashcode-development] template stuff

[Patrick G. <cap...@sl...>]

Chris Nandor wrote:

doesn't really look bad at all. It seems to primarily consist of $value .=
..., which totally makes sense when you consider templates deal with mainly
output.

It looks like the code before we removed the content!

> OK, attached are a pair of compiled templates.  I think the reason why I
> had performance problems with these templates before was that I turned off
> caching by accident.  I need to run some more tests, but I though you might
> want to see the compiled versions of the templates.  You can see why if I
> have 50 comments to view, and this code is created from the template for
> each, it would take a long time.  However, if this code is created only
> once, and executed a bunch of times from a subroutine reference stored in
> the Template object, it should be quite quick.
>
> In fact, I don't see any significant reason why code in a template should
> execute more slowly than code in a module or script, except that code in
> the template is calling more object methods than you would in a script or
> module, which is less efficient.  But I really don't think that difference
> will be significant.
>
> So what does all this mean?  I am going to load up my DB withg a bunch of
> comments and see what happens.  :-)
>
> We still need to discuss caching of templates.  There's a lot to consider.
> First, do we want to keep templates in memory for the life of an httpd
> process?  If not, how should we decide to expire them?  Over time, or with
> an LRU, or based on a timestamp in the DB?  And do we want to cache the
> templates on disk, so all the httpds can share the compiled templates, so
> they are compiled only once?  If we do that, how do we decide to expire
> them?  As I said, a lot to consider there.  We can discuss it some time
> this week.
>
> --
> Chris Nandor                      pu...@po...    http://pudge.net/
> Open Source Development Network    pu...@os...    http://osdn.com/
>
>   ------------------------------------------------------------------------
>                  Name: header.ttc
>    header.ttc    Type: unspecified type (application/octet-stream)
>              Encoding: x-uuencode
>
>                       Name: dispComment.ttc
>    dispComment.ttc    Type: unspecified type (application/octet-stream)
>                   Encoding: x-uuencode

--
Patrick Galbraith               Open Source Development Network
Senior Software Developer       50 Nagog Park
Slash Code Development Team     Acton, MA 01720

"Energy and Persistence conquer all things".
Benjamin Franklin

[Slashcode-development] template stuff

[Chris N. <pu...@po...>]

OK, attached are a pair of compiled templates.  I think the reason why I
had performance problems with these templates before was that I turned off
caching by accident.  I need to run some more tests, but I though you might
want to see the compiled versions of the templates.  You can see why if I
have 50 comments to view, and this code is created from the template for
each, it would take a long time.  However, if this code is created only
once, and executed a bunch of times from a subroutine reference stored in
the Template object, it should be quite quick.

In fact, I don't see any significant reason why code in a template should
execute more slowly than code in a module or script, except that code in
the template is calling more object methods than you would in a script or
module, which is less efficient.  But I really don't think that difference
will be significant.

So what does all this mean?  I am going to load up my DB withg a bunch of
comments and see what happens.  :-)

We still need to discuss caching of templates.  There's a lot to consider.
First, do we want to keep templates in memory for the life of an httpd
process?  If not, how should we decide to expire them?  Over time, or with
an LRU, or based on a timestamp in the DB?  And do we want to cache the
templates on disk, so all the httpds can share the compiled templates, so
they are compiled only once?  If we do that, how do we decide to expire
them?  As I said, a lot to consider there.  We can discuss it some time
this week.

-- 
Chris Nandor                      pu...@po...    http://pudge.net/
Open Source Development Network    pu...@os...     http://osdn.com/

[Slashcode-development] Slash Docs

[Chris N. <pu...@po...>]

OK, I did an overhaul of the POD for Slash and Slash::Utility, and some
minor changes to Slash::Display, Slash::Display::Plugin, and
Slash::Display::Provider.

If you have any questions about the docs or how they should be formatted,
ask.  Here's a recap.  In the =head2 part, put the optional parameters in
the [].  List all the parameters under "Paramaters" (one per =item).
"Return value" is required, all the others should only be there if there is
something to put there (that is, if no Parameters, leave that section out,
same for Side effects and Dependencies).  See the existing POD for examples.


#========================================================================

=head2 foo( [, ])

Describe function here.

=over 4

=item Parameters

=over 4

=item

=back

=item Return value


=item Side effects


=item Dependencies

=back

=cut



-- 
Chris Nandor                      pu...@po...    http://pudge.net/
Open Source Development Network    pu...@os...     http://osdn.com/

Re: [Slashcode-development] RSS 1.0

[Chris N. <pu...@po...>]

At 16:19 +0000 2000.12.04, Dave Aiello wrote:
>> OK, here is a list of the data for stories that we have available,
>> that we might want to put in to RSS 1.0 files.  I listed pretty much
>> all the data for stories, and just a few things for site.  Comments
>> welcome.
>>
>> The purpose of this is to put these items into available RSS fields,
>> either in the RSS core in Dublin Core, or in our own Slash XML module.
>>
>> From here, we could then move on to defining fields for comments
>> and users, and any other data we want to search on, for the purpose
>> of returning search data to users and to any other searches we want
>> to enable.
>>
>
>CTDATA supports RSS 0.91 on all of its Slash 0.3-based sites.  In our RSS file
>creation process we map introtext to description.  The description field is
>used by a number of RSS aggregation sites, including my.userland.com and
>xmltree.com.
>
>Is there any reason why this same approach would not be advisable under RSS
>1.0, apart from the fact that some people in the RSS 1.0 working group have
>tried to "route around" RSS 0.91, apparently for political reasons?

Well, it is not so much political reasons, but philosophical reasons.  RSS
0.9 was "RDF Site Summary."  RSS 0.91 dropped the "RDF."  They think RDF is
the right way to go, so went back to RDF.  This is better IMO because with
RSS 0.91 and its path, to extend RSS you needed to come out with a new
version of RSS.  With RSS 1.0, you can extend it by adding new modules.
Very nice, IMO.

And that's the whole point: we now have a set of metadata for these
stories, and we want to include them in our RSS files.  With RSS 0.91, we
would have to find a way to squeeze them into existing elements.  With RSS
1.0, we can see if existing elements work for us, and for those remaining
elements, we can create a new module and add them in there.  We will try to
use existing elements, just like you do under 0.91.  But where those
elements don't work, we will create our own.

Does this make sense, or did I miss your point?  Thanks,

-- 
Chris Nandor                      pu...@po...    http://pudge.net/
Open Source Development Network    pu...@os...     http://osdn.com/

[Slashcode-development] Dynamic story content...

[<con...@ho...>]

greetings,

does anyone here have experince and/or ideas with making story content
dynamic? for example putting some PHP / perl / db queries / etc into the
text of a story so when it's outputted to the page the dynamic stuff would
be parsed first.

this would be really useful for our site.

ciao,
k

Re: [Slashcode-development] RSS 1.0

[<con...@ho...>]

i just got on the list. mind giving me a little history on this request so i
can get my mind cranking in the right direction?

ciao,
k

----- Original Message -----
From: "Chris Nandor" <pu...@po...>
To: <sla...@li...>
Sent: Monday, December 04, 2000 6:17 AM
Subject: [Slashcode-development] RSS 1.0


> OK, here is a list of the data for stories that we have available,
> that we might want to put in to RSS 1.0 files.  I listed pretty much
> all the data for stories, and just a few things for site.  Comments
> welcome.
>
> The purpose of this is to put these items into available RSS fields,
> either in the RSS core in Dublin Core, or in our own Slash XML module.
>
> From here, we could then move on to defining fields for comments
> and users, and any other data we want to search on, for the purpose
> of returning search data to users and to any other searches we want
> to enable.
>
> site
> ----
> URL
> name
> language
> slogan (description)
> contact email
> anything else?
>
>
> stories
> -------
> URL (story ID)
> title
> author
> section
> topic
> department
> number of comments
> date
>
> other (probably not useful, or should not be included):
> introtext
> bodytext
> writestatus
> hits
> displaystatus
> commentstatus
> hitparade
> relatedtext
> extratext
>
> --
> Chris Nandor                      pu...@po...    http://pudge.net/
> Open Source Development Network    pu...@os...     http://osdn.com/
> _______________________________________________
> Slashcode-development mailing list
> Sla...@li...
> http://lists.sourceforge.net/mailman/listinfo/slashcode-development
>

Re: [Slashcode-development] RSS 1.0

[Dave A. <dav...@ct...>]

> OK, here is a list of the data for stories that we have available,
> that we might want to put in to RSS 1.0 files.  I listed pretty much
> all the data for stories, and just a few things for site.  Comments
> welcome.
> 
> The purpose of this is to put these items into available RSS fields,
> either in the RSS core in Dublin Core, or in our own Slash XML module.
> 
> From here, we could then move on to defining fields for comments
> and users, and any other data we want to search on, for the purpose
> of returning search data to users and to any other searches we want
> to enable.
> 

CTDATA supports RSS 0.91 on all of its Slash 0.3-based sites.  In our RSS file 
creation process we map introtext to description.  The description field is 
used by a number of RSS aggregation sites, including my.userland.com and 
xmltree.com.

Is there any reason why this same approach would not be advisable under RSS 
1.0, apart from the fact that some people in the RSS 1.0 working group have 
tried to "route around" RSS 0.91, apparently for political reasons?

Dave Aiello
CTDATA

[Slashcode-development] RSS 1.0

[Chris N. <pu...@po...>]

OK, here is a list of the data for stories that we have available,
that we might want to put in to RSS 1.0 files.  I listed pretty much
all the data for stories, and just a few things for site.  Comments
welcome.

The purpose of this is to put these items into available RSS fields,
either in the RSS core in Dublin Core, or in our own Slash XML module.

From here, we could then move on to defining fields for comments
and users, and any other data we want to search on, for the purpose
of returning search data to users and to any other searches we want
to enable.

site
----
URL
name
language
slogan (description)
contact email
anything else?


stories
-------
URL (story ID)
title
author
section
topic
department
number of comments
date

other (probably not useful, or should not be included):
introtext
bodytext
writestatus
hits
displaystatus
commentstatus
hitparade
relatedtext
extratext

-- 
Chris Nandor                      pu...@po...    http://pudge.net/
Open Source Development Network    pu...@os...     http://osdn.com/

[Slashcode-development] Template problems

[Chris N. <pu...@po...>]

OK, awhile back I broke Template caching.  Note: this very well may be why
I had poor performance.  Makes sense.  So anyway, the reason why I
(unknowingly) broke it was that I was having problems with variable values
in the templates being persistent, when I thought they shouldn't be.  So
the short of it is that after my next commit (today before 5 p.m. :),
please let me know if you see any weird variable value persistance problems
in templates.  I am going to do a lot more of tearing into the guts of
templates next week, and will try to find it, but if you happen to see it,
that would be good.  Thanks,

-- 
Chris Nandor                      pu...@po...    http://pudge.net/
Open Source Development Network    pu...@os...     http://osdn.com/

Re: [Slashcode-development] one-liner patch to install-slashsite

[Brian A. <br...@ta...>]

Jamie McCarthy wrote:
> (not that apxs works anyway on my setup, I'm guessing this isn't
> vital since mod_so is required and it's still listed as
> "experimental" under unix)
I haven't pulled out the reference to apxs yet, but I will be.
I was thinking today that it might make sense to generate
a httpd.conf include file for people if they want to go
that route. It would be nice to have a set of tables in
a separate instance someday that stored the values of 
installs (and might make for a good place to build a 
multi-site admin tool).
	-Brian

Re: [Slashcode-development] Frickin' Reply-To:'s

[Chris N. <pu...@po...>]

At 12:05 -0800 2000.11.29, Dave Krieger wrote:
>Apologies for multiple spams to slashcode-development today... I keep
>trying to reply to Chris, but his "Today's Commits" announcement has a
>"Reply-To" header for the list that I continue to fail to notice until
>after I send the message.  Whoever's moderating these, feel free to blow
>those away.

Actually, I think posts to the list are better anyway.  That way we can all
read what each other is doing, help out when someone doesn't know an
answer, etc.

-- 
Chris Nandor                      pu...@po...    http://pudge.net/
Open Source Development Network    pu...@os...     http://osdn.com/

[Slashcode-development] one-liner patch to install-slashsite

[Jamie M. <ja...@mc...>]

A suggested patch to install-slashsite:

$ cvs diff install-slashsite 
Index: install-slashsite
===================================================================
RCS file: /cvsroot/slashcode/slash/bin/Attic/install-slashsite,v
retrieving revision 1.1.2.17
diff -r1.1.2.17 install-slashsite
191a192
>       local $ENV{PATH} = "$ENV{PATH}:/usr/local/apache/bin";

(not that apxs works anyway on my setup, I'm guessing this isn't
vital since mod_so is required and it's still listed as
"experimental" under unix)
--
        Jamie McCarthy
        ja...@mc...    <-- note change
 http://jamie.mccarthy.org/

Re: [Slashcode-development] today's commits

[Patrick G. <cap...@sl...>]

Brian Aker wrote:

> Dave Krieger wrote:
> > 1) Those new commits don't include the Template editor yet, right?
> Not sure, ask Pat.

yes, the latest on CVS has all that

>
>
> > 2) Did you guys ever succeed in duplicating the "POSTs don't work but GETs do" bug?  With the new stuff today, it seems to be intermittent rather than consistent; did anyone intentionally try to address that, or did I just get lucky?  (This will help me diagnose whether this is possibly a local Apache problem rather than a Slash problem...)
> >
> There were a number of bugs with the different templates, but I don'
> know of
> any where just GET work.
>         -Brian
> _______________________________________________
> Slashcode-development mailing list
> Sla...@li...
> http://lists.sourceforge.net/mailman/listinfo/slashcode-development

[Slashcode-development] Frickin' Reply-To:'s

[Dave K. <da...@fq...>]

Apologies for multiple spams to slashcode-development today... I keep trying to reply to Chris, but his "Today's Commits" announcement has a "Reply-To" header for the list that I continue to fail to notice until after I send the message.  Whoever's moderating these, feel free to blow those away.

Dave

Re: [Slashcode-development] today's commits

[Chris N. <pu...@po...>]

At 11:25 -0800 2000.11.29, Dave Krieger wrote:
>1) Those new commits don't include the Template editor yet, right?

I dunno.


>2) Did you guys ever succeed in duplicating the "POSTs don't work but GETs
>do" >bug?  With the new stuff today, it seems to be intermittent rather
>than
>consistent; did anyone intentionally try to address that, or did I just
>get lucky?  (This will help me diagnose whether this is possibly a local
>Apache problem rather than a Slash problem...)

I've never seen the problem, that I know of.

-- 
Chris Nandor                      pu...@po...    http://pudge.net/
Open Source Development Network    pu...@os...     http://osdn.com/

[Slashcode-development] Real bug or deprecated feature?

[Dave K. <da...@fq...>]

When I try to create an Author, I get the following in the Apache error log:

[Wed Nov 29 12:49:38 2000] [error] Can't locate object method "createAuthor" via package "Slash::DB" at /usr/local/slash/bender.perldiver.com/htdocs/admin.pl line 283.

Is this a legititmate bug, or did this function go away as a result of the merger of Users and Authors?  (If the latter, what is the new interface for upgrading Users to Authors?)

Re: [Slashcode-development] today's commits

[Brian A. <br...@ta...>]

Dave Krieger wrote:
> 1) Those new commits don't include the Template editor yet, right?
Not sure, ask Pat.

> 2) Did you guys ever succeed in duplicating the "POSTs don't work but GETs do" bug?  With the new stuff today, it seems to be intermittent rather than consistent; did anyone intentionally try to address that, or did I just get lucky?  (This will help me diagnose whether this is possibly a local Apache problem rather than a Slash problem...)
> 
There were a number of bugs with the different templates, but I don'
know of 
any where just GET work.
	-Brian

Re: [Slashcode-development] today's commits

[Dave K. <da...@fq...>]

BTW, just wanted to check:

1) Those new commits don't include the Template editor yet, right?

2) Did you guys ever succeed in duplicating the "POSTs don't work but GETs do" bug?  With the new stuff today, it seems to be intermittent rather than consistent; did anyone intentionally try to address that, or did I just get lucky?  (This will help me diagnose whether this is possibly a local Apache problem rather than a Slash problem...)

[Slashcode-development] today's commits

[Chris N. <pu...@po...>]

I made a bunch of commits just now, all having to do with the strip_*
functions mentioned previously, and the 1.0.9.x code.  That is, I've merged
all the changes from v1_0_9_0 -> v1_0_9_6 into bender.

I've started on documentation, I'll do more before the day is out, so y'all
can get started.

When Rael announces his presence (you there?) I'll post stuff about our
story data and metadata so we can get started on RSS 1.0.

-- 
Chris Nandor                      pu...@po...    http://pudge.net/
Open Source Development Network    pu...@os...     http://osdn.com/

[Slashcode-development] RSS 1.0

[Chris N. <pu...@po...>]

We will be discussing the possiblity of incorporating RSS 1.0 into Slash
tomorrow at noon Eastern time on #slash, on slashnet (see
http://www.slashnet.org/).

Background:

	http://www.egroups.com/files/rss-dev/specification.html

-- 
Chris Nandor                      pu...@po...    http://pudge.net/
Open Source Development Network    pu...@os...     http://osdn.com/

[Slashcode-development] stripByMode literal

[Chris N. <pu...@po...>]

In bender, this is strip_literal($str).  In MAIN, it is stripByMode($str,
'literal').  It basically just converts the existing string to a literal
counterpart that can be put directly into HTML, converting the < and > and
& appropiately.

This function does NOT insert whitespace into the strings, because it
should only be called by internal functions, never for a comment, but only
for inserting literal HTML into a TEXTAREA or somesuch.

So instead of all the calls to stripByMode($str, 'literal', 1) where the 1
would denote that whitespace should not be inserted, now whitespace will
not be inserted simply by virtue of the mode being 'literal'.

-- 
Chris Nandor                      pu...@po...    http://pudge.net/
Open Source Development Network    pu...@os...     http://osdn.com/