nms-cgi-devel — Developers discussion for NMS CGI programs

[Nms-cgi-devel] CVS Commit %s

[Jonathan S. <gel...@us...>]

uid=68026(gellyfish) gid=100(users) groups=100(users),40625(nms-cgi)
formmail FormMail.pl,1.4,1.5
Wed Nov 14 01:10:11 PST 2001
Update of /cvsroot/nms-cgi/formmail
In directory usw-pr-cvs1:/tmp/cvs-serv25090

Modified Files:
	FormMail.pl 
Log Message:
Added extra check on the referer.

Re: [Nms-cgi-devel] Further Updates to countdown.pl

[ryan.311 <rya...@os...>]

A quick fix:

>Ugh... Fine.  I added support for both ways, however:
>
>my @query_string = split(/,/, length param("date") > 0 ? param("date") : 
>unescape($ENV{QUERY_STRING}));
>@from_date = @query_string if(@query_string == 6);
>
>This will let you use either countdown.pl?2002,9,7,0,0,0 or 
>countdown.pl?date=2002,9,7,0,0,0.

Wog from perlmonks informed me that calling param("keywords") will return 
2002,9,7,0,0,0 when the query string is countdown.pl?2002,9,7,0,0,0.  So 
that makes the code:

my @query_string = split(/,/, length param("date") > 0 ? param("date") : 
param("keywords"));
@from_date = @query_string if(@query_string == 6);

[Nms-cgi-devel] Further Updates to countdown.pl

[ryan.311 <rya...@os...>]

Hey guys, sorry I haven't been responding in a while; I've been busy. 
Anyways, here are some thoughts:

 >You also changed the shebang line to #!perl -wT. We need to agree on a
 >standard for shebang lines, but it'll probably be #!/usr/bin/perl -wT

Well, I was under NT when I was running it... I changed it when I uploaded 
it this time.  Apologies!


 >I don't agree with having use CGI::Carpi qw(fatalsToBrowser) in production
 >code. It gives too much information to anyone trying to crack the site. The
 >standard CGI error message is vague for a good reason. The full error
 >messsage is written to the server error log where the webmaster can read
 >it. By all means add it to the code - but comment it out in released
 >versions.

I thought Jonathan's idea of using CGI::Carp's set_message function was an 
excellent idea.
However, the code snippet he provided didn't work ;)  I changed it to this:

BEGIN{set_message(sub{print"[Error]"})if!$DEBUGGING;}

That will output "[Error]" whenever there is an error in the script when 
$DEBUGGING is set to 0.
However, if there is an error earlier in the BEGIN statement, CGI::Carp 
will output errors regardless
of what $DEBUGGING is set to.

In addition, I added all of the "config" variables to global variable 
declarations and placed them
inside the begin.  $DEBUGGING needed to be a pre-declared value in order 
for set_message
to work correctly; and I figured that all of the "config" variables should 
be placed together to
prevent confusion.

 >Maybe we should have used the query_string functixpno from CGI.pm as that
 >probably does the URL unencoding.

Hmmm... I couldn't find that function in the pod.
Are you talking about unescape() in CGI::Util?
Good idea, I'll put that in.

 >Otherwise, we _can't_ change the format of the parameters as this script is
 >supposed to work exactly the same way as Matt's. Users are supposed to be
 >able to copy our version on top and Matt's and it'll work straight off. If
 >you change the format, then that's not going to happen. I agree that your
 >format makes far more sense, but because of the rules of nms, we're stuck
 >with Matt's format.

Ugh... Fine.  I added support for both ways, however:

my @query_string = split(/,/, length param("date") > 0 ? param("date") : 
unescape($ENV{QUERY_STRING}));
@from_date = @query_string if(@query_string == 6);

This will let you use either countdown.pl?2002,9,7,0,0,0 or 
countdown.pl?date=2002,9,7,0,0,0.


 >> 4. I removed the start_html and end_html, big bold headers, etc. A script
 >> like this will most likely be used via SSI, so it makes sense
 >> to only output what is needed. Let the designer decide the output format :)
 >
 >Another good idea, but again we're stuck with Matt's definition of the
 >output. What would be good would be to have a simple template stored in the
 >__DATA__ filehandle. The default would need to be Matt's version, but it
 >would be easy to edit to something else.

I don't thing it really matters whether the start_html, etc. are not 
included; however, it does make a difference
if they are. If the script is used as stand-alone, the page will still 
display despite lack of head information; however,
html head information in the middle of the document is NOT a good idea.  If 
the end user wants big boldness, then
they can wrap the SSI in <h1> tags.  Lets not make the same mistake Matt 
did. :)

Also, some extra notes:
First of all, the version of CGI::Carp that we need is 1.09, according to 
its pod.  However, I am not sure what version first became start of the 
standard distribution.

Secondly, I am not sure how to upload on sourceforge.  For the time being, 
I re-uploaded to jcwren's server (http://jryan.perlmonk.org/nms/countdown.txt).

Thirdly, I have two midterms coming up over the next two days, so my time 
for NMS will be limited until my midterms are over.  I should be able to 
thoroughly look over another script during that time, however, so 
suggestions for the next script to pick apart are welcome.

Forthly (to Dave): My last name is Ryan, not my first.  Call me Joe please :)

CGI::Carp (was Re: [Nms-cgi-devel] Welcome)

[Jonathan S. <gel...@ge...>]

On Tue, 13 Nov 2001, Dave Cross wrote:
>
> I certainly like this way of doing things. If it works under our target
> version then I'll be more than happy to use it.
>

I altered all the programs to use it anyhow as I got bored on the train :)
I'll take it out or think of an alternative if it transpires that its a no
go - it's not all that much code in Carp.pm anyhow - we could implement
it in our own code if necessary.

/J\
-- 
Jonathan Stowe                      |
<http://www.gellyfish.com>          |      This space for rent
                                    |

[Nms-cgi-devel] NMS-Config

[Jonathan S. <gel...@ge...>]

Whilst looking at the body of code and thinking that people might be using
more than one of these programs I am thinking of a potential enhancement
that would aid the support burden.

I am thinking that we should have a configuration tool that will allow the
user to configure, via the web, all of the options for all of the
installed applications.  I have a basic idea how this might be implemented
- anyone got any feelings ?

/J\
-- 
Jonathan Stowe                      |
<http://www.gellyfish.com>          |      This space for rent
                                    |

Re: [Nms-cgi-devel] Welcome

[Dave C. <da...@da...>]

On Tue, Nov 13, 2001 at 07:15:15PM +0000, Jonathan Stowe (gel...@ge...) wrote:

[on CGI::Carp]

> I've got a better idea:
> 
> use CGI::Carp qw(fatalsToBrowser set_message);
> 
> use vars qw($DEBUGGING);
> 
> BEGIN
> {
>    $DEBUGGING = 1;
> 
>    $error_handler = sub
>                     {
>                       my ($mesg) = @_;
>                       print "text/html\n\n";
>                       print "<strong>Something nasty happened</strong>";
> 
>                       print "Perl said : $message" if $DEBUGGING;
>                     }
> 
>     set_message($error_handler);
> }
> 
> I'm not sure when that feature came about :)

This is important to know. I've made the (arbitrary) decision that nms
scripts should work under standard Perl 5.004_04. If we insist on anything
later we'll alienate a large part of our potential audience. I still run
web sites on hosted servers that run 5.004_04!
 
> Then you get the best of both worlds :)  It has to be borne in mind that
> someone installing the application and configuring it is likely to find
> themselves debugging their install environment - it would be nice to be
> able to tell them to just alter the $DEBUGGING variable to display
> suitable diagnostics - not that I'm subscribed to the support list ;-}

I certainly like this way of doing things. If it works under our target 
version then I'll be more than happy to use it.

I'll see if I can find out which version of CGI and CGI::Carp shipped with
Perl 5.004_04.

Dave...

-- 

  Don't dream it... be it

[Nms-cgi-devel] Altered CVS

[Jonathan S. <gel...@ge...>]

I have added a new directory 'docs' in the 'NMS' module which I intend to
use for non specific documentation such as a FAQ etc.

Ignore any module 'doc' you might see - this is an appalling import cockup
I made which I can't completely expunge without shell access to the CVS
server ;-}

/J\
-- 
Jonathan Stowe                      |
<http://www.gellyfish.com>          |      This space for rent
                                    |

Re: [Nms-cgi-devel] Re: Style

[Dave C. <da...@da...>]

On Tue, Nov 13, 2001 at 09:02:29PM +0000, Jonathan Stowe (gel...@ge...) wrote:
> I noticed in formmail/FormMail.pl that there where some &some_sub; things
> - is this a legacy of the original source and can I fix them to be be
> some_sub() in the club style or is it someones preference :)

No, that's fine. In fact, your way is safer as it doesn't implicity pass
on the contents of @_.
 
> On this matter do we have any style guidelines - you may have noticed that
> my indenting style is somewhat different to Dave's and I don't want to
> upset anyone.

I don't think there's any need to impose style on anyone. But if you're
editing a file, please stay consistant with the existing style.

Does anyone have any objections to that?

Dave...

-- 

  .sig missing...

[Nms-cgi-devel] Re: Style

[Jonathan S. <gel...@ge...>]

I noticed in formmail/FormMail.pl that there where some &some_sub; things
- is this a legacy of the original source and can I fix them to be be
some_sub() in the club style or is it someones preference :)

On this matter do we have any style guidelines - you may have noticed that
my indenting style is somewhat different to Dave's and I don't want to
upset anyone.

/J\
-- 
Jonathan Stowe                      |
<http://www.gellyfish.com>          |      This space for rent
                                    |

Re: [Nms-cgi-devel] Welcome

[Jonathan S. <gel...@ge...>]

On Tue, 13 Nov 2001, Dave Cross wrote:
> From: Jonathan Stowe <gel...@ge...>
>
> > and added CGI::Carp in those places where the script
> > might die() this will at least produce something
> > moderately less confusing than a plain 500 in the event
> > of failure
>
> I don't like this idea. I think that web servers give vanilla
> 500 errors for a good reason. The more information a user gets
> from an error message then the more information they have about
> the workings of the script and the more chance they have of cracking
> the site.
>
> I'm happy to have CGI::Carp included in the scripts and used
> for development and debugging - but I'd like it commented out
> of anything checked into CVS.
>

I've got a better idea:

use CGI::Carp qw(fatalsToBrowser set_message);

use vars qw($DEBUGGING);

BEGIN
{
   $DEBUGGING = 1;

   $error_handler = sub
                    {
                      my ($mesg) = @_;
                      print "text/html\n\n";
                      print "<strong>Something nasty happened</strong>";

                      print "Perl said : $message" if $DEBUGGING;
                    }

    set_message($error_handler);
}

I'm not sure when that feature came about :)

Then you get the best of both worlds :)  It has to be borne in mind that
someone installing the application and configuring it is likely to find
themselves debugging their install environment - it would be nice to be
able to tell them to just alter the $DEBUGGING variable to display
suitable diagnostics - not that I'm subscribed to the support list ;-}

> > - I think in the long term it would be best to come up
> > with a slightly different and more configurable strategy
> > that can be used consistently across the programs.
>
> Agreed.
>

With the above we can postpone that awhile although someone might want to
look at making some standard niceness we can use across the whole , er,
suite ;-}

> > I think something that we ought to be looking at in the
> > programs is introducing better ways of doing things - it
> > is not absolutely necessary that we emulate the original
> > programs down to the strange methods of storage (for
> > instance) if they are being deployed in a green field site
> > that has not yet been tainted with Matt's scripts.
>
> But don't forget, that we are supposed to be creating _drop-in_
> replacements for MSA. I see a lot our users being people who
> have Matt's scripts in place who want to move to something "better"
> (for some meaning of the word). It's therefore important that
> we can deal with all of Matt's baroque interface ideas.
>

You started typing that before you finished reading the (albeit overlong)
sentence didn't you :)


> > You will for instance see in 'ffa.pl' the variable
> > $emulate_matts_ffa : although not completely implemented
> > yet when this is switched off the program will store its
> > data in a flat file and generate the HTML page on the fly
> > rather than updating the HTML page directly as it does
> > now.  The guestbook and wwwboard should have this option
> > as well although we do have to retain compatibility with
> > Matt's peculiar storage mechanisms if we are to hope to
> > encourage to upgrade to our programs :)  I thought I had
> > started doing this with the guestbook but I can't find
> > the code anywhere :(
>
> Agreed. Maybe rename the variable to $emulate_matts_code and
> use the same one in all scripts. The default should, of course,
> be on.
>

done.

/J\
-- 
Jonathan Stowe                      |
<http://www.gellyfish.com>          |      This space for rent
                                    |

Re: [Nms-cgi-devel] Welcome

[Dave C. <da...@da...>]

From: Jonathan Stowe <gel...@ge...>
Date: 11/13/01 8:31:22 AM

>On Mon, 12 Nov 2001, Dave Cross wrote:
>
>> Hey look, we have people on the list. Thanks for 
>> joining :)
>>
>> Some things that need doing:
>>
>> 1/ Test, test, test. On as many platforms as possible.
>>
>
> Maybe we ought to get a Tripod or Virtual Avenue account -
> as this is likely to be very much the environment most of 
> the user base will be in - besides we won't mind so much 
> if the site gets r00ted ;-}

Good idea. 

>> 2/ Document, document, document. Currently the scripts 
>> have READMEs that are very closely (in some cases, _too_ 
>> closely) modelled on Matt's. Better docs are important.
>
> We probably want to come up with a boilerplate to do with 
> the general configuration and troubleshooting of CGI 
> programs.  We shouldnt worry too much about not 
> reinventing stuff that can be found elsewhere because it 
> is unlikely that the target audience are going to find 
> the stuff in the first place - give them enough to stop 
> them from becoming guest of honour at a flamewar on 
> comp.lang.perl.misc.

Yep. I'd love it if we had a really simple guide to installing
CGI scripts included with every nms tarball. We should assume
no knowledge at all on the part of the users.

>> 3/ Peer review. Most of the code was written by me in 
>> the course of a ten hour hackfest one weekend. It can no 
>> doubt be improved. I'd love suggestions.
>
> In the first instance I have taken the duplicated code 
> out of some of the files (it looks like you may have 
> done ':r' in vi twice :)

Hmm... unlikely as I'm an emacs user :) Don't where that came
from.

> and added CGI::Carp in those places where the script 
> might die() this will at least produce something 
> moderately less confusing than a plain 500 in the event
> of failure

I don't like this idea. I think that web servers give vanilla
500 errors for a good reason. The more information a user gets
from an error message then the more information they have about
the workings of the script and the more chance they have of cracking
the site.

I'm happy to have CGI::Carp included in the scripts and used
for development and debugging - but I'd like it commented out
of anything checked into CVS.

> - I think in the long term it would be best to come up 
> with a slightly different and more configurable strategy 
> that can be used consistently across the programs.

Agreed.

> I think something that we ought to be looking at in the 
> programs is introducing better ways of doing things - it 
> is not absolutely necessary that we emulate the original 
> programs down to the strange methods of storage (for 
> instance) if they are being deployed in a green field site
> that has not yet been tainted with Matt's scripts.  

But don't forget, that we are supposed to be creating _drop-in_
replacements for MSA. I see a lot our users being people who
have Matt's scripts in place who want to move to something "better"
(for some meaning of the word). It's therefore important that
we can deal with all of Matt's baroque interface ideas.

> You will for instance see in 'ffa.pl' the variable 
> $emulate_matts_ffa : although not completely implemented 
> yet when this is switched off the program will store its 
> data in a flat file and generate the HTML page on the fly 
> rather than updating the HTML page directly as it does 
> now.  The guestbook and wwwboard should have this option 
> as well although we do have to retain compatibility with
> Matt's peculiar storage mechanisms if we are to hope to 
> encourage to upgrade to our programs :)  I thought I had 
> started doing this with the guestbook but I can't find 
> the code anywhere :(

Agreed. Maybe rename the variable to $emulate_matts_code and
use the same one in all scripts. The default should, of course,
be on.

>> 4/ Comments. At last week's london.pm meeting, it was 
>> suggested to me that a higher than usual level of 
>> comments might be a good idea - given the number of 
>> people that learned Perl from MSA.
>
> I've started on that with FFA - elucidating in some 
> detail what the configurable bits mean. 

Great. I'll take a look at those later.

> I would consider it moot whether we should start 
> introducing a quantity of tutorial material into the 
> code - perhaps we could provide an annotated set of 
> source as part of the project documentation (actually 
> this might be a valuable project in its own right).

Damn right. Actually, I'm considering writing it all up into
something that might just become a replacement for Matt Wright's
excreable book :)

[snip stuff about IIS fuckwittage]

Thanks for the detail. That's something that needs to make it
into the docs. Probably a FAQ.

> Oh yeah by the time you read this I will have tinkered 
> with the CVS so that you can check out the whole thing 
> with 'co NMS' - this does mean that you will need to edit 
> CVSROOT/modules if you add a new module - that'll be
> man 5 cvs to you ;-}

Cool. Thanks.

Dave...

-- 
<http://www.dave.org.uk>

"Let me see you make decisions, without your television"
   - Depeche Mode (Stripped)

Re: [Nms-cgi-devel] Welcome

[Jonathan S. <gel...@ge...>]

On Mon, 12 Nov 2001, Dave Cross wrote:

> Hey look, we have people on the list. Thanks for joining :)
>
> Some things that need doing:
>
> 1/ Test, test, test. On as many platforms as possible.
>

Maybe we ought to get a Tripod or Virtual Avenue account - as this is
likely to be very much the environment most of the user base will be in -
besides we won't mind so much if the site gets r00ted ;-}

> 2/ Document, document, document. Currently the scripts have READMEs that
> are very closely (in some cases, _too_ closely) modelled on Matt's. Better
> docs are important.
>

We probably want to come up with a boilerplate to do with the general
configuration and troubleshooting of CGI programs.  We shouldnt worry too
much about not reinventing stuff that can be found elsewhere because it is
unlikely that the target audience are going to find the stuff in the first
place - give them enough to stop them from becoming guest of honour at a
flamewar on comp.lang.perl.misc.

> 3/ Peer review. Most of the code was written by me in the course of a ten
> hour hackfest one weekend. It can no doubt be improved. I'd love suggestions.
>

In the first instance I have taken the duplicated code out of some of the
files (it looks like you may have done ':r' in vi twice :) and added
CGI::Carp in those places where the script might die() this will at least
produce something moderately less confusing than a plain 500 in the event
of failure - I think in the long term it would be best to come up with a
slightly different and more configurable strategy that can be used
consistently across the programs.

I think something that we ought to be looking at in the programs is
introducing better ways of doing things - it is not absolutely necessary
that we emulate the original programs down to the strange methods of
storage (for instance) if they are being deployed in a green field site
that has not yet been tainted with Matt's scripts.  You will for instance
see in 'ffa.pl' the variable $emulate_matts_ffa : although not completely
implemented yet when this is switched off the program will store its data
in a flat file and generate the HTML page on the fly rather than updating
the HTML page directly as it does now.  The guestbook and wwwboard should
have this option as well although we do have to retain compatibility with
Matt's peculiar storage mechanisms if we are to hope to encourage to
upgrade to our programs :)  I thought I had started doing this with the
guestbook but I can't find the code anywhere :(

> 4/ Comments. At last week's london.pm meeting, it was suggested to me that
> a higher than usual level of comments might be a good idea - given the
> number of people that learned Perl from MSA.
>

I've started on that with FFA - elucidating in some detail what the
configurable bits mean. I would consider it moot whether we should start
introducing a quantity of tutorial material into the code - perhaps we
could provide an annotated set of source as part of the project
documentation (actually this might be a valuable project in its own
right).

> Feel free to pick on on something that interests you and get on with it.
>

Er ...

> Something else I'd like advice on from Microsoft experts. Liz Castro has
> found that since she's added -T to the scripts in her book, it's harder to
> run them in IIS. It seems that IIS ignores the shebang line until it's too
> late and then barfs on it. I guess there's a config option somewhere in
> IIS that defines the command line options passed to perl. Does anyone know
> where it is?
>

Yep.  in the script map you have (e.g):

   .pl    ->    perl.exe %s %s

This can be changed to :

   perl.exe -T %s %s

For IIS3 and earlier this has to be done through regedit after that it is
done through the Internet services management thingy.

You might also be able to bodge it up by using pl2bat and then editing the
resulting .bat file.

I'll do a document up about configuring various versions of IIS in the
fullness of time (although this might not help the majority of the target
audience as they will mostly be on Web Hotels our shared web hosting
with an ISP that might not be willing to change this globally [ its not
global on IIS5 actually and I am not sure of 4 but it certainly is with
3]).

In the end I am not sure if we can insist on '-T or death' as these
programs are going to find themselves running in environments where it
just wont be possible to make it work - in the end of the day as long as
they have developed and tested with taint checking on they *are* taint
safe even with it off.  Of course the problems arise when people start
fiddling with the code.  Bah, another document is called for I think ..


What else.

Oh yeah by the time you read this I will have tinkered with the CVS so
that you can check out the whole thing with 'co NMS' - this does mean that
you will need to edit CVSROOT/modules if you add a new module - that'll be
man 5 cvs to you ;-}

/J\
-- 
Jonathan Stowe                      |
<http://www.gellyfish.com>          |      This space for rent
                                    |

[Nms-cgi-devel] Welcome

[Dave C. <da...@da...>]

Hey look, we have people on the list. Thanks for joining :)

Some things that need doing:

1/ Test, test, test. On as many platforms as possible.

2/ Document, document, document. Currently the scripts have READMEs that
are very closely (in some cases, _too_ closely) modelled on Matt's. Better
docs are important.

3/ Peer review. Most of the code was written by me in the course of a ten
hour hackfest one weekend. It can no doubt be improved. I'd love suggestions.

4/ Comments. At last week's london.pm meeting, it was suggested to me that
a higher than usual level of comments might be a good idea - given the 
number of people that learned Perl from MSA.

Feel free to pick on on something that interests you and get on with it.

Something else I'd like advice on from Microsoft experts. Liz Castro has
found that since she's added -T to the scripts in her book, it's harder to
run them in IIS. It seems that IIS ignores the shebang line until it's too
late and then barfs on it. I guess there's a config option somewhere in
IIS that defines the command line options passed to perl. Does anyone know
where it is?

Let me know if you've got any questions.

Dave...

-- 

  "Don't you boys know any _nice_ songs?"