pas-dev — Development mailing list.

Re: [Pas-dev] for better or worse, XSS filtering is here

[Kyle R . B. <mo...@vo...>]

> Also, what about the possibility of malicious code in the query parameter 
> names themselves?

Ok, I'm about to check in a changed version of the function.  The changes
include support for defanging the query parameter names, and it accounts
for multiple values for a query parameter.  It also uses the logging to 
log a warning that includes the remote addr (not completely reliable, but
better than nothing).


Kyle


==>

my $scriptingRegex =
  qr/(<\s*script[\s+?.*?>|>].*?\<\s*?\/script\s*?>)| 
     (<\s*embed[\s+?.*?>|>].*?\<\s*?\/embed\s*?>)|
     (<\s*applet[\s+?.*?>|>].*?\<\s*?\/applet\s*?>)|
     (<\s*object[\s+?.*?>|>].*?\<\s*?\/object\s*?>)/imx;

sub inspect_request
{
  my ($q)  = @_;

  my @badParameters = ();
  foreach my $p ($q->param()){
    if ($p =~ $scriptingRegex ) {
      $log->warn("possible XSS attack found while scrubbing ",
        "parameter names: '$p' REMOTE_ADDR: ",$ENV{'REMOTE_ADDR'}
      );
      push @badParameters, $p;
    }
  }

  foreach my $badParam (@badParameters) {
    my @values = $q->param($badParam);
    $q->delete($badParam);
    my $p = HTML::Entities::encode_entities($badParam);
    $q->param($p,@values);
  }

  foreach my $p ($q->param()){
    my @x = $q->param($p);
    foreach my $x ( @x ) {
      if ($x =~ $scriptingRegex ) {
        $log->warn("possible XSS attack found while scrubbing ",
          "parameter values: '$p' => '$x' REMOTE_ADDR: ",$ENV{'REMOTE_ADDR'}
        );
        $x = HTML::Entities::encode_entities($x);
        #$q->param($p,$x);
      }
    }
    $q->param($p,@x);
  }

  return $q;
}


-- 

------------------------------------------------------------------------------
Wisdom and Compassion are inseparable.
        -- Christmas Humphreys
mo...@vo...                            http://www.voicenet.com/~mortis
------------------------------------------------------------------------------

Re: [Pas-dev] for better or worse, XSS filtering is here

[Kyle R . B. <mo...@vo...>]

> > Hopefully it'll be a welcome addition to our features. 
> 
> Does it account for the possibility of multiple values for a single query 
> parameter?

Also, what about the possibility of malicious code in the query parameter 
names themselves?


Kyle

-- 

------------------------------------------------------------------------------
Wisdom and Compassion are inseparable.
        -- Christmas Humphreys
mo...@vo...                            http://www.voicenet.com/~mortis
------------------------------------------------------------------------------

Re: [Pas-dev] for better or worse, XSS filtering is here

[Kyle R . B. <mo...@vo...>]

> Hopefully it'll be a welcome addition to our features. 

Does it account for the possibility of multiple values for a single query 
parameter?

Kyle

-- 

------------------------------------------------------------------------------
Wisdom and Compassion are inseparable.
        -- Christmas Humphreys
mo...@vo...                            http://www.voicenet.com/~mortis
------------------------------------------------------------------------------

[Pas-dev] for better or worse, XSS filtering is here

[Mental <Me...@Ne...>]

Hopefully it'll be a welcome addition to our features. 
 


--
Mental (Me...@Ne...)

I took a baby shower.
--Steven Wright

GPG public key: http://www.neverlight.com/Mental.asc

Re: [Pas-dev] basic unit testing....

[Mental <me...@ne...>]

On Wed, 2002-05-22 at 15:36, Kyle R . Burton wrote:
> > Hey. I'm trying to write a unit test for this request inquisitor. I
> > think I have all the third party modules I need installed. This works on
> > hades, but the test keeps blowing up. I know the code is right. 
> > 
> > Working on desynchronized codebases is a complete pain in the ass.
> > Especially since pas seems to have changed quite a bit since I started
> > using it. 
> > 
> > I'm trying to think of ways to use this and still be able to do my own
> > shit.
> > 
> > I could check out pas and nuke pages/ and htdocs/. I could make those 2
> > dirs a link to my own repository of content and page objects.
> > 
> > Any ideas? 
> 
> There are no incompatible chagnes in the current version of CVS that I am 
> aware of.  Why would you have to nuke your pages/ and htdocs/ directories?
> 

I was being bitten by the makeGetSet stuff. I spaced on it and didnt
catch it til I tried the word-salad method of debuging.


> Some of the code has changed, but it's all internals, none of the external
> api stuff has changed, so I don't understand what the issue is.  Unless
> I'm missing or forgetting something fundamental.
> 
> Why can't you just use what's in CVS for your development work?
> 

Thats what I wanna do. You misunderstand me. I dont propose to remove MY
stuff, I plan on nuking the sample crap that comes with pas. And
actually, I dont even need to do that. I just need to setup my config
file to use a different directory for htdocs and pages. So this is a non
issue. 

What I'd been doing (and I'd like to avoid) is have everyhthing in my
own cvs repository. So in effect I was working on 2 codebases. Pas and
Pas from a few months ago + my minor stuff. Since I'm mainly doing psp
and page object work moreso than work on pas proper, I was falling
behind. 


> What is the stuff that's changed since you started with Pas?
> 

All the 3rd party modules. I had to build debs. If anyone is interested
I'll probably setup a deb repository where you can get some of hte
unbundled modules for perl. No sense cluttering up your box with a bunch
of cruft if you can avoid it. So far I've created these:
libappconfig-perl_1.52-1_all.deb
liblog-dispatch-config-perl_1.01-1_all.deb
liblog-dispatch-perl_2.00-1_all.deb

I'll put together a README.Debian file later. 
I'll have an apt source you can add to your sources.list file. I may
very well create a debian package for these modules and pas. I know
enough maintainers that getting my gpg key signed shouldn't be
difficult.... anyways.

I'm going to add a TIPS file for generic I-wish-i'd-done-this-before
stuff. 



-- 
Mental (Me...@Ne...)

Re: [Pas-dev] basic unit testing....

[Kyle R . B. <mo...@vo...>]

> Hey. I'm trying to write a unit test for this request inquisitor. I
> think I have all the third party modules I need installed. This works on
> hades, but the test keeps blowing up. I know the code is right. 
> 
> Working on desynchronized codebases is a complete pain in the ass.
> Especially since pas seems to have changed quite a bit since I started
> using it. 
> 
> I'm trying to think of ways to use this and still be able to do my own
> shit.
> 
> I could check out pas and nuke pages/ and htdocs/. I could make those 2
> dirs a link to my own repository of content and page objects.
> 
> Any ideas? 

There are no incompatible chagnes in the current version of CVS that I am 
aware of.  Why would you have to nuke your pages/ and htdocs/ directories?

Some of the code has changed, but it's all internals, none of the external
api stuff has changed, so I don't understand what the issue is.  Unless
I'm missing or forgetting something fundamental.

Why can't you just use what's in CVS for your development work?

What is the stuff that's changed since you started with Pas?


Kyle

-- 

------------------------------------------------------------------------------
Wisdom and Compassion are inseparable.
        -- Christmas Humphreys
mo...@vo...                            http://www.voicenet.com/~mortis
------------------------------------------------------------------------------

Re: [Pas-dev] perldoc perlre

[Justin B. <ju...@le...>]

Kyle R . Burton wrote:
>>Hey, I suck at this stuff. I've condensed this down from 4 nested ifs to
>>one. I'd like to go further, but
>>1. I'm not sure how
>>2. I'm not sure anybody'd be able to read it. 
>>
>>    # find the script, embed, applet and object tags anywhere in a 
>>    # query parameter and defang them. No original content is lost,
>>    # so you can always decode_entities to restore the data.
>>    if ($x =~ /(<\s*script[\s+?.*?>|>].*?\<\s*?\/script\s*?>)| 
>>               (<\s*embed[\s+?.*?>|>].*?\<\s*?\/embed\s*?>)|
>>               (<\s*applet[\s+?.*?>|>].*?\<\s*?\/applet\s*?>)|
>>               (<\s*object[\s+?.*?>|>].*?\<\s*?\/object\s*?>)/imx ){
>>      $x = HTML::Entities::encode_entities($x);
>>      $q->param($p,$x);
>>    }
>>  }
> 
> 
> How about something like:
> 
>   foreach my $paramName ( $q->param() ) {
>     my @values = ();
>     foreach my $value ( $q->param($paramName) ) {
>       foreach my $badTag ( qw( script embed applet object ) ) {
>         $value =~ s|<\s*($badTag\b.[^>]+)>(.*?)<(\s*?/$badTag\s*)>|&lt;$1&gt;$2&lt;$3&gt;|igms;
>       }
>       push @values, $value;
>     }
>     $q->param($paramName,@values);
>   }
> 
> Is that any more readable?  That only hits the tags we're interested in
> defanging...
> 
> Don't forget that you can have mulitple values for a CGI parameter.

how about this: make badTag configurable inside pas.conf.  whatever the variable 
will be named, returns an array.  so if someone wants it for all HTML tags, they 
just set the variable to '.*'.  have that example & the 'script embed applet 
object' example.

justin

Re: [Pas-dev] perldoc perlre

[Kyle R . B. <mo...@vo...>]

> Hey, I suck at this stuff. I've condensed this down from 4 nested ifs to
> one. I'd like to go further, but
> 1. I'm not sure how
> 2. I'm not sure anybody'd be able to read it. 
> 
>     # find the script, embed, applet and object tags anywhere in a 
>     # query parameter and defang them. No original content is lost,
>     # so you can always decode_entities to restore the data.
>     if ($x =~ /(<\s*script[\s+?.*?>|>].*?\<\s*?\/script\s*?>)| 
>                (<\s*embed[\s+?.*?>|>].*?\<\s*?\/embed\s*?>)|
>                (<\s*applet[\s+?.*?>|>].*?\<\s*?\/applet\s*?>)|
>                (<\s*object[\s+?.*?>|>].*?\<\s*?\/object\s*?>)/imx ){
>       $x = HTML::Entities::encode_entities($x);
>       $q->param($p,$x);
>     }
>   }

How about something like:

  foreach my $paramName ( $q->param() ) {
    my @values = ();
    foreach my $value ( $q->param($paramName) ) {
      foreach my $badTag ( qw( script embed applet object ) ) {
        $value =~ s|<\s*($badTag\b.[^>]+)>(.*?)<(\s*?/$badTag\s*)>|&lt;$1&gt;$2&lt;$3&gt;|igms;
      }
      push @values, $value;
    }
    $q->param($paramName,@values);
  }

Is that any more readable?  That only hits the tags we're interested in
defanging...

Don't forget that you can have mulitple values for a CGI parameter.

-- 

------------------------------------------------------------------------------
Wisdom and Compassion are inseparable.
        -- Christmas Humphreys
mo...@vo...                            http://www.voicenet.com/~mortis
------------------------------------------------------------------------------

Re: [Pas-dev] perldoc perlre

[Kyle R . B. <mo...@vo...>]

> >   foreach ... {
> >     $param =~ s/<\s*SCRIPT\b/&lt;SCRIPT/ge;
> >     $param =~ s/<\s*CODE\b/&lt;CODE/ge;
> >     $param =~ s/<\s*APPLET\b/&lt;APPLET/ge;
> >   }
> > 
> > and just forget the trailing '>'?  Or is that necessary?  How to escape
> > it without potentialy manipulating data we're not supposed to is a 
> > delicate issue.
> 
> not getting the trailing '>' my lead to a problem with the display of the page. 
>   also don't forget to get the '</SCRIPT>', etc. out of the code.
> 
> hmm.. not translating all '>' or '<' can lead to some interesting pages.  i 
> could start posting in '</td>' or '</input>' or '</form>'.  they're not viscious 
>   but, going back to the discussion board example, you could feasibly break a 
> thread in a discussion board by passing some simple HTML.

Yes, that's right.  For discussion boards, we could have a re-encode function
that re-enabled alot of the 'safe' stuff.  To go backwards for 'safe' tags,
we could just have:

  foreach ... {
    foreach my $tag ( qw( a img p br pre code font ) ) {
      $param =~ s#&lt;($tag.+?)&gt;#<$1>#gi;
      $param =~ s#&lt;(/$tag.+?)&gt;#<$1>#gi;
    }
    ...
  }

Gak, you always hear people say that you can't swat HTML with regexes...and
they say that for good reason.  The proper appraoch is to use a fully blown
HTML parser...but that's really overly heavy-weight for this application...

Discussion boards might want to turn off the blanket scrubber, and use
their own, _or_ un-scrub the stuff they want to allow...I don't know...if
we write one that scrubs appropriatly for the discussion board example,
maybe we should just make that the default...




k

-- 

------------------------------------------------------------------------------
Wisdom and Compassion are inseparable.
        -- Christmas Humphreys
mo...@vo...                            http://www.voicenet.com/~mortis
------------------------------------------------------------------------------

Re: [Pas-dev] perldoc perlre

[Justin B. <ju...@le...>]

>>Ultimately what I wanted to do was make this so it was something you
>>could subclass and add your own rules to. If you're running a kids site,
>>you could subclass it and immediately clean up (censor) any submitted
>>data with the 7 dirty words.
>>
>>I was going to add a parameter to pas.conf that contained the name of
>>the package to use to run the data through eventually. I just need to
>>get around the tricky bit of needing to use that package in the
>>Request.pm :/
>>
>>Does this make sense?
> 
> 
> Yes, it does.  Could we just change it to:
> 
>   foreach ... {
>     $param =~ s/<\s*SCRIPT\b/&lt;SCRIPT/ge;
>     $param =~ s/<\s*CODE\b/&lt;CODE/ge;
>     $param =~ s/<\s*APPLET\b/&lt;APPLET/ge;
>   }
> 
> and just forget the trailing '>'?  Or is that necessary?  How to escape
> it without potentialy manipulating data we're not supposed to is a 
> delicate issue.

not getting the trailing '>' my lead to a problem with the display of the page. 
  also don't forget to get the '</SCRIPT>', etc. out of the code.

hmm.. not translating all '>' or '<' can lead to some interesting pages.  i 
could start posting in '</td>' or '</input>' or '</form>'.  they're not viscious 
  but, going back to the discussion board example, you could feasibly break a 
thread in a discussion board by passing some simple HTML.

justin

Re: [Pas-dev] perldoc perlre

[Kyle R . B. <mo...@vo...>]

> > A few quick questions, is HTML::Entities a core module or something
> > you have to get from CPAN?  If it's not core, please add it to the
> > INSTALL instructions.

> Package: libwww-perl
> 
> I'll add libwww and lwp user agent to the INSTALL.
> Or is lwp stuff part of libwww? I forget.

I think the lwp stuff is part of libwww.  Just adding that should be
fine.  Thanks.

> I was trying to make this as non-invasive as possible. What if you're on
> a discussion board and want to embed a url in your message? Or an IMG
> SRC tag? Neither of those are malicious in and of themselves. 

Good point.

> Ultimately what I wanted to do was make this so it was something you
> could subclass and add your own rules to. If you're running a kids site,
> you could subclass it and immediately clean up (censor) any submitted
> data with the 7 dirty words.
> 
> I was going to add a parameter to pas.conf that contained the name of
> the package to use to run the data through eventually. I just need to
> get around the tricky bit of needing to use that package in the
> Request.pm :/
> 
> Does this make sense?

Yes, it does.  Could we just change it to:

  foreach ... {
    $param =~ s/<\s*SCRIPT\b/&lt;SCRIPT/ge;
    $param =~ s/<\s*CODE\b/&lt;CODE/ge;
    $param =~ s/<\s*APPLET\b/&lt;APPLET/ge;
  }

and just forget the trailing '>'?  Or is that necessary?  How to escape
it without potentialy manipulating data we're not supposed to is a 
delicate issue.

Kyle

-- 

------------------------------------------------------------------------------
Wisdom and Compassion are inseparable.
        -- Christmas Humphreys
mo...@vo...                            http://www.voicenet.com/~mortis
------------------------------------------------------------------------------

Re: [Pas-dev] perldoc perlre

[Mental <me...@ne...>]

On Mon, 2002-05-20 at 13:08, Kyle R . Burton wrote:

> A few quick questions, is HTML::Entities a core module or something
> you have to get from CPAN?  If it's not core, please add it to the
> INSTALL instructions.
> 

Package: libwww-perl
Priority: optional
Section: interpreters
Installed-Size: 884
Maintainer: Michael Alan Dorman <md...@de...>
Architecture: all
Version: 5.64-1
Depends: perl (>= 5.6.0-16), libnet-perl (>= 1:1.09),
libdigest-md5-perl, libmime-base64-perl (>= 2.1), liburi-perl (>= 1.10),
libhtml-parser-perl (>= 2.20), libhtml-tree-perl (>= 3.11)
Recommends: libdigest-md5-perl, libmailtools-perl, libhtml-format-perl
Suggests: libcrypt-ssleay-perl | libio-socket-ssl-perl
Filename: pool/main/libw/libwww-perl/libwww-perl_5.64-1_all.deb
Size: 293062
MD5sum: 6a3f908c2fd7654201319d2d4b83e028
Description: WWW client/server library for Perl
 Libwww-perl is a collection of Perl modules which provides a simple
 and consistent programming interface (API) to the World-Wide Web.  The
 main focus of the library is to provide classes and functions that
 allow you to write WWW clients, thus libwww-perl said to be a WWW
 client library. The library also contain modules that are of more
 general use, as well as a simple HTTP/1.1-compatible server
 implementation.
 .
 The URI modules have been split off into liburi-perl; install that
 package if you need them.


I'll add libwww and lwp user agent to the INSTALL.
Or is lwp stuff part of libwww? I forget.


> Also, do we need to transliterate all of the possible html entities?
> Can we just do '<', and '>' and get the desired effect?
> 
> Can we just encode the dangerous entites regardless of what else may be in
> the string (i.e. don't bother looking for SCRIPT/CODE/etc., just do the
> scrubbing)?
> 

I was trying to make this as non-invasive as possible. What if you're on
a discussion board and want to embed a url in your message? Or an IMG
SRC tag? Neither of those are malicious in and of themselves. 


Ultimately what I wanted to do was make this so it was something you
could subclass and add your own rules to. If you're running a kids site,
you could subclass it and immediately clean up (censor) any submitted
data with the 7 dirty words.

I was going to add a parameter to pas.conf that contained the name of
the package to use to run the data through eventually. I just need to
get around the tricky bit of needing to use that package in the
Request.pm :/

Does this make sense?

-- 
Mental (Me...@Ne...)

Re: [Pas-dev] perldoc perlre

[Kyle R . B. <mo...@vo...>]

> Hey, I suck at this stuff. I've condensed this down from 4 nested ifs to
> one. I'd like to go further, but
> 1. I'm not sure how
> 2. I'm not sure anybody'd be able to read it. 
> 
>     # find the script, embed, applet and object tags anywhere in a 
>     # query parameter and defang them. No original content is lost,
>     # so you can always decode_entities to restore the data.
>     if ($x =~ /(<\s*script[\s+?.*?>|>].*?\<\s*?\/script\s*?>)| 
>                (<\s*embed[\s+?.*?>|>].*?\<\s*?\/embed\s*?>)|
>                (<\s*applet[\s+?.*?>|>].*?\<\s*?\/applet\s*?>)|
>                (<\s*object[\s+?.*?>|>].*?\<\s*?\/object\s*?>)/imx ){
>       $x = HTML::Entities::encode_entities($x);
>       $q->param($p,$x);
>     }
>   }

A few quick questions, is HTML::Entities a core module or something
you have to get from CPAN?  If it's not core, please add it to the
INSTALL instructions.

Also, do we need to transliterate all of the possible html entities?
Can we just do '<', and '>' and get the desired effect?

Can we just encode the dangerous entites regardless of what else may be in
the string (i.e. don't bother looking for SCRIPT/CODE/etc., just do the
scrubbing)?

Perhaps something like: (warning, this is pseudo code)

  sub scrubParams
  {
    my($self) = @_;
    my %map = ( '<' => '&lt;', '>' => '&gt;' );
    foreach my $paramName ( $self->query()->param() ) {
      my @newValues = ();
      foreach my $param ( $self->query()->param($paramName) ) {
        foreach my $entity ( keys %map; ) {
          $param =~ s/$entity/ $map{$entity} /ge;
          push @newValues, $param;
        }
      }
      $self->query()->param($paramName,@newValues);
    }
    return 1;
  }

That's a nice standard thing to do -- then if people actualy _want_ them
unscrubbed, they can reverse the process...we are going on the assumption
that angle brackets (gt/lt) won't be in query parameters very often.  If
they are, developers have to handle it explicitly -- kind of like tainted
variables.  If you really want to have it that way, the onus is on you
to untaint the data as you see fit.

To reverse, something like:

  sub unScrubParams
  {
    my($self) = @_;
    my %map = ( '&lt;' => '<', '&gt;' => '>' );
    foreach my $paramName ( $self->query()->param() ) {
      my @newValues = ();
      foreach my $param ( $self->query()->param($paramName) ) {
        foreach my $entity ( keys %map; ) {
          $param =~ s/$entity/ $map{$entity} /ge;
          push @newValues, $param;
        }
      }
      $self->query()->param($paramName,@newValues);
    }
    return 1;
  }

Does that fit with what you're trying to implement?

Kyle

-- 

------------------------------------------------------------------------------
Wisdom and Compassion are inseparable.
        -- Christmas Humphreys
mo...@vo...                            http://www.voicenet.com/~mortis
------------------------------------------------------------------------------

[Pas-dev] perldoc perlre

[Mental <me...@ne...>]

Hey, I suck at this stuff. I've condensed this down from 4 nested ifs to
one. I'd like to go further, but
1. I'm not sure how
2. I'm not sure anybody'd be able to read it. 

    # find the script, embed, applet and object tags anywhere in a 
    # query parameter and defang them. No original content is lost,
    # so you can always decode_entities to restore the data.
    if ($x =~ /(<\s*script[\s+?.*?>|>].*?\<\s*?\/script\s*?>)| 
               (<\s*embed[\s+?.*?>|>].*?\<\s*?\/embed\s*?>)|
               (<\s*applet[\s+?.*?>|>].*?\<\s*?\/applet\s*?>)|
               (<\s*object[\s+?.*?>|>].*?\<\s*?\/object\s*?>)/imx ){
      $x = HTML::Entities::encode_entities($x);
      $q->param($p,$x);
    }
  }



-- 
Mental (Me...@Ne...)

Re: [Pas-dev] proposal

[Kyle R . B. <mo...@vo...>]

> Get the pas.conf OUT of cvs!

Good point.  Ever since Justin added the ability to set PAS_CONFIG, I
should have stopped using the pas.conf that's in CVS.  Sorry about that.

> Rename it to pas.conf.sample. It gets in the way when upgrading between
> releases. The whole world will not want their conf file in cvs :) Just
> give a sample that can be built on. That way it'll be a little easier to
> import this stuff into your other projects that use it.
> 
> comments?

I completely agree.  We should set up the conf/pas.conf as a default,
that is capable of running the example pages that we ship with Pas out
of the box.

So, what kinds of examples should we have in the pa.conf?


Thanks,
Kyle

-- 

------------------------------------------------------------------------------
Wisdom and Compassion are inseparable.
        -- Christmas Humphreys
mo...@vo...                            http://www.voicenet.com/~mortis
------------------------------------------------------------------------------

[Pas-dev] proposal

[Mental <me...@ne...>]

Get the pas.conf OUT of cvs!

Rename it to pas.conf.sample. It gets in the way when upgrading between
releases. The whole world will not want their conf file in cvs :) Just
give a sample that can be built on. That way it'll be a little easier to
import this stuff into your other projects that use it.

comments?
-- 
Mental (Me...@Ne...)

[Pas-dev] migration to Log::Dispatch

[Kyle R . B. <mo...@vo...>]

Should be completed.  I gutted the old logging code out of the Base
object hierarchy and replaced all the calls to log(), msg() log_error()
and log_warning() with a log() method on Base that returns a 
Log::Dispatch::Config object.  An example config file is in conf/log.conf.

It's extremely flexible.  Though now that I've used it, I don't think it's
nearly as flexible as the real Log4j -- I don't know how to change settings
for particular packages like you can with Log4j.


The unit tests and the example pages all seem to work.



I need your opinions: should we migrate the namespace to just 'Pas'?  It
kind of grates on me that I have to keep typing 'Org::Bgw::' all over the
place.  Not that big a deal, and if you all say it's not worth the hastle,
we'll stick with the old prefix.  Personaly I'd like to change it.


Should we try migrating to Graham Barr's Error module next?


Kyle

-- 

------------------------------------------------------------------------------
Wisdom and Compassion are inseparable.
        -- Christmas Humphreys
mo...@vo...                            http://www.voicenet.com/~mortis
------------------------------------------------------------------------------

[Pas-dev] new changes...

[Kyle R . B. <mo...@vo...>]

vX.X.XX Sun May 19 18:13:24 EDT 2002
  * krb: added in basic unit tests for every package in the project
  * krb: revamped the db object generation to use PSPs instead, which
    is much simpler.  See the files bin/dbObjectGenerator,
    test/objectTemplate.psp, and test/testDbGen.sh for examples.

Kyle

-- 

------------------------------------------------------------------------------
Wisdom and Compassion are inseparable.
        -- Christmas Humphreys
mo...@vo...                            http://www.voicenet.com/~mortis
------------------------------------------------------------------------------

[Pas-dev] unit tests

[Kyle R . B. <mo...@vo...>]

I went through the codebase and added a simple unit test (only one test
method to test compilation) for every calss (except one of the ones under 
Org::Bgw::HTML, which relies on an external module).

As I can, I'll add more test methods to each of the unit tests to flesh
things out.

Kyle

-- 

------------------------------------------------------------------------------
Wisdom and Compassion are inseparable.
        -- Christmas Humphreys
mo...@vo...                            http://www.voicenet.com/~mortis
------------------------------------------------------------------------------

Re: [Pas-dev] more on generic DB session obj

[Kyle R . B. <mo...@vo...>]

> Neuron death. I stripped it out. I'll just encode the <'s to be &lt;
> &gt;. Then you'll see the whole script.

Cool.  That sets a precedent that the transformation is potentialy reversable.
If users want to have the ligatures translated back into their specal 
charactes, they can by passing it through a couple of regexes:  

  $value =~ s/&lt;/</g;
  $value =~ s/&gt;/>/g;

> My thought was, removing the verb from the script would defang it.
> Seeing ugly goo on the screen is much better than having all your base
> taken. 

Agreed.  I completely agree.

> Semantics. Does the request handler not 'handle' passing the raw query
> object into the profanity filter? Request.pm is fairly straight up
> collection of get/sets. My reasoning was to keep the application logic
> (config setting lookups/branching) encapsulated in the requesthandler.
> If you want me to move it, I will. 

All the request handler is supposed to do, IMO, is:

- figure out what page object to use to service the inbound request
- manage the object's life cycle to service the request (construct, 
  set_up, execute, tear_down)
- catch any exceptions that might crop up and re-forward the error to
  an error page.

It should not try to force convention on the operation of the page objects.
For instance, it cares and knows not if a page has or wants session support.
It only deals with the most raw of concepts relating to the HTTP request.

Our request object is there to wrap the CGI.pm object so we can add additional
behavior - just like what we're talking about using.  We could set up a config
option in the pas.conf for the request object type -- then we could derive
an Org::Bgw::Pas::Request:ScriptKiddySafe object that processes the CGI data 
to make it safe.  There are also issues with non GET/POST http requests like
PUT, for which there is no parameter data.  POSTs also do not have to have the
xxx-form-url-encoded mime type either.  Though I'm not sure if thi sis an 
argument for or against doing the transform to all the data.  It'd be nice
to do the transform _before_ CGI.pm has a chance to parse it, but that's
probably not worth doing.


Kyle


-- 

------------------------------------------------------------------------------
Wisdom and Compassion are inseparable.
        -- Christmas Humphreys
mo...@vo...                            http://www.voicenet.com/~mortis
------------------------------------------------------------------------------

Re: [Pas-dev] proposal: drop the Org::Bgw prefix in favor of just Pas

[Kyle R . B. <mo...@vo...>]

> Well, yes. Assuming nobody is building anything thats not in your cvs
> repository, that works fine. I may have issues with other stuff I've
> done that isnt in sf's cvs archive. But, thats why I have a separate
> tree for my project.
> 
> Still, its probably better to do it now if at all. 


+1


-- 

------------------------------------------------------------------------------
Wisdom and Compassion are inseparable.
        -- Christmas Humphreys
mo...@vo...                            http://www.voicenet.com/~mortis
------------------------------------------------------------------------------

Re: [Pas-dev] more on generic DB session obj

[Kyle R . B. <mo...@vo...>]

> talk talk talk. you always wanna carry on about something. :) 

Yes, I don't always trust myself.

> I think we should all get together again and hash some stuff out.
> There's been several good ideas today. How bout monday? I'll get a case
> of delerium and we'll figure stuff out. Evidently offering good belgian
> ale makes everyone clear their day planers and actually show up :)
> Tomorrow I'm going to the Scottish Games with the family, sunday I have
> cleared for movie stuff.
> 
> That all said, dig this crap:
> http://www.neverlight.com/pas/css.psp?foo=%3Cscript%3Ealert('oops')%3C/script%3E
> 
> in one round of botched hackery, none of my pages are vulnerable to CSS
> attacks that use the <script tag. :)

Neato.  That's pretty awesome.  Why is the starting part of the tag
disappearing?  Are you just stripping it or are you translating it into
an html entity?  (&lt; and &gt;)

> Happy day. Now to make it not look like hell. Once I'm happy with
> implementation details, I'll merge it into the main pas tree.
> 
> Right now, I'm doing everything in RequestHandler. We can break it out
> easily as I wrote my own inspect-the-request-and-break-it methods.  I'm
> just not sure what the best way to go is.

How about in the request object?  That seems like the right place for it.
When the request handler sets the query in the request, it could do
the key/val arg checking at that point.  It seems more appropraite in
the request object instead of the handler.  The handler should just set
things up and let them go.

> Should I make the config file the place where you specify what package
> RequestHandler should use to purify data? If I do that, should we
> support stacked (specialized) handlers? I dunno. Maybe I'm over
> engineering. I just wish I was more devious. I'm pretty much just
> wandering around security focus for no-brainer scripts to throw at the
> code. So I'm not at all sure how bullet proof this will be.
> 
> Oh, and I'm not at all sure I know all the tricks you can do to sneak
> the tags past.
> 
> Like script... it has to look like
> <script>
> or < script> or whatever... right? Like, there cant be anything between
> the word script and the closing bracket, right? so like.. 
> <script name> isnt the same thing. Sorry if this is dumb, but I think
> I'm down to one brain cell today.
> 
> Anybody wanna help a brother out?

At least put in the first implementation -- we'll reworkit if necessary.
Some protection is better than zero.


Thanks,
Kyle

-- 

------------------------------------------------------------------------------
Wisdom and Compassion are inseparable.
        -- Christmas Humphreys
mo...@vo...                            http://www.voicenet.com/~mortis
------------------------------------------------------------------------------

Re: [Pas-dev] more on generic DB session obj

[Mental <me...@ne...>]

On Fri, 2002-05-17 at 15:48, Kyle R . Burton wrote:

> > Any questions?  Does this sound good?  Can I take a chain saw to it?  I'll at 
> > least get this working for MySQL in the near future.  Other implementations 
> > shouldn't be hard.
> 
> It sounds good so far, I just want to talk about it some more until I feel
> comfortable about it.

talk talk talk. you always wanna carry on about something. :) 

I think we should all get together again and hash some stuff out.
There's been several good ideas today. How bout monday? I'll get a case
of delerium and we'll figure stuff out. Evidently offering good belgian
ale makes everyone clear their day planers and actually show up :)
Tomorrow I'm going to the Scottish Games with the family, sunday I have
cleared for movie stuff.

That all said, dig this crap:
http://www.neverlight.com/pas/css.psp?foo=%3Cscript%3Ealert('oops')%3C/script%3E

in one round of botched hackery, none of my pages are vulnerable to CSS
attacks that use the <script tag. :)

Happy day. Now to make it not look like hell. Once I'm happy with
implementation details, I'll merge it into the main pas tree.

Right now, I'm doing everything in RequestHandler. We can break it out
easily as I wrote my own inspect-the-request-and-break-it methods.  I'm
just not sure what the best way to go is.

Should I make the config file the place where you specify what package
RequestHandler should use to purify data? If I do that, should we
support stacked (specialized) handlers? I dunno. Maybe I'm over
engineering. I just wish I was more devious. I'm pretty much just
wandering around security focus for no-brainer scripts to throw at the
code. So I'm not at all sure how bullet proof this will be.

Oh, and I'm not at all sure I know all the tricks you can do to sneak
the tags past.

Like script... it has to look like
<script>
or < script> or whatever... right? Like, there cant be anything between
the word script and the closing bracket, right? so like.. 
<script name> isnt the same thing. Sorry if this is dumb, but I think
I'm down to one brain cell today.

Anybody wanna help a brother out?


-- 
Mental (Me...@Ne...)

Re: [Pas-dev] more on generic DB session obj

[Justin B. <ju...@le...>]

Kyle R . Burton wrote:
>>Poking around the directory some more:
>>
>>- Org::Bgw::Pas::Session::MySQL becomes Org::Bgw::Pas::Session::DB.  _factory() 
>>method gets re-written to look up the driver from the config file and 
>>instantiate the appropriate DB factory driver for the session.
>>
>>- Then we create a new Org::Bgw::Pas::Session::Factory.  Stuff that can be moved 
>>from Org::Bgw::Pas::Session::MySQL::Factory will be moved into this new class.
>>
>>- The MySQL directory gets moved to a new name Factory.
>>
>>- Org::Bgw::Pas::Session::MySQL::Factory becomes 
>>Org::Bgw::Pas::Session::Factory::MySQL.  All generic stuff is striped so it's 
>>MySQL specific and it's parent becomes Org::Bgw::Pas::Session::Factory.
> 
> I like the idea of using a factory...but I don't want it to do a config lookup
> every time it creates a session object...perhaps we can cache it somewhere?
> Or am I just being paranoid?  Are the config lookups just as fast?  In fact,
> that might be what it's doing now...

Well, that's what it looks like it's doing for username, password, etc.

> I'm not yet set on what I think the cleanest/best design should be.  This 
> looks good at first glance.  I'm not necessarily convinced that what I wrote
> for the MySQL/Factory objets was a good appraoch to start with :)

Well, it might not have been the best approach to start with (I'm not completely 
sure without reviewing the other objects), but what has been proposed is easy to 
implement, won't really hurt anything and will allow plugins for different 
databases for the session.  If we want to restructure stuff at a later date, we 
won't be wasting much time with this improvement now.

>>- In the conf file we should add the driver to make it easy for the person to 
>>tell the app what DB they want to run for the session.  So you'll now have up to 
>>7 session variables for the config file:
>>
>>   pas.session.dbname
>>   pas.session.driver
>>   pas.session.dsn
>>   pas.session.hostname
>>   pas.session.password
>>   pas.session.table
>>   pas.session.username
> 
> 
> Maybe prefix these with pas.session.db.* to follow the convention?  We
> currently have pas.session.file.*, pas.session.apache.*, and 
> pas.session.mysql.*, so perhaps it should be pas.session.db.*?

Sure.  Sounds good.  I missed the other prefixes.

>>Of course, if you specify the dsn, you can skip the dbname & hostname.  The 
>>driver parameter will be a requirement!
> 
> The reason dsn is in there as a parameter is because it's actualy hard to
> dynamicly build a dsn depending on the driver and the options the user
> wants to pass. Maybe we should just forget the dbname, and hostname and
> expect the user to specify them in the dsn?

I understand the reason.  If we want to get rid of hostname & dbname, that 
simplifies things!

>>Any questions?  Does this sound good?  Can I take a chain saw to it?  I'll at 
>>least get this working for MySQL in the near future.  Other implementations 
>>shouldn't be hard.
> 
> It sounds good so far, I just want to talk about it some more until I feel
> comfortable about it.

It works well with the structure that's there.  To make the changes and have it 
up and running for MySQL will take less time then these conversations.  Then 
going forward for Oracle & PostgresSQL won't take much time either (we should 
just have to overriding 2 methods, insert & update).  Most of that time will be 
researching on how to deal with blobs.

Justin

Re: [Pas-dev] more on generic DB session obj

[Kyle R . B. <mo...@vo...>]

> Poking around the directory some more:
> 
> - Org::Bgw::Pas::Session::MySQL becomes Org::Bgw::Pas::Session::DB.  _factory() 
> method gets re-written to look up the driver from the config file and 
> instantiate the appropriate DB factory driver for the session.
>
> - Then we create a new Org::Bgw::Pas::Session::Factory.  Stuff that can be moved 
> from Org::Bgw::Pas::Session::MySQL::Factory will be moved into this new class.
> 
> - The MySQL directory gets moved to a new name Factory.
> 
> - Org::Bgw::Pas::Session::MySQL::Factory becomes 
> Org::Bgw::Pas::Session::Factory::MySQL.  All generic stuff is striped so it's 
> MySQL specific and it's parent becomes Org::Bgw::Pas::Session::Factory.

I like the idea of using a factory...but I don't want it to do a config lookup
every time it creates a session object...perhaps we can cache it somewhere?
Or am I just being paranoid?  Are the config lookups just as fast?  In fact,
that might be what it's doing now...

I'm not yet set on what I think the cleanest/best design should be.  This 
looks good at first glance.  I'm not necessarily convinced that what I wrote
for the MySQL/Factory objets was a good appraoch to start with :)

> - In the conf file we should add the driver to make it easy for the person to 
> tell the app what DB they want to run for the session.  So you'll now have up to 
> 7 session variables for the config file:
> 
>    pas.session.dbname
>    pas.session.driver
>    pas.session.dsn
>    pas.session.hostname
>    pas.session.password
>    pas.session.table
>    pas.session.username

Maybe prefix these with pas.session.db.* to follow the convention?  We
currently have pas.session.file.*, pas.session.apache.*, and 
pas.session.mysql.*, so perhaps it should be pas.session.db.*?

> Of course, if you specify the dsn, you can skip the dbname & hostname.  The 
> driver parameter will be a requirement!

The reason dsn is in there as a parameter is because it's actualy hard to
dynamicly build a dsn depending on the driver and the options the user
wants to pass. Maybe we should just forget the dbname, and hostname and
expect the user to specify them in the dsn?

> Any questions?  Does this sound good?  Can I take a chain saw to it?  I'll at 
> least get this working for MySQL in the near future.  Other implementations 
> shouldn't be hard.

It sounds good so far, I just want to talk about it some more until I feel
comfortable about it.


Kyle

-- 

------------------------------------------------------------------------------
Wisdom and Compassion are inseparable.
        -- Christmas Humphreys
mo...@vo...                            http://www.voicenet.com/~mortis
------------------------------------------------------------------------------