Webware for Python

I've just done a little more work on this in the experimental code.  
Here are the config settings I've implemented and tested so far:

DirectoryFiles = ['index','Index','main','Main', 'default','Default',]

## these 2 only affect requests with no extension specified
# same as before
ExtensionsToHide = ['.pyc','.pyo','.py~', '.bak', '.tmpl', '.py_bak']
# only use if a list is given
ExtensionsToServe = None  
# ExtensionsToServe = ['.py','.html']


# if multiple files are found for a URI without the ext specified
# cascade through this list in sequence till one is found.
# 404 if none match
UseCascadingExtensions = True
ExtensionCascadeOrder = ['.html', '.py', '.psp', '.tmpl']

# a list of glob patterns to filter out after all the rest of the 
# path matching is finished.  404 if matches
FilesToHide = ['.*','*~', '*bak', '*.tmpl', ]

# a list glob patterns to serve from exclusively.
# if the file found for the URI doesn't match then 404
# done after FilesToHide
FilesToServe = None  # only used if a list is given
#FilesToServe = ['*.py', '*.jpg','*.gif']


Regardless of whether the rest of the experimental code is used I 
feel this stuff should definitely make it in. What do you think about 
the names I've given the settings?  ExtensionsToServe and 
FilesToServe are a bit ambiguous.  I'm leaning towards 
FilePatternsToServe.

Tavis

On Wednesday 12 December 2001 11:57, Love, Jay wrote:
> We've talked about having an ExtensionsToServe numerous times. 
> Perhaps this should be a configuration option, say
> "LimitFileTpesServed", and then ExtensionsToServe would list what
> may be served.
>
> J
>
> > -----Original Message-----
> > From: Geoffrey Talvola [mailto:gtalvola@...]
> > Sent: Wednesday, December 12, 2001 2:51 PM
> > To: tavis@...; Webware-devel@...
> > Subject: Re: [Webware-devel] security hole in WebKit
> >
> > At 11:55 AM 12/12/01 -0800, Tavis Rudd wrote:
> > >Hi,
> > >in the cvs version of WebKit (and I assume all previous
> > > versions) it's possible to access backup versions of the .py
> > > servlet files: http://localhost/WK/Welcome.py~ for example.
> > > This could expose information about the site that should be
> > > kept private.  Consider http://localhost/WK/.htpasswd. While
> > > the ExtensionsToIgnore setting works when the extension isn't
> > > specified in the URI, it provides no protection when it is.
> > >
> > >A solution is to make WebKit accept a list of files that it will
> > >never serve ('FilesToIgnore' or 'FilesToHide').  The setting
> > > could be a list of plain string filenames, or a list of
> > > patterns to match. Conversely, it should accept a list of
> > > files/patterns that it will serve from exclusively
> > > ('FilesToServe').
> > >
> > >Also, I propose that 'ExtensionsToIgnore' be renamed
> > >'ExtensionsToHide', making its purpose clearer. 
> > > 'ExtensionsToServe' should be implemented as well.
> >
> > Also, even if you're not editing your live site and leaving
> > backup files
> > lying around, you'll still have *.pyc files in there that can
> > be fetched
> > and then potentially decompiled.
> >
> >
> > --
> >
> > - Geoff Talvola
> >    gtalvola@...
> >
> > _______________________________________________
> > Webware-devel mailing list
> > Webware-devel@...
> > https://lists.sourceforge.net/lists/listinfo/webware-devel
>
> -------------------------------------------------------------------
>---------
>
> This e-mail and any attachments may be confidential or legally
> privileged. If you received this message in error or are not the
> intended recipient, you should destroy the e-mail message and any
> attachments or copies, and you are prohibited from retaining,
> distributing, disclosing or using any information contained herein.
>  Please inform us of the erroneous delivery by return e-mail.
>
> Thank you for your cooperation.
>
> -------------------------------------------------------------------
>---------

2 questions:

- Can this be made backward-compatible by also allowing the name 
"ExtensionsToIgnore", perhaps emitting a deprecation warning message?

- Now that you've done the work in your experimental version, could you 
adapt it to create a patch for Webware CVS?

At 01:40 PM 12/12/01 -0800, Tavis Rudd wrote:
>I've just done a little more work on this in the experimental code.
>Here are the config settings I've implemented and tested so far:
>
>DirectoryFiles = ['index','Index','main','Main', 'default','Default',]
>
>## these 2 only affect requests with no extension specified
># same as before
>ExtensionsToHide = ['.pyc','.pyo','.py~', '.bak', '.tmpl', '.py_bak']
># only use if a list is given
>ExtensionsToServe = None
># ExtensionsToServe = ['.py','.html']
>
>
># if multiple files are found for a URI without the ext specified
># cascade through this list in sequence till one is found.
># 404 if none match
>UseCascadingExtensions = True
>ExtensionCascadeOrder = ['.html', '.py', '.psp', '.tmpl']
>
># a list of glob patterns to filter out after all the rest of the
># path matching is finished.  404 if matches
>FilesToHide = ['.*','*~', '*bak', '*.tmpl', ]
>
># a list glob patterns to serve from exclusively.
># if the file found for the URI doesn't match then 404
># done after FilesToHide
>FilesToServe = None  # only used if a list is given
>#FilesToServe = ['*.py', '*.jpg','*.gif']
>
>
>Regardless of whether the rest of the experimental code is used I
>feel this stuff should definitely make it in. What do you think about
>the names I've given the settings?  ExtensionsToServe and
>FilesToServe are a bit ambiguous.  I'm leaning towards
>FilePatternsToServe.
>
>Tavis

--

- Geoff Talvola
   gtalvola@...

On Thursday 13 December 2001 11:59, Geoffrey Talvola wrote:
> 2 questions:
>
> - Can this be made backward-compatible by also allowing the name
> "ExtensionsToIgnore", perhaps emitting a deprecation warning
> message?

Sure, why don't we just stick with ExtensionsToIgnore for now?  I 
think 'ExtensionsToHide' makes the purpose of the setting clearer and 
should be used in 0.7.  

> - Now that you've done the work in your experimental version, could
> you adapt it to create a patch for Webware CVS?

Sure, though it might take me a while to remember/regrok how this 
stuff is dealt with in Webware 0.6 ;)


> At 01:40 PM 12/12/01 -0800, Tavis Rudd wrote:
> >I've just done a little more work on this in the experimental
> > code. Here are the config settings I've implemented and tested so
> > far:
> >
> >DirectoryFiles = ['index','Index','main','Main',
> > 'default','Default',]
> >
> >## these 2 only affect requests with no extension specified
> ># same as before
> >ExtensionsToHide = ['.pyc','.pyo','.py~', '.bak', '.tmpl',
> > '.py_bak'] # only use if a list is given
> >ExtensionsToServe = None
> ># ExtensionsToServe = ['.py','.html']
> >
> >
> ># if multiple files are found for a URI without the ext specified
> ># cascade through this list in sequence till one is found.
> ># 404 if none match
> >UseCascadingExtensions = True
> >ExtensionCascadeOrder = ['.html', '.py', '.psp', '.tmpl']
> >
> ># a list of glob patterns to filter out after all the rest of the
> ># path matching is finished.  404 if matches
> >FilesToHide = ['.*','*~', '*bak', '*.tmpl', ]
> >
> ># a list glob patterns to serve from exclusively.
> ># if the file found for the URI doesn't match then 404
> ># done after FilesToHide
> >FilesToServe = None  # only used if a list is given
> >#FilesToServe = ['*.py', '*.jpg','*.gif']
> >
> >
> >Regardless of whether the rest of the experimental code is used I
> >feel this stuff should definitely make it in. What do you think
> > about the names I've given the settings?  ExtensionsToServe and
> > FilesToServe are a bit ambiguous.  I'm leaning towards
> >FilePatternsToServe.
> >
> >Tavis

I submitted at patch to the webware SF Patches page a while back
that implements "ExtensionsToServe":

  https://sourceforge.net/tracker/index.php?func=detail&aid=486598&group_id=4866&atid=304866

But the fix is incomplete: it only applies to URIs that don't specify
a trailing extension.  Changing it to apply to all requests,
and making ExtensionsToServe the default, is certainly worth doing.

Here's a patch to implement ExtensionsToServe, FilesToHide, and 
FilesToServe. They work as I documented yesterday, and will also need 
to be added to the default config file.  

IMHO,there's far too much coupling in there between the Application 
and Request classes, which makes it harder than it should be to grok, or 
implement new stuff like this.

Tavis


On Thursday 13 December 2001 11:59, Geoffrey Talvola wrote:
> 2 questions:
>
> - Can this be made backward-compatible by also allowing the name
> "ExtensionsToIgnore", perhaps emitting a deprecation warning
> message?
>
> - Now that you've done the work in your experimental version, could
> you adapt it to create a patch for Webware CVS?
>
> At 01:40 PM 12/12/01 -0800, Tavis Rudd wrote:
> >I've just done a little more work on this in the experimental
> > code. Here are the config settings I've implemented and tested so
> > far:
> >
> >DirectoryFiles = ['index','Index','main','Main',
> > 'default','Default',]
> >
> >## these 2 only affect requests with no extension specified
> ># same as before
> >ExtensionsToHide = ['.pyc','.pyo','.py~', '.bak', '.tmpl',
> > '.py_bak'] # only use if a list is given
> >ExtensionsToServe = None
> ># ExtensionsToServe = ['.py','.html']
> >
> >
> ># if multiple files are found for a URI without the ext specified
> ># cascade through this list in sequence till one is found.
> ># 404 if none match
> >UseCascadingExtensions = True
> >ExtensionCascadeOrder = ['.html', '.py', '.psp', '.tmpl']
> >
> ># a list of glob patterns to filter out after all the rest of the
> ># path matching is finished.  404 if matches
> >FilesToHide = ['.*','*~', '*bak', '*.tmpl', ]
> >
> ># a list glob patterns to serve from exclusively.
> ># if the file found for the URI doesn't match then 404
> ># done after FilesToHide
> >FilesToServe = None  # only used if a list is given
> >#FilesToServe = ['*.py', '*.jpg','*.gif']
> >
> >
> >Regardless of whether the rest of the experimental code is used I
> >feel this stuff should definitely make it in. What do you think
> > about the names I've given the settings?  ExtensionsToServe and
> > FilesToServe are a bit ambiguous.  I'm leaning towards
> >FilePatternsToServe.
> >
> >Tavis

At 04:04 PM 12/13/01 -0800, Tavis Rudd wrote:
>Here's a patch to implement ExtensionsToServe, FilesToHide, and
>FilesToServe. They work as I documented yesterday, and will also need
>to be added to the default config file.
>
>IMHO,there's far too much coupling in there between the Application
>and Request classes, which makes it harder than it should be to grok, or
>implement new stuff like this.
>
>Tavis

Excellent!  I'll check in the patch to CVS sometime in the next couple of 
days, unless Chuck gets to it before I do.




- Geoff Talvola
   gtalvola@...

On Friday 14 December 2001 06:55, Geoffrey Talvola wrote:
> At 04:04 PM 12/13/01 -0800, Tavis Rudd wrote:
> >Here's a patch to implement ExtensionsToServe, FilesToHide, and
> >FilesToServe. They work as I documented yesterday, and will also
> > need to be added to the default config file.
>
> Excellent!  I'll check in the patch to CVS sometime in the next
> couple of days, unless Chuck gets to it before I do.

Before you do ... here's an update of that patch that also implements 
'ExtensionCascadeOrder', which is a list of extensions to cascade 
through when there are multiple files with different extensions found 
for a requested basename.  The first match from the list 
'ExtensionCascadeOrder' is used.  If no match is found a 404 error is 
returned.  The setting 'UseCascadingExtensions' turns the behaviour 
on and off.  The patch includes the changes to the default 
Configs/Application.config file.

Tavis