Webware for Python

At 03:51 PM 12/21/01 -0500, Edmund Lian wrote:

>Geoff wrote:
>
> >>The Extensions* settings only affect requests where you left off the
>extension.
>
>The Files* settings affect all requests, with or without the extension as
>part of the URL.<<
>
>Aha! Thank you! these are two important sentences. Perhaps they should be
>in the docs?

I just updated the docs for Files* in CVS to indicate that they apply to 
all requests, not just ones with autodetected extensions.  The docs for 
Extensions* already indicate pretty clearly that they only apply to 
requests with autodetected extensions.

I also added *.pyc, *.pyo, and *.config to the default setting for FilesToHide.


--

- Geoff Talvola
   gtalvola@...

On Friday 21 December 2001 12:51, Edmund Lian wrote:
> Geoff wrote:
> >>The Extensions* settings only affect requests where you left off
> >> the
>
> extension.
>
> The Files* settings affect all requests, with or without the
> extension as part of the URL.<<
>
> Aha! Thank you! these are two important sentences. Perhaps they
> should be in the docs?
>
> ...Edmund.

Maybe we should also rename 'ExtensionsToServe' as 'ExtensionsToUse' 
so there is no confusion with 'FilesToServe'.

ExtenionsToUse
ExtenionsToIgnore

FilesToServe
FilesToHide

Your thoughts?

Tavis wrote:

>>Maybe we should also rename 'ExtensionsToServe' as 'ExtensionsToUse'
so there is no confusion with 'FilesToServe'.

ExtenionsToUse
ExtenionsToIgnore

FilesToServe
FilesToHide

Your thoughts?<<

How about:

ExtensionsToTry
ExtensionsToMask

FilesToServe
FilesToMask

...Edmund.

On Tuesday 01 January 2002 13:14, Edmund Lian wrote:
> How about:
>
> ExtensionsToTry
> ExtensionsToMask

Sounds good to me.

> FilesToServe
> FilesToMask

In this case I prefer FilesToHide or FilesToProtect as it raises a 
403 Forbidden error.

Tavis wrote:


>> FilesToServe
>> FilesToMask

>In this case I prefer FilesToHide or FilesToProtect as it raises a
403 Forbidden error.<

Personally, I think FilesToProtect makes sense given the 403 error...

...Edmund.

On Tue, 1 Jan 2002, Edmund Lian wrote:

> >In this case I prefer FilesToHide or FilesToProtect as it raises a
> 403 Forbidden error.<
>
> Personally, I think FilesToProtect makes sense given the 403 error...

Might want to make two options:

	FilesToProtect :	raises 403 Forbidden
	FilesToHide :		raises 404 Not Found

This way, you don't give outsiders more information than they need about
the "hidden" or "protected" contents of your site. With 403, they know the
file is there, they just don't know how to get it (yet...)

tlilley wrote:

>>Might want to make two options:

           FilesToProtect :         raises 403 Forbidden
           FilesToHide :                  raises 404 Not Found

This way, you don't give outsiders more information than they need about
the "hidden" or "protected" contents of your site. With 403, they know the
file is there, they just don't know how to get it (yet...)<<

Indeed. If so though, why even give the would-be cracker such precise
information? Perhaps a 404 error should be returned no matter what the
cause, and just use a FilesToProtect setting alone to simplify things. I'm
not sure that having such granularity (FilesToProtect and FilesToHide)
would buy us anything extra.

...Edmund.

On Tuesday 01 January 2002 08:58 pm, Edmund Lian wrote:
> Indeed. If so though, why even give the would-be cracker such precise
> information? Perhaps a 404 error should be returned no matter what
> the cause, and just use a FilesToProtect setting alone to simplify
> things. I'm not sure that having such granularity (FilesToProtect and
> FilesToHide) would buy us anything extra.

I agree. Unless someone has an argument for 403 Forbidden, I prefer to 
just have 404 Not Found.

-Chuck

On Wednesday 02 January 2002 00:00, Chuck Esterbrook wrote:
> On Tuesday 01 January 2002 08:58 pm, Edmund Lian wrote:
> > Indeed. If so though, why even give the would-be cracker such
> > precise information? Perhaps a 404 error should be returned no
> > matter what the cause, and just use a FilesToProtect setting
> > alone to simplify things. I'm not sure that having such
> > granularity (FilesToProtect and FilesToHide) would buy us
> > anything extra.
>
> I agree. Unless someone has an argument for 403 Forbidden, I prefer
> to just have 404 Not Found.
>

I'm not sure we gain anything extra by returning a 404 instead of 
403.  This is essentially security by obscurity, but it's not clear 
what we're trying to obscure.  Anyone familiar with WebKit will know 
that .pyc files exist and that .py~ files probably exist.  What else 
might we be revealing?

On the other hand, a 404 error in the logs mean something different 
than a 403 error.  Using 404 in this case would obscure what is 
really going on with bogus requests.

Apache returns 403 for a .htaccess, .htpasswd, etc. so I prefer it to 
404. In fact, it returns 403 even if those files don't exist. The 
WebKit stuff is all happening after the webserver's initial request 
processing so those who do want a 404 message can still use 
mod_rewrite to get one.

Tavis

On Wednesday 02 January 2002 10:04 am, Tavis Rudd wrote:
> > I agree. Unless someone has an argument for 403 Forbidden, I prefer
> > to just have 404 Not Found.
>
> I'm not sure we gain anything extra by returning a 404 instead of
> 403.  This is essentially security by obscurity, but it's not clear
> what we're trying to obscure.  Anyone familiar with WebKit will know
> that .pyc files exist and that .py~ files probably exist.  What else
> might we be revealing?

Regarding security, I prefer the position "What is the motivation for 
revealing internal details of the system?" If there is no such 
motivation, I don't reveal the detail.

I think that's a safer approach than exposing unnecessary details of a 
system because we can't currently imagine any harm.


> On the other hand, a 404 error in the logs mean something different
> than a 403 error.  Using 404 in this case would obscure what is
> really going on with bogus requests.

True, although I can't say I really mind.


> Apache returns 403 for a .htaccess, .htpasswd, etc. so I prefer it to
> 404. In fact, it returns 403 even if those files don't exist. The
> WebKit stuff is all happening after the webserver's initial request
> processing so those who do want a 404 message can still use
> mod_rewrite to get one.

I find it obscure that Apache would 403 a non-existant file since it 
would lead me to believe it was there (perhaps I'm poking around as an 
admin).


In any case, should we have FilesToForbid and FilesToNotFind? That way 
the names match the response and you can do as you like. Or we could 
have something more generic that lets you specify the status code and 
string, perhaps even multiple groups.

Thoughts?



Additionally, I would like to deny the use of extensions for my site 
for various reasons:
  - it's unnecessary
  - some of my servlet code (in SitePage) parses the URL and I don't 
want to have to test the site with and without extensions
  - if the user bookmarks such a URL, it could break later when I 
switch techniques

I haven't thought about where to put that in the process.


-Chuck

On Wed, Jan 02, 2002 at 10:56:20AM -0800, Chuck Esterbrook wrote:
> On Wednesday 02 January 2002 10:04 am, Tavis Rudd wrote:
> > > I agree. Unless someone has an argument for 403 Forbidden, I prefer
> > > to just have 404 Not Found.
> >
> > I'm not sure we gain anything extra by returning a 404 instead of
> > 403.  This is essentially security by obscurity, but it's not clear
> > what we're trying to obscure.  Anyone familiar with WebKit will know
> > that .pyc files exist and that .py~ files probably exist.  What else
> > might we be revealing?
> 
> Regarding security, I prefer the position "What is the motivation for 
> revealing internal details of the system?" If there is no such 
> motivation, I don't reveal the detail.
> 
> I think that's a safer approach than exposing unnecessary details of a 
> system because we can't currently imagine any harm.

Forbidden doesn't necessarily mean the file exists.  It just means the 
server is denying the request for some policy reason.  For instance,
maybe there's a DENY FROM ALL on the entire directory, or maybe your
site is blacklisted, or maybe the maintainer is doing updates and wants
to lock that section out until he's done.  

Forbidden means "Go away!  Scram!  You're not wanted here!"  Not found
may be interpreted as, "Oops, you may have mistyped the URL, try again."

On the other hand, if we want to pretend *.pyc and *.py~ aren't in the
webspace, maybe Not Found would be appropriate.

I agree that we should follow Apache's model and use Forbidden for any
security-sensitive files like .webkit, whether or not they exist.

-- 
-Mike (Iron) Orr, iron@...  (if mail problems: mso@...)
   http://iron.cx/     English * Esperanto * Russkiy * Deutsch * Espan~ol

On Wednesday 02 January 2002 10:56, Chuck Esterbrook wrote:
> In any case, should we have FilesToForbid and FilesToNotFind? That
> way the names match the response and you can do as you like. Or we
> could have something more generic that lets you specify the status
> code and string, perhaps even multiple groups.
>
> Thoughts?

Whatever approach we go with, I agree that the name should be 
explicit.

> Additionally, I would like to deny the use of extensions for my
> site for various reasons:
>   - it's unnecessary
>   - some of my servlet code (in SitePage) parses the URL and I
> don't want to have to test the site with and without extensions
>   - if the user bookmarks such a URL, it could break later when I
> switch techniques
>
> I haven't thought about where to put that in the process.

You'd still want to make sure extensions worked for .gif, .jpeg, etc.

Apache's Rewrite directive or mod_rewrite could be used to make sure 
that extensions are left off.  Do you want 404s or graceful URL 
rewrites when the extension is given?

well said.

On Wednesday 02 January 2002 11:32, Mike Orr wrote:
> On Wed, Jan 02, 2002 at 10:56:20AM -0800, Chuck Esterbrook wrote:
> > On Wednesday 02 January 2002 10:04 am, Tavis Rudd wrote:
> > > > I agree. Unless someone has an argument for 403 Forbidden, I
> > > > prefer to just have 404 Not Found.
> > >
> > > I'm not sure we gain anything extra by returning a 404 instead
> > > of 403.  This is essentially security by obscurity, but it's
> > > not clear what we're trying to obscure.  Anyone familiar with
> > > WebKit will know that .pyc files exist and that .py~ files
> > > probably exist.  What else might we be revealing?
> >
> > Regarding security, I prefer the position "What is the motivation
> > for revealing internal details of the system?" If there is no
> > such motivation, I don't reveal the detail.
> >
> > I think that's a safer approach than exposing unnecessary details
> > of a system because we can't currently imagine any harm.
>
> Forbidden doesn't necessarily mean the file exists.  It just means
> the server is denying the request for some policy reason.  For
> instance, maybe there's a DENY FROM ALL on the entire directory, or
> maybe your site is blacklisted, or maybe the maintainer is doing
> updates and wants to lock that section out until he's done.
>
> Forbidden means "Go away!  Scram!  You're not wanted here!"  Not
> found may be interpreted as, "Oops, you may have mistyped the URL,
> try again."
>
> On the other hand, if we want to pretend *.pyc and *.py~ aren't in
> the webspace, maybe Not Found would be appropriate.
>
> I agree that we should follow Apache's model and use Forbidden for
> any security-sensitive files like .webkit, whether or not they
> exist.

On Wednesday 02 January 2002 12:42 pm, Tavis Rudd wrote:
> You'd still want to make sure extensions worked for .gif, .jpeg, etc.

Why? If I switch from .gif to .png then my IMG tags break.

In my own sites, I have no use for extensions at all. But I'm not 
proposing that WebKit always cut them off. I realize some people will 
want to use them.


> Apache's Rewrite directive or mod_rewrite could be used to make sure
> that extensions are left off.  Do you want 404s or graceful URL
> rewrites when the extension is given?

404s. From an external perspective my site is neither Webware nor CGI 
nor ASP or whatever. I believe in opaqueness. Users of my site need a 
UI, not technical access.


-Chuck

On Wednesday 02 January 2002 12:02, Chuck Esterbrook wrote:
> On Wednesday 02 January 2002 12:42 pm, Tavis Rudd wrote:
> > You'd still want to make sure extensions worked for .gif, .jpeg,
> > etc.
>
> Why? If I switch from .gif to .png then my IMG tags break.

Fair enough.

> In my own sites, I have no use for extensions at all. But I'm not
> proposing that WebKit always cut them off. I realize some people
> will want to use them.
>
> > Apache's Rewrite directive or mod_rewrite could be used to make
> > sure that extensions are left off.  Do you want 404s or graceful
> > URL rewrites when the extension is given?
>
> 404s. From an external perspective my site is neither Webware nor
> CGI nor ASP or whatever. I believe in opaqueness. Users of my site
> need a UI, not technical access.

Apache would do either, but I'm not sure how to do it with WebKit 
alone.

On Wed, 2002-01-02 at 14:02, Chuck Esterbrook wrote:
> On Wednesday 02 January 2002 12:42 pm, Tavis Rudd wrote:
> > You'd still want to make sure extensions worked for .gif, .jpeg, etc.
> 
> Why? If I switch from .gif to .png then my IMG tags break.

It's not as big a deal, though, because you can change all your links on
your site.  Very few people will do deep linking to an image, unlike an
HTML page.

  Ian