mod-security-developers

Re: [Mod-security-developers] Positive filter modelization proposal

[Vincent D. <vin...@in...>]

Ivan Ristic wrote:

>> Greetings,
>>
>> Here is a suggestion of XML modelization for an HTTP filter, based on a
>> positive model (ie, not blacklisting known attacks, but rather accepting
>> only known, validated good requests). This model is heavily inspired
>> from Ivan's publication of november 2005.
>
> I am somewhat confused. You took my work, changed it heavily, and
> now you are sending it back to me as your proposal. Wouldn't it
> have been better to point to the inefficiencies of my work so that
> we can improve it through discussion?

Yes, it would have been better, and I am sorry we couldn't.
As I thought I mentionned in former private emails, we had to work in big
hurry on the project, and we had to discuss the model internally in order
to get something we could use quickly. That is why we didn't have that
normal exchange, that I agree would have been much better for everyone.
Though, I think we have advanced, and I hope you can agree on some points
that we have changed from your initial proposal.

How do you want us to work for the better?
We can try to provide comments about all changes we made, and why we did,
in order to open a discussion.

PS : sorry I am destroying the thread this was started in ; for some
reason I did not receive your answer, even though I am subscribed to the
mailing list. I had to paste it from archives to write this answer.

Re: [Mod-security-developers] Positive filter modelization proposal

[Ivan R. <iv...@we...>]

Vincent Deffontaines wrote:
> Ivan Ristic wrote:

Vincent,

First of all my apologies for the extended delay.


> How do you want us to work for the better?
> We can try to provide comments about all changes we made, and why we did,
> in order to open a discussion.

That would certainly be interested. Personally I would advance by
compiling a definitive requirement list. It is something we could
use later on to test if we did all we set out to do.

Here are my original requirements:

 1. Access to some areas of a web site need not be controlled (allow all).
 2. Access to some areas of a web site must never be allowed (deny all).
 3. Web site is a collection of resources.
 4. Each resource can be used in one or more different ways (resource profiles).
 5. Each profile needs to support a different set of parameters. (For
    example, one set of parameters is used when the resource is invoked
    with GET, another set of parameters when resource invoked with POST.)

 - It sounds like 4 and 5 describe the same thing.

 6. Parameters are identified by their name (choose between case sensitive
    match and case insensitive match). Parameter attributes:

    6.1. Type (query string field, body field, or file)
    6.2. Cardinality
    6.3. Character class (or a fixed range of acceptable values)
         [we will probably need to support regular expressions and
          functions here]
    6.4. Length (minimum and maximum)
    6.5. Character distribution
    6.6. Max file size (for files only)
    6.7. File type (not based on the supplied C-T, of course)

    [One change I would like to make to my own model is to make the
     attributes listed above to be standalone constraints on parameters.]

 7. Additional support for parameter arrays (dynamically created paramters) is needed.
 8. Support for parameters transported in PATH_INFO is needed.
 9. Support for parameters transported in the URI (e.g. URI append) is needed.
10. Support for parameters that come from other places, for example request
    headers and cookies.
11. Metadata. A set of rules must be wrapped somehow, signed by a
    publisher and published. Change control (e.g. serial number) needs
    to be supported.
12. Support for partial models, e.g. continuous learning.
13. Compatibility with WAPL (see http://www.modsecurity.org/projects/ppr/)
    also see the same document for ideas that related to the requirements
    listed here.
14. Integration with negative security. E.g. if a part of the model is
    weak run the request through the negative security tests. If the anomaly
    score is low use the request for learning. If it is high - don't.
16. The format must be simple enough that can be written by hand.

-- 
Ivan Ristic, Technical Director
Thinking Stone, http://www.thinkingstone.com
ModSecurity: Open source Web Application Firewall
Apache Security (O'Reilly): http://www.apachesecurity.net

Re: [Mod-security-developers] Positive filter modelization proposal

[Vincent D. <vin...@in...>]

Hello Ivan and the list,

This is the answer from INL to those 16 items you listed. We are sorry
for the long delay.

We'll comment on the 16 topics you mentioned to tell how our model deals
or does not deal with them, and also add an extra requirement we had in
mind.
Before that, here is a short summary:

Some parts of our model[1] are ad-hoc and do not fit well with your
approach, especially with regard to topics 3 to 5 (web site as a
collection of resources, to handle with profiles).  We would like to
redesign it to fit with your approach.  Apart from that, we like the
flexibility and versatility of our system of tags and inheritence and
suggest this be included in the final model.

On Thu, 27 Jul 2006 20:44:29 +0100 Ivan Ristic <iv...@we...> wrote:

>  1. Access to some areas of a web site need not be controlled (allow all).

In our model, an allow-all policy can be expressed with a rule that
matches every resource of a given area.

>  2. Access to some areas of a web site must never be allowed (deny all).

Our model allows for a deny-all policy for a chosen area through the use
of tags and inheritence.  One can set a tag to block every rule which
inherits this tag (deny all).  However the tag/inheritence system may
not be the most simple device to deal with this since one has to add the
tag to every relevant rule (though a user interface or a script could
make this easy).

>  3. Web site is a collection of resources.
>  4. Each resource can be used in one or more different ways (resource
profiles).
>  5. Each profile needs to support a different set of parameters. (For
>     example, one set of parameters is used when the resource is invoked
>     with GET, another set of parameters when resource invoked with POST.)

Due to special requirements in our goals, we saw it the other way around:
our model is a collection of rules.  We think it needs to be redesigned
in order to switch to your approach, which seems better.

>  6. Parameters are identified by their name (choose between case sensitive
>     match and case insensitive match). Parameter attributes:

We match them with regular expressions.

>     6.5. Character distribution

What do you mean?

>     6.2. Cardinality
>     6.3. Character class (or a fixed range of acceptable values)
>          [we will probably need to support regular expressions and
>           functions here]
>     6.4. Length (minimum and maximum)
>
>     [One change I would like to make to my own model is to make the
>      attributes listed above to be standalone constraints on parameters.]
>  7. Additional support for parameter arrays (dynamically created
paramters) is needed.

We use regular expressions for that.

>  9. Support for parameters transported in the URI (e.g. URI append) is
needed.

Not fully supported in our model but this would typically be redesigned
along with the modsecurity rule format.

> 11. Metadata. A set of rules must be wrapped somehow, signed by a
>     publisher and published. Change control (e.g. serial number) needs
>     to be supported.

Our model is an ordered collection of rules.  Consequently, rules from
external publishers would have either to be placed correctly by the user
or to be independant from other possible rules. Maybe the model should
evolve to something not ordered, but rather with a "priority" tag, to
deal with rules that "conflict" with each other. But this seems like a
heavy task for the (admin) user interface : the interface would need to
detect which rules conflict with one another, which seems tough if rules
refer to regular expressions.
How do you see it?

> 12. Support for partial models, e.g. continuous learning.

If we understand correctly, our model is not very good at that because
of its ordered rules (see previous paragraph).

> 16. The format must be simple enough that can be written by hand.

Our format, based on XML, can be painfully written by hand; it was not
specifically designed for this.  It is supposed to be generated with a
user interface or a script.  The most obvious flaw is the numerical
attribute "id" (the ids are unique).  For inheritence, parents must be
specified with their id.  Thus, readability relies on comments, e.g. :
<parent id="3"/>  <!-- host some-site.com -->


We would add a requirement to this list:

17. It should be possible to register which rule blocked a request.
This is for debugging but also improving the set of rules.  This
concerns more the implementation than the model design.  Modsecurity
already allows for this.


Finally, here we copy the requirements which our model does not meet (as
is):

>     6.1. Type (query string field, body field, or file)
>     6.6. Max file size (for files only)
>     6.7. File type (not based on the supplied C-T, of course)
>  8. Support for parameters transported in PATH_INFO is needed.
> 10. Support for parameters that come from other places, for example
request
>     headers and cookies.
> 13. Compatibility with WAPL (see http://www.modsecurity.org/projects/ppr/)
>     also see the same document for ideas that related to the requirements
>     listed here.
> 14. Integration with negative security. E.g. if a part of the model is
>     weak run the request through the negative security tests. If the
anomaly
>     score is low use the request for learning. If it is high - don't.

Regards,



[1] What is mentionned "our model" herein is work we suggested in last
email to this list, and is indeed based on Ivan's initial proposal.

-- 
François Toussenel
Vincent Deffontaines
INL, http://www.inl.fr/

Re: [Mod-security-developers] Positive filter modelization proposal

[Ivan R. <iva...@br...>]

Vincent Deffontaines wrote:
> Hello Ivan and the list,
> 
> This is the answer from INL to those 16 items you listed. We are sorry
> for the long delay.

Hi Vincent,

Thank you for your answers. I just wanted to let you know I have been
overwhelmed with work recently and that I am not finding any spare time
to dedicate to responding to your email. That does not mean I don't want
to or that I've given up. I will answer eventually, it is just that it
may be a while.

I am sorry I cannot provide a better answer at this time.

-- 
Ivan Ristic