ipfilter-devel — Mailing list for IPFilter developers

[sf-ipf-dev] SNMP things that still need to be discussed and decided.

[Darren R. <da...@re...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
 
So there are a couple of things that I should mention...

In SNMP traps, the trap message should include the IP address
of the sender. This raises the obvious question of which address,
of the many configured, should ipmon use?

Should the address be configurable per action or should the same
address be used for all traps?

Next is the request id. This is SNMPv2 specific. At present I set
this to 0...but should it be something else?

All of the error fields, in both v1 and v2, I set to 0. Should one of
these reflect some specific thing from the ipfilter log information?

Last but not least, each SNMP trap carries a time field in it that
says how long "it" has been up. I'm not sure what to do about this.
At present I just report the time of the log event. But should it
report the "ipf ticks" value or actual system uptime?

Cheers,
Darren

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
 
iEYEARECAAYFAkjI5G0ACgkQP7JIXtvLbFU6XACePW8ooNXjBqyiFBdpkX+bY3o3
urkAoOqLoyqja9QmW+rzdisdmS6pHFUs
=I204
-----END PGP SIGNATURE-----

[sf-ipf-dev] SNMP trap sending added to IPFilter 5

[Darren R. <da...@re...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
 
At some point in the past I implemented SNMP trap sending for IPFilter.

That code seemed to get lost, so I had to start again from scratch.
Bitch.

But now I seem to have both v1 and v2 trap sending from ipmon working.

The configuration goes in ipmon.conf and is like this:

match { logtag = 10000 }
do { send-trap v1 community public 192.168.1.239 };
#
match { logtag = 10000 }
do { send-trap v2 community read 192.168.1.239 };
#

Cheers,
Darren
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
 
iEYEARECAAYFAkjI4uQACgkQP7JIXtvLbFUPLgCfXou7RIHDjLlmkXJI5ECqHEXK
9UEAoKNtY7SC1DPbKMqShkitCGerRysR
=7ObW
-----END PGP SIGNATURE-----

[sf-ipf-dev] Changes to implement instances in IP filter.

[Darren R. <da...@re...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
 
Over the last month I've been making steady progress on implementing
support for multiple instances of ipfilter at runtime. Most of the code is
done and I'm currently working on adding in comments where required.

I haven't yet tested with zones and IP instances ... there's a chunk of
code waiting to be written for that yet, too.  But ipftest builds and so
too do the kernel modules for BSD.

The current changes can be found at:
http://coombs.anu.edu.au/~avalon/ipf5-instances.diffs,gz
MD5 (ipf5-instances.diffs.gz) = ebbbc334901c9af1c0e02e0928fdaf10

And when uncompressed, wc says:
~   50564  160626 1374441 ipf5-instances.diffs

Darren
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
 
iEYEARECAAYFAkiX9lEACgkQP7JIXtvLbFXt4wCfVGA/YKQA5Cr3pu5dPqbVqRxP
gVUAniLJO8YiXdc91BYjydmaYUDh6lMX
=afMz
-----END PGP SIGNATURE-----

[sf-ipf-dev] Proposed cleanup of iterator functions ...

[John O. <John.Ojemann@Sun.COM>]

Hi,

While investigating a memory leak and system panic, it didn't take long 
to realize that the various iterator functions, and how tokens were 
handled in and around them, needed a bit of cleaning up.

When a user utility completes any list processing, the token used to 
walk the data should be freed up.  Currently, this only happens in some 
cases using ipfstat command when an additional ioctl call from the user 
program cleans up the token.

Two bugs were submitted, 1979488 for head branch and 1979427 for release 
branch, to address this problem.  The proposed changes not only clean up 
each of the functions, but they remove any token processing requirements 
from the user level utilities.

Please check out the proposed changes attached to the two bugs.  The 
changes build and test for both branches without any issues, and have 
been thoroughly verified on Solaris platform.  Unless someone notices a 
problem, I'll plan to commit them at the end of the month.

Thanks,
John

Re: [sf-ipf-dev] making table sizes and queue timeouts changable at runtime (ipf 5.0).

[S. N. <ane...@gm...>]

looks good, however I'm still in temptation to try to use trees instead of
hashtables. the penalty in term of more CPU cycles to lookup entry is
acceptable,
since it solve many headaches curent users have with maintainance of hashtables.

I'll see how much time I'll will have over the summer and I'll try to hack some
existing implementation into IPF.

regards
sasha

2008/5/12 Darren Reed <da...@re...>:
> Over the weekend (and a bit of tonight/friday night), I hacked
>  up some changes to support using "ipf -T" on timeouts and table
>  sizes at runtime, without having to disable IPFilter.  This will
>  also open the door to being able to do "set <name> <value>;" in
>  ipf.conf - something that should make life easier for a lot of
>  people.  The only catch will be if changing the size of the table
>  is requested at runtime and there are a large number of active
>  entries (think closer to 100,000 or more) as the resize needs to
>  reinsert every entry back into the new table.
>
>  The preliminary diffs are at:
>  http://coombs.anu.edu.au/~avalon/ipf-T-runtime.patch
>
>  If nobody can see a problem with this approach, I'll commit these
>  changes later on in the week.
>
>  I have no plans to backport this to ipf 4.1.*
>
>  Cheers,
>  Darren
>
>
>  -------------------------------------------------------------------------
>  This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>  Don't miss this year's exciting event. There's still time to save $100.
>  Use priority code J8TL2D2.
>  http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
>  _______________________________________________
>  Ipfilter-devel mailing list
>  Ipf...@li...
>  https://lists.sourceforge.net/lists/listinfo/ipfilter-devel
>

[sf-ipf-dev] making table sizes and queue timeouts changable at runtime (ipf 5.0).

[Darren R. <da...@re...>]

Over the weekend (and a bit of tonight/friday night), I hacked
up some changes to support using "ipf -T" on timeouts and table
sizes at runtime, without having to disable IPFilter.  This will
also open the door to being able to do "set <name> <value>;" in
ipf.conf - something that should make life easier for a lot of
people.  The only catch will be if changing the size of the table
is requested at runtime and there are a large number of active
entries (think closer to 100,000 or more) as the resize needs to
reinsert every entry back into the new table.

The preliminary diffs are at:
http://coombs.anu.edu.au/~avalon/ipf-T-runtime.patch

If nobody can see a problem with this approach, I'll commit these
changes later on in the week.

I have no plans to backport this to ipf 4.1.*

Cheers,
Darren

[sf-ipf-dev] Recent changes for destination lists

[Darren R. <da...@re...>]

The recent commits I made to add destination lists to ippool.conf allow
for the input syntax between ipf.conf and ippool.conf to be as is shown
below.  Some updating of manual pages will be required...

Darren

pool ipf/in dst-list(name john; <style>) { bge0:1.2.2.3; hme0:2.2.2.2;}

style = round-robin | weighted [connection|session|bytes] | random

pass in on le0 to pool/john proto tcp from any to any port = 80

pool ipf/out tree(name mark;) {};

pool nat/in hash(name mary; size 101; seed 55;) {};

pool ipf/in group-map(name peter;) {};

pool nat/out dst-list ( name venus; weighted bytes;) { 1.2.3.4; 5.6.7.8; };

[sf-ipf-dev] First cut of ipv6 NAT committed

[Darren R. <da...@re...>]

Last night I did a commit to add in the first set
of changes to support IPv6 NAT.  The current task
to follow that on is to expand the test suite
coverage of IPv6 NAT to further gain confidence
in it working.  I haven't yet created a .tar.gz
file with all of this in it, so you'll need to
grab it directly from CVS on sourceforge, at
present, if you want to give it a whirl.

Cheers,
Darren

[sf-ipf-dev] IPv6 NAT patches for IPFilter

[Darren R. <da...@re...>]

This weekend I've done some hacking and slashing to make a
start on IPv6 NAT support in IPFilter.

There are a few niggles at the moment, with some of the NAT
facilities (such as map-block and other new things in 5.0)
not yet implemented because there's no native 128bit math
operations in C yet.  But all of the ordinary things are
there...


This set of patches puts the IPv4 and IPv6 NAT entries into
the same hash table, so things like the hash table size and
number of entries are configured once for both protocols.

http://coombs.anu.edu.au/~avalon/ipv6nat.diffs.gz

Cheers,
Darren

[sf-ipf-dev] off topic: spam and your mailbox...

[Darren R. <da...@re...>]

In the last day or so, I've seen spam target ipfilter-devel
on sourceforge but none of the other ipfilter lists there,
so I'm going to assume that someone who's subscribed to the
list has had their mailbox or address book "raped and pillaged",
but I've no idea who.  Or someone "planted" the mailing list
address somewhere (less likely, I hope.)

I'd therefore encourage subscribers to look at how exposed
you are to outsiders, because I'm pretty positive that the
exposure of the list has not been deliberate.

Cheers,
Darren

Re: [sf-ipf-dev] IPF release cycle model

[<ane...@gm...>]

Hello,

> There's more than one reason for that...
> And mostly the reasons for "bursts" is not enough time
> to test/compile/build everywhere.

that's the point. Since there is no time to do testing of release version, it
would be better to put fix into head and force people to try out the head
version to see if problem is fixed there or not.

also some problems were introduced as 'code clean ups' into release version,
only fixes should be allowed there in general. the maintainer of release head
should be very conservative to accept a code clean up, which has ''no effect''
to living code except it makes it to look less ugly.

the maintainer of release branch would have a right to refuse patch, on the
other hand he would be responsible for seamless release.

> That's wrong.  Everyone who's part of the project on sourceforge as a
> developer, including you, is able to commit to the CVS repository on
> sourceforge.  Currently the only other person who has is from NetBSD
> (Martti.)

I know. The reason I'm not commiting into project is I'm afraid of I could
break something. Therefore I'm trying to set up some rules such as where bugfix
should go first and so on just to minimize impact to users.

> Before making such rules, you need to understand that there are exceptions.
>
> For example, the HISTORY file and others with the version number are updated
> for every release.  This requires a committed change in CVS.  It is silly to
> have a bugid for that.  So (2) needs to be waived for release engineering.

I know. Pavel has the same objection. I'm trying to focus on the code part.
It's definitely good point. the anything else than code should be handled
differently.

Talking about the HISTORY file it should be prepared by RELASE maintainer, or
even generated from CVS messages, providing each commit in code has its BugID.

> > (3) every change must appear in HEAD first, no excuse.
>
> I disagree.  Sometimes a change is only relevant to a release branch.

I agree, we can think of some more exceptions i.e. bugs with security impact.
We can think of exceptions once we will have some sort of framework and some
roles assigned.

> There are three factors that influence when I do releases at present:
> 1) release cycles of BSDs
we should probably ignore a release cycle of BSDs, and let those people to
judge, which release version they want to get into their release.  or another
option is to get them involved and ask them for help with maintanance of
release branch and focus on play in HEAD.

> 2) the nature of bugs that have been fixed but are not in a release
we can use priorities in bugtracker, let's say bug with priority lass than five
won't never make it into release branch...

> 3) time
I know what are you talking about...

> On the topic of patching and developing IPFilter, I'd like to propose that
> we set up a web page, say ipfilter.sourceforge.net/changes or something
> else and upload diff files converted to html that have proposed changes
> to review for non-trivial fixes.
how about to commit it into a HEAD? with BugID in comments, everything can be
discussed in BTS, diffs can be seen in webcvs on sourceforge.
in case of a real huge fix separate branch can be created.
how does that sound ?

regards
sasha

Re: [sf-ipf-dev] IPF release cycle model

[Pavel C. <pavel@NetBSD.org>]

On Wed, Nov 21, 2007 at 06:59:09PM +0100, Sa¹a Nedvìdický wrote:
> (2) any change should be recorded in bugtracking system. each change coming to
> HEAD must have a BugID in commit comments. the commit without BugIF should
> be back out. even if change is just code clean up it should be recorded in
> bug tracking system (BTS). The BTS currently runs on sourcerforge.
> This is the only strict rule in fact, but it is very reasonable if there are
> considered more than one single developer participating on project.

IMO, this would lead to really annoying bureaucracy and a high barrier
for changes. Imagine that I notice a typo in a manual page, should I file
a bug for it first?

There are also changes that are not bugfixes.

Pavel

Re: [sf-ipf-dev] IPF release cycle model

[Darren R. <da...@re...>]

Saša Nedvědický wrote:
> Hello,
> ...
>
> (3)	the release branch (v4-1-RELEASE) can be safely assumed as HEAD branch
> 	at these days (bugfixes are committed right there first, the regressions
> 	appears quite often)
>   

No, that's not at all true.
There are new features and changes to HEAD that are not on v4-1-RELEASE.

> (4)	the maintanance releases come out in bursts
>   

There's more than one reason for that...
And mostly the reasons for "bursts" is not enough time
to test/compile/build everywhere.

> So let me allow to dicuss problems stated above in more details. I'll try
> to put some proposals how to solve those problems.
>
> (1) There is currently single committer in IPF project.

That's wrong.
Everyone who's part of the project on sourceforge as a developer, 
including you,
is able to commit to the CVS repository on sourceforge.  Currently the 
only other
person who has is from NetBSD (Martti.)

If you wanted to fix a trivial bug (such as the patch I committed today) 
then the
only thing stopping you is you.


> The only way for those
> people to get involved is to ask Darren to integrate their patch. This way
> blocks development of new (potentially major features), which require more
> significant change (i.e. rpc rules for IPF, better stats, ...)
>
>
> (2) any change should be recorded in bugtracking system. each change coming to
> HEAD must have a BugID in commit comments. the commit without BugIF should
> be back out. even if change is just code clean up it should be recorded in
> bug tracking system (BTS). The BTS currently runs on sourcerforge.
> This is the only strict rule in fact, but it is very reasonable if there are
> considered more than one single developer participating on project.
>   

Before making such rules, you need to understand that there are exceptions.

For example, the HISTORY file and others with the version number are 
updated for every
release.  This requires a committed change in CVS.  It is silly to have 
a bugid for that.
So (2) needs to be waived for release engineering.

> (3) every change must appear in HEAD first, no excuse.

I disagree.  Sometimes a change is only relevant to a release branch.

> (4) the only person who will decide to release new maintanance release (the
> release from RELEASE branch) is RELEASE branch maintainer. He should try to
> establish some regular release cycles - every two months or so. it will help
> users to track when bugfix for bug they eventually reported will be available.
>   

There are three factors that influence when I do releases at present:
1) release cycles of BSDs
2) the nature of bugs that have been fixed but are not in a release
3) time

At the moment, there should be a 4.1.29, and maybe on the weekend
I'll do this, because there have been some important changes for FreeBSD
and it should really have 4.1.29 (and not 4.1.28) as the version number
that will be in FreeBSD 6.3 and FreeBSD 7.0.

So moving on...

On the topic of patching and developing IPFilter, I'd like to propose that
we set up a web page, say ipfilter.sourceforge.net/changes or something
else and upload diff files converted to html that have proposed changes
to review for non-trivial fixes.

Darren

[sf-ipf-dev] IPF release cycle model

[<ane...@gm...>]

Hello,

let me post some non-technical topic within this post. I want to start some
discussion (not flamewar) about IPF release policy/source code maintanance etc.
I see there are those problems in IPF project:
	
(1)	there are no writen rules set up if any one (except Darren) would like
	to submit/comitt his own source to IPF.

(2)	not all changes in CVS are linked to each commit

(3)	the release branch (v4-1-RELEASE) can be safely assumed as HEAD branch
	at these days (bugfixes are committed right there first, the regressions
	appears quite often)

(4)	the maintanance releases come out in bursts

So let me allow to dicuss problems stated above in more details. I'll try
to put some proposals how to solve those problems.

(1) There is currently single committer in IPF project. However there are
definitely more people, who are interested to add some new feature to IPF
or improve IPF or just to help with bugfixing. The only way for those
people to get involved is to ask Darren to integrate their patch. This way
blocks development of new (potentially major features), which require more
significant change (i.e. rpc rules for IPF, better stats, ...)

to change this some approach similar to *BSD models should be used.
there will be a HEAD branch and RELEASE branch. Any IPF developer can commit
to HEAD branch (bugfixes, features, clean ups, anything goes to HEAD first).
the RELEASE branch maintainer takes patches from HEAD and applies them to
release. the only man, who can commit to release branch is RELEASE branch
keeper (maintainer) no one else is allowed to do so.

the only rule should be applied to HEAD: the HEAD must compile at any time,
the commit, which breaks a build on any supported platform must be fixed ASAP
or back out.

(2) any change should be recorded in bugtracking system. each change coming to
HEAD must have a BugID in commit comments. the commit without BugIF should
be back out. even if change is just code clean up it should be recorded in
bug tracking system (BTS). The BTS currently runs on sourcerforge.
This is the only strict rule in fact, but it is very reasonable if there are
considered more than one single developer participating on project.

(3) every change must appear in HEAD first, no excuse. Only release branch
maintainer will decide when the fix will be also integrated into release
branch. I have no idea it is possible to clone bugs in bug tracker at
sourcerforge but the workflow example in terms of bugfix can be as follows:

	user reports bug -> BugID

	developer fixes bug and commits change to HEAD. once fix is commited
	and works developer will clone and closes the original BugID reported
	by user.  BugID -> ClnBugID
	(disclaimer - I have no idea whether it is possible with BTS at sourceforge,
	the idea is to have two related bugs - one for HEAD, the second one
	for RELASE branch)

	RELEASE branch maintainer should consider whether the bug is 'critical'
	enough to let the fix go into release. if it is critical/serious bug,
	it should be integrated there and ClnBugID should be closed, if the bug
	is not serious enough to let fix to appear in RELEASE branch, the ClnBugID
	should be closed as Won'tFix.

don't know if it is possible to set up such workflow with tools provided by
sourcefroge, but we should try to achieve the best we can.	

(4) the only person who will decide to release new maintanance release (the
release from RELEASE branch) is RELEASE branch maintainer. He should try to
establish some regular release cycles - every two months or so. it will help
users to track when bugfix for bug they eventually reported will be available.

so this just a topic to provoke a discussion about IPF project in more general
level.

regards
sasha

[sf-ipf-dev] Adding in destination lists for IPFilter 5

[Darren R. <da...@re...>]

Over the weekend, I've been working on improving the format of 
ippool.conf so
that it is more readable as well as updating the implementation in the 
kernel to be
more OO.  ippool will continue to read the old format.

The new format will look like this:
pool ipf/in dst-list(name john; round-robin;) { bge0:1.2.2.3; hme0:2.2.2.2;}

pool ipf/out tree(name mark;) {;};

pool nat/in hash(name mary; size 101; seed 55;) {;};

pool ipf/in group-map(name peter;) {;};

pool nat/out dst-list ( name venus; weighted bytes;) { 1.2.3.4; 5.6.7.8; };

The intention is to allow rules to be written like:

pass in on foo0 to dst-list/john from foo to bar
rdr bar0 201.1.1.1 port 80 -> dst-list/venus port 80 tcp

For the existing code, this means moving a bunch of logic out of ip_lookup.c
(this becomes more of an accessor interface) and into ip_htable.c and 
ip_pool.c
which now export only an ipf_lookup_t structure that contains a bunch of 
function
pointers and a type field to identify which module it is.

There will be a new ip_dstlist.c to provide the new pool style above.

Another change that I'm currentuy exploring is changing some use of the IP
protocol version (4 or 6) to be address family (ie AF_INET, etc) so that 
IPFilter
can be more easily adapted to other protocol families.

When I've got some more of this done and a little bit of preliminary testing
done, I'll post a reference to a patch for people to ry.

Cheers,
Darren