[Shorewall-users] Small HOWTO: Setting up an IP accounting system with Shorewall

[Gonzalo S. <go...@li...>]

Hi All

I came to work on a Monday morning (gotta love Mondays...) with a nice
e-mail from my boss asking me to setup an IP Accounting system so he can
find out how much inbound/outbound data each workstation does on an
hourly/daily/monthly basis. We only get so many GB's per month of
inbound data free so this information would help him find who's ass to
kick. :) Anyway, I decided to write this little HOWTO on how I did it.
The distribution I used was Gentoo Linux v1.2 (http://www.gentoo.org)
and Shorewall v1.3.3.

The following pre-requisites had to be met:

1. Management must be able to see IP accounting statistics
(web/email/whatever)
2. Stats must be updated frequently (every few minutes if possible)
3. Stats must be accounted for on a per-workstation basis (accounting
for the entire subnet is not suffice)
4. No machine on the network is to have Internet access unless they are
part of the accounting system

In order for management to be able to keep track of stats, I thought the
best way of doing this was to provide the stats over http. ipac-ng was
an option, but it looked difficult to implement with shorewall.
Another package recommended to me was BW-ACCT
(http://www.freshmeat.net/projects/bw-acct).
The screenshot looked good, so I installed MySQL and followed the
BW-ACCT INSTALL file instructions.

It was time to setup the firewall rules to account for traffic.

Also, FWIW, in my setup: eth0 =3D external, eth1 =3D DMZ, eth2 =3D Internal=
.

I created a new file, /etc/shorewall/ipacct, with the following script
in it:

---------------------- cut ----------------------
#################################
#        IP Accounting          #
#################################
#
# Create chains
createchain ipacct no

in=3D"-i"
out=3D"-o"

# Files containing subnets to account for
subnets=3D`find_file ipacct.subnets`

# Inbound
#
# Internet to LAN
run_iptables -I net2loc -j ipacct
# Internet to DMZ
run_iptables -I net2dmz -j ipacct
# Internet to FW
run_iptables -I net2fw -j ipacct

# Outbound
#
# LAN to Internet
run_iptables -I loc2net -j ipacct
# DMZ to Internet
run_iptables -I dmz2net -j ipacct
# FW to Internet
run_iptables -I fw2net -j ipacct


#########
# Rules #
#########

# The following inserts rules into iptables to account
# the total inbound/outbound data for each subnet
#
# the file "/etc/shorewall/ipacct.subnets" contains
# each subnet, one per line. ie:
#
# 192.168.1.0/24
# 203.3.3.3/24
#
for sn in `cat $subnets`; do
        # Create the chain for each subnet
        iptables -N "$sn"
        # Add inbound/outbound rules to each chain
        iptables -A $sn ${in} eth0 -d $sn
        iptables -A $sn ${out} eth0 -s $sn
        # Add rules into the ipacct chain to link to each subnets chain
        iptables -A ipacct ${in} eth0 -d $sn -j $sn
        iptables -A ipacct ${out} eth0 -s $sn -j $sn
done

# Hosts
masqfile=3D`find_file masq`

if [ -f $masqfile ]; then
        accthosts=3D`cat $masqfile|egrep -v "^#|^$"|awk -F' ' '{print
$2}'`
        for accthost in $accthosts; do
                # Create the chain for each subnet
                iptables -N "$accthost"
                # Add inbound/outbound rules to each chain
                iptables -A $accthost ${in} eth0 -d $accthost
                iptables -A $accthost ${out} eth0 -s $accthost
                # Add rules into the ipacct chain to link to each
subnets chain
                iptables -A ipacct ${in} eth0 -d $accthost -j $accthost
                iptables -A ipacct ${out} eth0 -s $accthost -j $accthost
        done
fi

#
# Extra Hosts to account for (space separated) not listed=20
# in the /etc/shorewall/masq file
#
extrahosts=3D"192.168.1.23"
for accthost in $extrahosts; do
        # Create the chain for each subnet
        iptables -N "$accthost"
        # Add inbound/outbound rules to each chain
        iptables -A $accthost ${in} eth0 -d $accthost
        iptables -A $accthost ${out} eth0 -s $accthost
        # Add rules into the ipacct chain to link to each subnets chain
        iptables -A ipacct ${in} eth0 -d $accthost -j $accthost
        iptables -A ipacct ${out} eth0 -s $accthost -j $accthost
done

# DMZ Hosts

dmzhosts=3D"203.3.3.3 203.3.3.4 203.3.3.5"

for dmzhost in $dmzhosts; do
        # Create the chain for each subnet
        iptables -N "$dmzhost"
        # Add inbound/outbound rules to each chain
        iptables -A $dmzhost ${in} eth0 -d $dmzhost
        iptables -A $dmzhost ${out} eth0 -s $dmzhost
        # Add rules into the ipacct chain to link to each subnets chain
        iptables -A ipacct ${in} eth0 -d $dmzhost -j $dmzhost
        iptables -A ipacct ${out} eth0 -s $dmzhost -j $dmzhost
done
---------------------- cut ----------------------

So, basically, in my setup I list specifically which hosts are to be
masqueraded in /etc/shorewall/masq. This means that if I add a new
workstation, I just add it to the /etc/shorewall/masq file, restart
shorewall, add the workstation to the admin interface for BWACCT and
that's about it. I guess I could modify BWACCT to do this for me when I
add a new host but I haven't gotten that far yet :-)

Finally, I made the following changes to the /etc/init.d/shorewall
script to call the 'ipacct' script above:

---------------------- cut ----------------------
+########################################################################
+# IP Accounting                                                       #
+#######################################################################
+
+add_ipacct_rules() {
+       ipacctfile=3D`find_file ipacct`
+       if [ -f $ipacctfile ]; then
+               . $ipacctfile
+       fi
+}
+
+write_ipacct_stats() {
+       echo "Writing IP Accounting stats ..."
+       /home/httpd/htdocs/iptables-stats.pl 2>&1 >/dev/null
+}
+

+    echo "Adding IP Accounting rules ..."
+
+    add_ipacct_rules


     restart)
+       write_ipacct_stats

---------------------- cut ----------------------

The "+" indicates a line added by me. Basically during the start/restart
of shorewall (under the define_firewall section) it just adds the IP
Accounting rules (add_ipacct_rules). Under the restart section, before
it wipes all the rules & chains it runs the write_ipacct_stats section
to write the stats to the MySQL database before clearing them.

Anyway, I hope this helps someone. Tom, thank you for your great work!
:-)

Regards,

Gonzalo