[ISCS-devel] ISCS/PEP DNRead.comments,NONE,1.1 DNRead,1.7,1.8

[John A. S. I. <jsu...@us...>]

Update of /cvsroot/iscs/ISCS/PEP
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv5647/PEP

Modified Files:
	DNRead 
Added Files:
	DNRead.comments 
Log Message:
Final Access Group duplicate parsing split between SPM and DNRead

--- NEW FILE: DNRead.comments ---
#!/bin/sh
#
# Copyright (C) 2003 - 2005  John A. Sullivan III
# 
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
# 
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
# 
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.
# 
# e-mail: jsu...@op...

# DNRead is called from X509updown (a FreeS/WAN updown script) and is used to dynamically create
# iptables rules for the Accessor's IP source address based upon the fields of the Accessor's
# DER_ASN.1_DN ID.  It parses it against the file DNList (which must reside in the same directory)
# to determine Access Group membership (and thus access priviledges).  DNList contains a mapping of
# ID fields (including wildcards) to Access Groups.
#
# Revision 0 - January 28, 2003 - John Sullivan
# Revision 1 - August 2, 2003 - John Sullivan - removed jump to ProtectionFilter to conform to new
#      logic which places that jump in the iptables.boot file
# Revision 2 - July 15, 2005 - John Sullivan - completely culling of hierarchical duplicates to accommodate noon-hierarchical chainnames
#
# We need a way to determine the group membership of users attempting a connection to the VPN gateway.  Once we know the correct group, \
# we can then enter their IP address in the ACCESS_GROUP chain during the updown script and jump them to the appropriate chains.  We'll \
# need to confirm it but we should be able to delete the entry when the connection is dropped with the down portion of the updown script.
# We want to keep the footprint small on the enforcement devices so we will need to output the needed information from the database on the \
# Management Console to a text file that can be stored and parsed on the PEP.  We could create a list of groups and parse all groups for matching \
# DER_ASN.1_DN ID's but then we will need to read every group and every DN in those groups.
#
# I believe it is far simpler to create a flat table of all uniquely identified DN's with the groups of which they are a member - a sort of backwards \
# approach, but then we can grep this file for matches.  It should be fast and an extremely small footprint.  This way we only have to parse the known \
# DN's instead of every group and every DN and every instance of a DN living in multiple contexts.
#
# The DN list file should have the syntax "DN" "CA's DN" group1;group2;group 3 . . .  with each entry on a new line, e.g.:
# "C=*,O=Nexus,OU=Eng,CN=*" "C=US,O=Atlas,OU=PKI,CN=AtlasCA" Nexus/Eng
# "C=*,O=Nexus,OU=*,CN=gss" "C=US,O=Atlas,OU=PKI,CN=AtlasCA" Nexus/CM;Nexus/Exec/JVB;Nexus/Acct
# The fields are space delimited.  Since there may be spaces within a DN, the DN must be within quotes.  The groups are ";" delimited because of how \
# we need to parse them for duplicates and hierarchical duplicates before writing the iptables rules
# Actually, we do not need to include the * fields.  We could just not include those fields and the effect is the same but with less processing.
#
# We have a number of issues to work out.  One is efficiency.  If a provided cert happens to qualify as a member of multiple groups within a single \
# hierarchy chain, e,g, Nexus, Nexus/Exec and Nexus/Exec/JVB, it is senseless to make them jump to each group.  The inheritance feature will \
# automatically jump Nexus/Exec/JVB users into the Nexus/Exec chain which will, in turn, automatically jump them to the Nexus chain.  We will \
# need to assign membership with this in mind.  ONE RELATED ISSUE IS HOW THIS EFFICIENCY LOGIC WILL BE IMPACTED IF WE CREATE AN \
# INHERITED RIGHTS MASK
# No, because we no longer use hierarchical chain names, we cannot cull hierarchical duplicates in this script.  The DNList must be culled of \
# hierarchical duplicates by the SPM when it is created.  In fact, although the syntax of the DNList file is not technically changed, it will \
# look different.  It now looks like this:
# "C=*,O=Nexus,OU=Eng,CN=*" "C=US,O=Atlas,OU=PKI,CN=AtlasCA" c12
# "C=*,O=Nexus,OU=*,CN=gss" "C=US,O=Atlas,OU=PKI,CN=AtlasCA" c14;c23;c17
# Oops! That's not going to work either.  We can cull hierarchical duplicates for a single X.509 Accessor definition in the SPM but the SPM \
# has no idea if a particular user's DN is going to match more than one X.509 Accessor definition.  To use the above DNList as an example, \
# a user with DN = C=GB,O=Nexus,OU=Eng,CN=gss would use group memberships from both lines.  Only the PEP will know that so the PEP must also \
# cull regular and hierarchical duplicates.  Since the chainnames themselves contain no hierarchical information, we will need to slightly \
# change our format.  We will include both the Access Group names and the chain names in the last field of DNList.  They will still use a \
# semicolon (";") as a delimiter so they must actually be read in pairs.  That is, each delimited field is dropped into an array.  Thus \
# array[0] might hold "O=Nexus,OU=Eng"(the Access Group) and array[1] might hold "c12" (the chainname).  The DNList syntax thus becomes: \
# "DN" "CA's DN" group1;chainname1;group2;chainname2;group3;chainname3; . . . and the example becomes: \
# "C=*,O=Nexus,OU=Eng,CN=*" "C=US,O=Atlas,OU=PKI,CN=AtlasCA" Nexus/Eng;c12;
# "C=*,O=Nexus,OU=*,CN=gss" "C=US,O=Atlas,OU=PKI,CN=AtlasCA" Nexus;c14;Nexus/Exec;c23;Nexus/Sales;c17;
# 
# We need to know what happens if a DN has more fields than a group defintion.  This won't matter because we will only be testing the fields in the\
# definition
# We need to know what happens if a group definition has more fields than a DN.  This can be a problem.  It should always result in a fail unless \
# the definition sets the field to "*".  In that case, should we fabricate the field or should we not test any fields where the definition is "*"?  The \
# algorithm works fine as is.  If the DN doesn't have a field in the definition, if the definition sets it to "*", it is never tested.  If it doesn't, since \
# the pattern can't be found, it fails and the entire cycle breaks as it should.
#
# We will need to know which potential X.509  fields are used for security.  This can be housed in a separate db table that produces a form for \
# filling out DN definitions.  However, since we are now pulling the fields to check from the definition itself, such a table is not neccessary for \
# this routine.
# Can we use wildcards in the fields (this might force a change from fgrep to grep or egrep)? This is a real problem with the original process.  The \
# problem is that the input string to grep is not globbed.  We are going to have to turn it around and extract each line from the DNList file and use \
# that line as the pattern and the $PLUTO_PEER_ID as the input string.  That makes life much more complicated, especially since we still have to \
# test each field individually as per the multiple instances of the same field described below.  We will not be able to use fgrep.
# Can we use multiple instances of the same field, e.g., multiple OU's? Yes but only if we test each field separately in sequence.  For example, if \
# we do echo C=US,O=Nexus,OU=VPN,OU=Eng,CN=jas | grep C=.*,O=Nexus,OU=Eng,CN=.* , it will fail because \
# O=Nexus,OU=VPN,OU=Eng does not match O=Nexus,OU=Eng.  However, echo OU=VPN,OU=Eng,OU=Admin | grep OU=Eng does match.
# we have an additional problem with grep.  Since it uses RE's, we cannot use the same wildcards as normal shell globbing.  Thus the "*" in the \
# DNList will need to be changed to ".*" and the "?" to ".".  This assumes that we want "Neus" to NOT equal "Ne?us" otherwise we will need to \
# use egrep instead and change "?" to ".?".  NEVER MIND - we can do it faster and easier without grep by using built in bash string pattern matching! \
# However, we still must test each field individually as it is still pattern matching
# For the sake of efficiency, we will do everything we can with bash shell string manipulation rather than calling external programs
#
# PLUTO_PEER_CLIENT="192.168.113.4/32"  # TEST TEST TEST MUST BE REMOVED
#
# PLUTO_PEER_ID="C=US, L=Nexus-PWM, O=Nexus, OU=VPN, OU=Eng, CN=jas"  # TEST TEST TEST MUST BE REMOVED!!!!!!!
if [[ "$1" != [AD] ]] # make sure we've passed the needed parameters from X509updown
then
	echo "Needed action parameter not passed to iptables - aborting connection!!!!!"
	exit 13
fi
DelimitedID="$PLUTO_PEER_ID,"  # We need to add a , on the end or the match test for the last field will not work
#
# We only need one of the below rules rather than one for each group membership so we create them outside the loop
case $1 in
	"A" )
		iptables -I VPN_ALLOW 1 -s "$PLUTO_PEER_CLIENT" -j RETURN
  		;;
	"D" )
		iptables -D VPN_ALLOW -s "$PLUTO_PEER_CLIENT" -j RETURN
		;;
	* )
		echo "Needed action parameter not passed to iptables - aborting connection!!!!!"
		exit 13
		;;
esac
#
# Next we create the ACCESS_GROUP_DENY rules
# This is reading in the DNListDENY a line at a time.  $DN will hold the DN $CA the CA since only DN + CA is unique and GROUP will hold \
# the list of group memberships string.  $AccessGroups will hold the accumulation of all groups that have been found.  We will then have \
# to edit it to eliminate duplicates
#
AccessGroups=""
while read DN CA GROUP
do
	CA=${CA#\"}  # Strip the enclosing quotes
	CA=${CA%\"}
	if [ $PLUTO_PEER_CA != $CA ]; then continue; fi # No need to process if this ID is from a different CA
	DN=${DN#\"}
	DN=${DN%\"}
	match="true"
# I bet we could do this with a for loop on an unquoted string if we set IFS=","!!!!!
	OIFS="$IFS"
	IFS="," # WE MAY NEED TO ADD / AS A DELIMETER SINCE IT IS A LEGAL X.509 FIELD DELIMETER

	for field in $DN  # Since we can't now choose which field to start with and we want to start with the one that will eliminate the most options \
# first, we'll have to have the management platform export the DNList in the desired field order.
	
	do
		if [ "${field##*=}" == "*" ]; then continue; fi  # Don't test if the field is set to *

		if [ "${DelimitedID/"$field",/}" == "${DelimitedID}" ]   # What if $field is a substring, e.g., O=NexusO vs O=Nex? I think we'll need \
# to add a comma at the end which means we have to make sure there is a comma at the end of the $PLUTO_PEER_ID or the last field will always \
# mismatch.  This is all confirmed.  $field will slip through if it is a substring unless we add the comma and we do need to add it to $PLUTO_PEER_ID.
		then
			match="false"
			break
		fi
# If any one field doesn't match, don't waste time, jump to \
# the next group definition.  The logic is that we try find $field in the PEER_ID and delete it.  If we found a match, the new PEER_ID will be \
# different from the old one and this test will fail.  If we couldn't find a match, nothing was deleted, the test succeeds, i.e., there is no match, and \
# we break out of the loop to test the next group definition file

	done
	IFS="$OIFS"
	if [ "$match" != "false" ];then AccessGroups="$GROUP;$AccessGroups";fi  # Build the AccessGroups string if we didn't have a mismatch
done < ./DNListDENY
#
# Now we need to fix the AccessGroups string by stripping duplicates
# I think we can use this logic: 
# Duplicates: if [ ${AccessGroups/"$candidate"} == ${AccessGroups//"$candidate"} ], i.e., removing one instance is the same as removing all \
# instances, then the $field is unique.  If not, we can eliminate all instances and then add $field back to the beginning - Alas, the shell string \
# manipulation did not seem to be up to the task.  It was very complicated to handle the first field or consecutive duplicates.  We wound up \
# dumping all the fields to an array and matching from there.
# Hierarchical duplicates: if the ENTIRETY of $candidate is found within another field both from the beginning (i.e. preceeded by a space) and \
# and followed immediately by an "/", then the original candidate can be discarded as a hierarchical duplicate.
# Hmmm . . . however, this assumes we are testing from shortest to longest! If we are planning to lop off the fields as we pass through the string, \
# rather than store them in an array, this will be a problem!
# Hmm . . . another problem - we can't use spaces as a delimiter.  When we test for duplicates, we are looking for a space to find a field \
# boundary.  That means, if a name has two spaces and the middle word happens to match the name of a group, it will be pulled out of the \
# AccessGroup list.  We need to find a different delimeter for the groups - let's use ";".
#
#
# Now we loop through the group membership to create the rest of the access rules
until [ "$AccessGroups" == "${AccessGroups#;}" ]
do
	AccessGroups="${AccessGroups#;}"  # Eliminate leading ; caused by
# excess newlines in DNList - DNList must have one and only one newline at the
# end
done
OIFS="$IFS"
IFS=";"  # this makes ; an illegal character even if escaped
declare -a group
group=($AccessGroups)  # Create an array to hold the individual elements of the string
while [ "${group[0]}" != "" ]  # Run to the end of the current array (because we may change the array on any pass)
do
#
# The logic will be:
# always start from the bottom of the array
# if we find a duplicate or a hierarchical duplicate (i.e., something directly in line further up the hierarchy), unset the duplicate.
# if we find that the bottom element is a hierarchical duplicate of the element we have encountered while walking the array, we unset \
# the bottom element, slide the array and start again from the bottom
# if the bottom element has made it all the way through this sieve without being unset, make the iptables group assignment and then \
# unset this bottom element and slide the array
# What if we unset the bottom element before we find all the duplicates? Will there be any hierarchical duplicates that will not be found by \
# another element or will it be less efficient? We should be fine on all counts.  Other duplicates will be eliminated by the hierarchical duplicate \
# that caused this unset
#
	count=2
	countmax=${#group[@]}
	while [ $count -lt $countmax ]
	do
		if [ ${group[0]} == ${group[$count]} ]  # If it is a duplicate
		then
			unset group[$count]
			unset group[$((count+1))]
		fi
		let "count+=2"
	done
	if [ "${group[0]}" != "" ] 
	then
		iptables -$1 ACCESS_GROUPS_DENY -s "$PLUTO_PEER_CLIENT" -j "${group[1]}_DENY"  # We will call this from updown with either a A or D
		unset group[0]
		unset group[1]
	fi
	group=(${group[@]})
done
IFS="$OIFS"
#
# Now we set up the ACCESS_GROUPS rules
#
AccessGroups=""
while read DN CA GROUP
do
	CA=${CA#\"}  # Strip the enclosing quotes
	CA=${CA%\"}
	if [ $PLUTO_PEER_CA != $CA ]; then continue; fi # No need to process if this ID is from a different CA
	DN=${DN#\"}
	DN=${DN%\"}  # Strip the enclosing quotes
	match="true"
	OIFS="$IFS"
	IFS="," # WE MAY NEED TO ADD / AS A DELIMETER SINCE IT IS A LEGAL X.509 FIELD DELIMETER

	for field in $DN  
	do
		if [ "${field##*=}" == "*" ]; then continue; fi  # Don't test if the field is set to *

		if [ "${DelimitedID/"$field",/}" == "${DelimitedID}" ]   
		then
			match="false"
			break
		fi
	done
	IFS="$OIFS"
	if [ "$match" != "false" ];then AccessGroups="$GROUP;$AccessGroups";fi  # Build the AccessGroups string if we didn't have a mismatch
done < ./DNList
#
# Now we need to fix the AccessGroups string by stripping literal and hierarchical duplicates
# I think we can use this logic: 
# Hierarchical duplicates: if the ENTIRETY of $candidate is found within another field both from the beginning (i.e. preceeded by a space) and \
# and followed immediately by an "/", then the original candidate can be discarded as a hierarchical duplicate.
until [ "$AccessGroups" == "${AccessGroups#;}" ]
do
	AccessGroups="${AccessGroups#;}"  # Eliminate leading ; caused by
done
OIFS="$IFS"
IFS=";" 
group=($AccessGroups)  
while [ "${group[0]}" != "" ] 
do
	count=2
	countmax=${#group[@]}
	while [ $count -lt $countmax ]
	do
		if [ ${group[0]} == ${group[$count]} ] || [ ${group[0]} != ${group[0]#${group[$count]}\/} ]  # If it is a duplicate or hierarchical dup
		then
			unset group[$count]
			unset group[$((count+1))]
		else
			if [ ${group[$count]} != ${group[$count]#${group[0]}\/} ] #If count is a child of 0, that is 0 is a hierarchical duplicate
			then
				unset group[0]
				unset group[1]
				break
			fi
		fi
		let "count+=2"
	done
	if [ "${group[0]}" != "" ]  # Am I here because I am hierarchically redundant and broke the loop or because I am hierarchically unique
	then
		iptables -$1 ACCESS_GROUPS -s "$PLUTO_PEER_CLIENT" -i ipsec+ -j "${group[1]}"  
		unset group[0]
		unset group[1]
	fi
	group=(${group[@]})
done

Index: DNRead
===================================================================
RCS file: /cvsroot/iscs/ISCS/PEP/DNRead,v
retrieving revision 1.7
retrieving revision 1.8
diff -C2 -d -r1.7 -r1.8
*** DNRead	15 Jul 2005 05:21:08 -0000	1.7
--- DNRead	15 Jul 2005 21:18:03 -0000	1.8
***************
*** 1,4 ****
--- 1,6 ----
  #!/bin/sh
  #
+ # ISCS DEVELOPMENT TEAM:  DO NOT DIRECTLY EDIT THIS FILE.  EDIT DNRead.comments, STRIP THE COMMENTS AND SAVE THE STRIPPED VERSION AS DNRead
+ #
  # Copyright (C) 2003 - 2005  John A. Sullivan III
  # 
***************
*** 17,84 ****
  # Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.
  # 
! # e-mail: jsu...@op...
! 
! # DNRead is called from X509updown (a FreeS/WAN updown script) and is used to dynamically create
! # iptables rules for the Accessor's IP source address based upon the fields of the Accessor's
! # DER_ASN.1_DN ID.  It parses it against the file DNList (which must reside in the same directory)
! # to determine Access Group membership (and thus access priviledges).  DNList contains a mapping of
! # ID fields (including wildcards) to Access Groups.
! #
! # Revision 0 - January 28, 2003 - John Sullivan
! # Revision 1 - August 2, 2003 - John Sullivan - removed jump to ProtectionFilter to conform to new
! #      logic which places that jump in the iptables.boot file
! #
! # We need a way to determine the group membership of users attempting a connection to the VPN gateway.  Once we know the correct group, \
! # we can then enter their IP address in the ACCESS_GROUP chain during the updown script and jump them to the appropriate chains.  We'll \
! # need to confirm it but we should be able to delete the entry when the connection is dropped with the down portion of the updown script.
! # We want to keep the footprint small on the enforcement devices so we will need to output the needed information from the database on the \
! # Management Console to a text file that can be stored and parsed on the PEP.  We could create a list of groups and parse all groups for matching \
! # DER_ASN.1_DN ID's but then we will need to read every group and every DN in those groups.
! #
! # I believe it is far simpler to create a flat table of all uniquely identified DN's with the groups of which they are a member - a sort of backwards \
! # approach, but then we can grep this file for matches.  It should be fast and an extremely small footprint.  This way we only have to parse the known \
! # DN's instead of every group and every DN and every instance of a DN living in multiple contexts.
! #
! # The DN list file should have the syntax "DN" "CA's DN" group1;group2;group 3 . . .  with each entry on a new line, e.g.:
! # "C=*,O=Nexus,OU=Eng,CN=*" "C=US,O=Atlas,OU=PKI,CN=AtlasCA" Nexus/Eng
! # "C=*,O=Nexus,OU=*,CN=gss" "C=US,O=Atlas,OU=PKI,CN=AtlasCA" Nexus/CM;Nexus/Exec/JVB;Nexus/Acct
! # The fields are space delimited.  Since there may be spaces within a DN, the DN must be within quotes.  The groups are ";" delimited because of how \
! # we need to parse them for duplicates and hierarchical duplicates before writing the iptables rules
! # Actually, we do not need to include the * fields.  We could just not include those fields and the effect is the same but with less processing.
! #
! # We have a number of issues to work out.  One is efficiency.  If a provided cert happens to qualify as a member of multiple groups within a single \
! # hierarchy chain, e,g, Nexus, Nexus/Exec and Nexus/Exec/JVB, it is senseless to make them jump to each group.  The inheritance feature will \
! # automatically jump Nexus/Exec/JVB users into the Nexus/Exec chain which will, in turn, automatically jump them to the Nexus chain.  We will \
! # need to assign membership with this in mind.  ONE RELATED ISSUE IS HOW THIS EFFICIENCY LOGIC WILL BE IMPACTED IF WE CREATE AN \
! # INHERITED RIGHTS MASK
! # No, because we no longer use hierarchical chain names, we cannot cull hierarchical duplicates in this script.  The DNList must be culled of \
! # hierarchical duplicates by the SPM when it is created.
! # 
! # We need to know what happens if a DN has more fields than a group defintion.  This won't matter because we will only be testing the fields in the\
! # definition
! # We need to know what happens if a group definition has more fields than a DN.  This can be a problem.  It should always result in a fail unless \
! # the definition sets the field to "*".  In that case, should we fabricate the field or should we not test any fields where the definition is "*"?  The \
! # algorithm works fine as is.  If the DN doesn't have a field in the definition, if the definition sets it to "*", it is never tested.  If it doesn't, since \
! # the pattern can't be found, it fails and the entire cycle breaks as it should.
! #
! # We will need to know which potential X.509  fields are used for security.  This can be housed in a separate db table that produces a form for \
! # filling out DN definitions.  However, since we are now pulling the fields to check from the definition itself, such a table is not neccessary for \
! # this routine.
! # Can we use wildcards in the fields (this might force a change from fgrep to grep or egrep)? This is a real problem with the original process.  The \
! # problem is that the input string to grep is not globbed.  We are going to have to turn it around and extract each line from the DNList file and use \
! # that line as the pattern and the $PLUTO_PEER_ID as the input string.  That makes life much more complicated, especially since we still have to \
! # test each field individually as per the multiple instances of the same field described below.  We will not be able to use fgrep.
! # Can we use multiple instances of the same field, e.g., multiple OU's? Yes but only if we test each field separately in sequence.  For example, if \
! # we do echo C=US,O=Nexus,OU=VPN,OU=Eng,CN=jas | grep C=.*,O=Nexus,OU=Eng,CN=.* , it will fail because \
! # O=Nexus,OU=VPN,OU=Eng does not match O=Nexus,OU=Eng.  However, echo OU=VPN,OU=Eng,OU=Admin | grep OU=Eng does match.
! # we have an additional problem with grep.  Since it uses RE's, we cannot use the same wildcards as normal shell globbing.  Thus the "*" in the \
! # DNList will need to be changed to ".*" and the "?" to ".".  This assumes that we want "Neus" to NOT equal "Ne?us" otherwise we will need to \
! # use egrep instead and change "?" to ".?".  NEVER MIND - we can do it faster and easier without grep by using built in bash string pattern matching! \
! # However, we still must test each field individually as it is still pattern matching
! # For the sake of efficiency, we will do everything we can with bash shell string manipulation rather than calling external programs
! #
! # PLUTO_PEER_CLIENT="192.168.113.4/32"  # TEST TEST TEST MUST BE REMOVED
! #
! # PLUTO_PEER_ID="C=US, L=Nexus-PWM, O=Nexus, OU=VPN, OU=Eng, CN=jas"  # TEST TEST TEST MUST BE REMOVED!!!!!!!
  if [[ "$1" != [AD] ]] # make sure we've passed the needed parameters from X509updown
  then
--- 19,23 ----
  # Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.
  # 
! # e-mail: jsu...@op...
  if [[ "$1" != [AD] ]] # make sure we've passed the needed parameters from X509updown
  then
***************
*** 87,92 ****
  fi
  DelimitedID="$PLUTO_PEER_ID,"  # We need to add a , on the end or the match test for the last field will not work
- #
- # We only need one of the below rules rather than one for each group membership so we create them outside the loop
  case $1 in
  	"A" )
--- 26,29 ----
***************
*** 101,110 ****
  		;;
  esac
- #
- # Next we create the ACCESS_GROUP_DENY rules
- # This is reading in the DNListDENY a line at a time.  $DN will hold the DN $CA the CA since only DN + CA is unique and GROUP will hold \
- # the list of group memberships string.  $AccessGroups will hold the accumulation of all groups that have been found.  We will then have \
- # to edit it to eliminate duplicates
- #
  AccessGroups=""
  while read DN CA GROUP
--- 38,41 ----
***************
*** 116,141 ****
  	DN=${DN%\"}
  	match="true"
- # I bet we could do this with a for loop on an unquoted string if we set IFS=","!!!!!
  	OIFS="$IFS"
  	IFS="," # WE MAY NEED TO ADD / AS A DELIMETER SINCE IT IS A LEGAL X.509 FIELD DELIMETER
  
! 	for field in $DN  # Since we can't now choose which field to start with and we want to start with the one that will eliminate the most options \
! # first, we'll have to have the management platform export the DNList in the desired field order.
  	
  	do
  		if [ "${field##*=}" == "*" ]; then continue; fi  # Don't test if the field is set to *
  
! 		if [ "${DelimitedID/"$field",/}" == "${DelimitedID}" ]   # What if $field is a substring, e.g., O=NexusO vs O=Nex? I think we'll need \
! # to add a comma at the end which means we have to make sure there is a comma at the end of the $PLUTO_PEER_ID or the last field will always \
! # mismatch.  This is all confirmed.  $field will slip through if it is a substring unless we add the comma and we do need to add it to $PLUTO_PEER_ID.
  		then
  			match="false"
  			break
  		fi
- # If any one field doesn't match, don't waste time, jump to \
- # the next group definition.  The logic is that we try find $field in the PEER_ID and delete it.  If we found a match, the new PEER_ID will be \
- # different from the old one and this test will fail.  If we couldn't find a match, nothing was deleted, the test succeeds, i.e., there is no match, and \
- # we break out of the loop to test the next group definition file
- 
  	done
  	IFS="$OIFS"
--- 47,63 ----
  	DN=${DN%\"}
  	match="true"
  	OIFS="$IFS"
  	IFS="," # WE MAY NEED TO ADD / AS A DELIMETER SINCE IT IS A LEGAL X.509 FIELD DELIMETER
  
! 	for field in $DN  
  	
  	do
  		if [ "${field##*=}" == "*" ]; then continue; fi  # Don't test if the field is set to *
  
! 		if [ "${DelimitedID/"$field",/}" == "${DelimitedID}" ]   
  		then
  			match="false"
  			break
  		fi
  	done
  	IFS="$OIFS"
***************
*** 143,186 ****
  done < ./DNListDENY
  #
- # Now we need to fix the AccessGroups string by stripping duplicates
- # I think we can use this logic: 
- # Duplicates: if [ ${AccessGroups/"$candidate"} == ${AccessGroups//"$candidate"} ], i.e., removing one instance is the same as removing all \
- # instances, then the $field is unique.  If not, we can eliminate all instances and then add $field back to the beginning - Alas, the shell string \
- # manipulation did not seem to be up to the task.  It was very complicated to handle the first field or consecutive duplicates.  We wound up \
- # dumping all the fields to an array and matching from there.
- # Hierarchical duplicates: if the ENTIRETY of $candidate is found within another field both from the beginning (i.e. preceeded by a space) and \
- # and followed immediately by an "/", then the original candidate can be discarded as a hierarchical duplicate.
- # Hmmm . . . however, this assumes we are testing from shortest to longest! If we are planning to lop off the fields as we pass through the string, \
- # rather than store them in an array, this will be a problem!
- # Hmm . . . another problem - we can't use spaces as a delimiter.  When we test for duplicates, we are looking for a space to find a field \
- # boundary.  That means, if a name has two spaces and the middle word happens to match the name of a group, it will be pulled out of the \
- # AccessGroup list.  We need to find a different delimeter for the groups - let's use ";".
- #
- #
- # Now we loop through the group membership to create the rest of the access rules
  until [ "$AccessGroups" == "${AccessGroups#;}" ]
  do
! 	AccessGroups="${AccessGroups#;}"  # Eliminate leading ; caused by
! # excess newlines in DNList - DNList must have one and only one newline at the
! # end
  done
  OIFS="$IFS"
! IFS=";"  # this makes ; an illegal character even if escaped
  group=($AccessGroups)  # Create an array to hold the individual elements of the string
! while [ "${group[0]}" != "" ]  # Run to the end of the current array (because we may change the array on any pass)
  do
! #
! # The logic will be:
! # always start from the bottom of the array
! # if we find a duplicate or a hierarchical duplicate (i.e., something directly in line further up the hierarchy), unset the duplicate.
! # if we find that the bottom element is a hierarchical duplicate of the element we have encountered while walking the array, we unset \
! # the bottom element, slide the array and start again from the bottom
! # if the bottom element has made it all the way through this sieve without being unset, make the iptables group assignment and then \
! # unset this bottom element and slide the array
! # What if we unset the bottom element before we find all the duplicates? Will there be any hierarchical duplicates that will not be found by \
! # another element or will it be less efficient? We should be fine on all counts.  Other duplicates will be eliminated by the hierarchical duplicate \
! # that caused this unset
! #
! 	count=1
  	countmax=${#group[@]}
  	while [ $count -lt $countmax ]
--- 65,79 ----
  done < ./DNListDENY
  #
  until [ "$AccessGroups" == "${AccessGroups#;}" ]
  do
! 	AccessGroups="${AccessGroups#;}"  
  done
  OIFS="$IFS"
! IFS=";"  
! declare -a group
  group=($AccessGroups)  # Create an array to hold the individual elements of the string
! while [ "${group[0]}" != "" ] 
  do
! 	count=2
  	countmax=${#group[@]}
  	while [ $count -lt $countmax ]
***************
*** 189,206 ****
  		then
  			unset group[$count]
  		fi
! 		let "count+=1"
  	done
! 	if [ "${group[0]}" != "" ]  # Am I here because I am hierarchically redundant and broke the loop or because I am hierarchically unique
  	then
! 		iptables -$1 ACCESS_GROUPS_DENY -s "$PLUTO_PEER_CLIENT" -j "${group[0]}"  # We will call this from updown with either a A or D
  		unset group[0]
  	fi
  	group=(${group[@]})
  done
  IFS="$OIFS"
- #
- # Now we set up the ACCESS_GROUPS rules
- #
  AccessGroups=""
  while read DN CA GROUP
--- 82,98 ----
  		then
  			unset group[$count]
+ 			unset group[$((count+1))]
  		fi
! 		let "count+=2"
  	done
! 	if [ "${group[0]}" != "" ] 
  	then
! 		iptables -$1 ACCESS_GROUPS_DENY -s "$PLUTO_PEER_CLIENT" -j "${group[1]}_DENY"  # We will call this from updown with either a A or D
  		unset group[0]
+ 		unset group[1]
  	fi
  	group=(${group[@]})
  done
  IFS="$OIFS"
  AccessGroups=""
  while read DN CA GROUP
***************
*** 228,236 ****
  	if [ "$match" != "false" ];then AccessGroups="$GROUP;$AccessGroups";fi  # Build the AccessGroups string if we didn't have a mismatch
  done < ./DNList
- #
- # Now we need to fix the AccessGroups string by stripping literal and hierarchical duplicates
- # I think we can use this logic: 
- # Hierarchical duplicates: if the ENTIRETY of $candidate is found within another field both from the beginning (i.e. preceeded by a space) and \
- # and followed immediately by an "/", then the original candidate can be discarded as a hierarchical duplicate.
  until [ "$AccessGroups" == "${AccessGroups#;}" ]
  do
--- 120,123 ----
***************
*** 242,246 ****
  while [ "${group[0]}" != "" ] 
  do
! 	count=1
  	countmax=${#group[@]}
  	while [ $count -lt $countmax ]
--- 129,133 ----
  while [ "${group[0]}" != "" ] 
  do
! 	count=2
  	countmax=${#group[@]}
  	while [ $count -lt $countmax ]
***************
*** 249,265 ****
  		then
  			unset group[$count]
  		else
  			if [ ${group[$count]} != ${group[$count]#${group[0]}\/} ] #If count is a child of 0, that is 0 is a hierarchical duplicate
  			then
  				unset group[0]
  				break
  			fi
  		fi
! 		let "count+=1"
  	done
  	if [ "${group[0]}" != "" ]  # Am I here because I am hierarchically redundant and broke the loop or because I am hierarchically unique
  	then
! 		iptables -$1 ACCESS_GROUPS -s "$PLUTO_PEER_CLIENT" -i ipsec+ -j "${group[0]}"  
  		unset group[0]
  	fi
  	group=(${group[@]})
--- 136,155 ----
  		then
  			unset group[$count]
+ 			unset group[$((count+1))]
  		else
  			if [ ${group[$count]} != ${group[$count]#${group[0]}\/} ] #If count is a child of 0, that is 0 is a hierarchical duplicate
  			then
  				unset group[0]
+ 				unset group[1]
  				break
  			fi
  		fi
! 		let "count+=2"
  	done
  	if [ "${group[0]}" != "" ]  # Am I here because I am hierarchically redundant and broke the loop or because I am hierarchically unique
  	then
! 		iptables -$1 ACCESS_GROUPS -s "$PLUTO_PEER_CLIENT" -i ipsec+ -j "${group[1]}"  
  		unset group[0]
+ 		unset group[1]
  	fi
  	group=(${group[@]})