[Ipsec-tools-users] Documentation for PlainRSA Authentication

[Simon C. <sim...@gm...>]

Hi there,

I am new to this list and had started using racoon this past year at
my house as part of VPN over wireless.  In working with
racoon/ipsec-tools, however, I have noticed that there is very scant
documentation on how to do plain RSA authentication (there is adequate
amount of docs on pre-shared keys).  After consulting with Michal
Ludvig, the developer who initially coded in RSA support for version
0.4, I decided to write this documentation to help those people who
might be having similar issues as I did and to help out with the
project as a whole.

I have two requests for you, the list readers:

1)  Please let me know how I can improve on this documentation.  This
doc is based on my personal experience in setting up my own gears at
home, and I am not immune to "mental blind spots".  So please let me
know of anything I can do better.

2)  This write-up talks about a few important options that are missing
from the version 0.6.6 manpages for racoon.conf.  If someone could let
me know how I can submit a more formal correction to the
documentations, I would really appreciate it.

Without further ado, here is my "how-to" for plain RSA authentication.

=======================================

HOW-TO

I wrote this documentation in response to the fact that there is only
one sample file in the samples directory that gives any hint as to the
proper way of setting up ipsec-tools to use plain RSA keys for
authentication.  I had to email Michal Ludvig himself for assistance
(Michal is the original developer who coded in support for plain RSA
authentication).  Additionally, the manpage for racoon.conf itself is
missing a few necessary options.  So here is a step-by-step
description of the process.

Before you begin, you should understand that the RSA authentication
mechanism hinges upon the idea of a split cryptographic key:  one used
by the public, the other readable only to you.  Any data that is
encrypted by a public key can be decrypted only by the corresponding
private key, so that the private key user can be assured that the
content of the transmission has not been examined by unauthorized
parties.  Similarly, any data encrypted by the private key can be
decrypted by the public key so that the public knows that this
transmission came from this user and nobody else (this idea is called
non-repudiation).  Also, the longer the key length, the more difficult
it would be for potential attacker to conduct brute-force discovery of
the keys.  So, what all this means for the security administrator is
that the setup needs a pair of reasonably long keys for each host that
wishes to authenticate in this manner.

With this in mind, it should be relatively straightforward to set up
RSA authentication.  For the purpose of this document, we assume that
we are setting up RSA authentication between two networked hosts
called Boston and Chicago.  Unless otherwise noted, all steps should
be performed on both hosts with corresponding key names.  Here are the
steps:

1)  Included in each default installation of ipsec-tools is a binary
called plainrsa-gen.  This executable is used to generate a pair of
RSA keys for the host.  There are only two parameters that you should
be concerned about: -b, which sets the number of bits for the keys,
and -f, which specifies the output file for plainrsa-gen to send the
results.  On an ordinary Pentium-II with 128 MB of RAM, it takes only
seconds to generate keys that are 2048 bits long, and only slightly
longer to generate 4096-bit keys.  Either key length should be
sufficient; any longer key length actually reduces performance and
does not increase security significantly.  You should therefore run it
as:

	/usr/local/ipsec-tools/sbin/plainrsa-gen -b 2048 -f /var/tmp/boston.keys

2)  When the process completes, you should have a text file that
includes both public and private keys.  GUARD THIS FILE CAREFULLY,
because once a private key is compromised it is no longer any good,
and you must generate a new pair from scratch.  Reading the file
itself, you should see several very long lines of alphanumeric data.
The only line you should be concerned with is the line towards the top
of the output file that begins with "# pubkey=0sAQPAmBdT/" or
something to that effect.  This line is your public key, which should
be made available to the other host that you are setting up.  Copy
this line to a separate file called "boston.pub" and change the
beginning of the line so that it reads ": PUB 0sAQPAmBdT/".
Alternatively, you can also grab the first line of the boston.keys
file and uncomment the line so that it reads the same as above.  Now
rename the file you generated initially to "boston.priv".

3)  You should now have two files, boston.priv and boston.pub
(chicago.priv and chicago.pub on Chicago).  The first file contains
your private key and the second file your public key.  Next you should
find a way to get the public key over to the other host involved.
Boston should have (1) its own key pair, and (2) Chicago's public key
ONLY.  Do not copy Chicago's private key over to Boston, because (a)
it is not necessary, and (b) you would now have two potential places
for losing control of your private key.

4)  You should now configure the racoon.conf configuration file for
each host to (a) turn on RSA authentication, and (b) designate each
host's private key and the remote host(s)'s public key(s).  Take all
your keys and place it in one directory and use the global directive
"path certificate" to specify the location of the keys.  This step is
especially important if you are running racoon with privilege
separation, because if racoon cannot find the keys inside the
directory you have just specified it will fail the authentication
process.  So, write the directive like the following:

	path certificate "/usr/local/ipsec-tools/var/racoon" ;

Next, you need to specify the host's own private key and the public
keys of all the remote peers involved.  Now, the manpages for
racoon.conf are missing several options under the Remote Node
Specifications section, all of which are critical to making this setup
work.  For your local private key and remote public key(s), you should
use the following directives:

	certificate_type plain_rsa "/usr/local/ipsec-tools/var/racoon/boston.priv";
	peers_certfile plain_rsa "/usr/local/ipsec-tools/var/racoon/chicago.pub";

Notice the option "plain_rsa" for both directives.

Finally, under the "proposal" statement section, you should specify
the "rsasig" option for "authentication_method".

5)  You have finished configuring the host for RSA authentication.
Now use racoonctl to reload the configuration or simply restart the
machine and you should be all set.

TROUBLESHOOTING

In the event that the hosts fail to communicate, first go back to the
instructions above and make sure that:
1)  You have placed all the keys in the directory that is specified by
the "path certificate" directive.  Keep in mind that privilege
separation will force racoon to look into that directory and nowhere
else.
2)  You have specified correctly the host's own private key and the
remote peer's public key.
3)  You have specified the "rsasig" method for authentication in the
proposal statement.

If you run into any further problems, you should try to use "racoon
-v" to debug the setup, and send a copy of the debug messages to the
mailing list so that we can help you determine what the problem is.

If you have any questions or comments about this document, email
sim...@gm....

-Last modified:  November 30, 2006