Re: [modauthkerb] Keytab not readable? Can't authenticate workstation for SSO

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

On 10/30/2013 10:58 PM, Jim Fisk wrote:
> Hi Douglas,
>
> Thanks for looking at this issue as well!  I've gone through and added responses below inline.
>
>
> On Wed, Oct 30, 2013 at 10:22 AM, Douglas E. Engert <dee...@an... <mailto:dee...@an...>> wrote:
>
>
>
>     On 10/29/2013 10:26 PM, Jim Fisk wrote:
>      > I have a setup with an Active Directory KDC, Windows 7 client workstations, and a Linux server (CentOS and Apache) outside the network with which I am trying to configure single sign on
>     functionality.
>      >   I'm having trouble getting the handshake to work between the client workstation and the Apache webserver.  When I go to https://cname.mysite.com/user/login/sso, I get a 500 internal server
>     error.  I
>      > configured Firefox by going to "about:config" and adding "cname.mysite.com <http://cname.mysite.com> <http://cname.mysite.com>" to "network.negotiate-auth.trusted-uris" and
>     "network.negotiate-auth.delegation-uris".
>
>     Minor point, usually network.negotiate-auth.delegation-uris is not set for a web server,
>     as it says it is OK to delegate credentials.
>
> Makes sense, thanks for letting me know.  I noticed that the "network.negotiate-auth.delegation-uris" option wasn't doing much in terms of changing results.  I'll leave it blank going forward.
>
>
>      > I figured
>      > I would have to configure the browser with the A Record instead of the CName, but when I do this I just get a 401 authorization required error and not much else happens (same as if I didn't
>     configure
>      > browser for kerberos at all).  When the browser is configured with the CName, I get the 500 error and couple of interesting things happen:
>      >
>      > 1) If I'm on a workstation within the network, I get a ticket for the service that appears to be correct.  A klist looks like this:
>      >
>      > Cached Ticket (#2 of 7)
>      > Client: act...@EX... <mailto:act...@EX...> <mailto:act...@EX... <mailto:act...@EX...>>
>      > Server: HTTP/are...@EX... <mailto:are...@EX...> <mailto:are...@EX... <mailto:are...@EX...>>
>      > KerbTicket Encryption Type: RSADSI RC4-HMAC(NT)
>      > Ticket Flags 0x40a00000 -> forwardable renewable pre_authent
>      > Start Time: 10/24/2013 15:16:14 (local)
>      > End Time: 10/25/2013 0:11:33 (local)
>      > Renew Time: 10/31/2013 14:11:33 (local)
>      > Session Key Type: RSADSI RC4-HMAC(NT)
>      >
>      > 2) I see the following in the Apache error logs:
>      >
>      > "GSS-API major_status:000d0000, minor_status:000186a4" which I understand to simply mean that Apache can't read the keytab file.
>      >
>      > "gss_acquire_cred() failed: Unspecified GSS failure.  Minor code may provide more information (, )" which could mean the keytab has the wrong key version #, or machine password.  It could also be a
>      > problem with the ticket cache on the workstation.
>      >
>      > I've tried to make it so Apache can read the keytab file by changing its ownership to the user that owns the httpd Apache process (chown root) and making sure that it is readable by that user
>     (chmod
>      > 400). However when I do this, the command to test if the keytab works on the webserver (kinit -k -t /etc/krb5.keytab HTTP/arecord.mysite.com <http://arecord.mysite.com>
>     <http://arecord.mysite.com>) no longer finishes without
>      > error.  Instead I receive "kinit: Generic preauthentication failure while getting initial credentials".
>      >
>      > <b>Here is some additional info about our setup:</b>
>      >
>      > If I do "kinit active_directory_user" from the webserver, I get prompted for a password and when entered correctly it completes without error.  I can then use "klist" to view what appears to be a
>      > proper ticket granting ticket (service principal = krbtgt/EXA...@EX... <mailto:EXA...@EX...> <mailto:EXA...@EX... <mailto:EXA...@EX...>>).
>      >
>      > I can then do "kvno HTTP/arecord.mysite.com <http://arecord.mysite.com> <http://arecord.mysite.com>" and I receive "HTTP/are...@EX... <mailto:are...@EX...>
>     <mailto:are...@EX... <mailto:are...@EX...>>: kvno = 6".  I then check this
>      > against the keytab via "klist -k" and the kvno and principal match exactly.  This also caches a service ticket so "klist" reveals a second ticket with HTTP/are...@EX...
>     <mailto:are...@EX...>
>      > <mailto:are...@EX... <mailto:are...@EX...>> as the service principal.
>      >
>      > Finally I test if the keytab works on the server using: "kinit -k -t /etc/krb5.keytab HTTP/arecord.mysite.com <http://arecord.mysite.com> <http://arecord.mysite.com>" and it completes without
>     error (unless the keytab user is
>      > changed to root as mentioned above).
>      >
>      > Here is the configuration of our /etc/krb5.conf file (I recently stripped this down in hopes to fix the issue):
>      >
>      > [libdefaults]
>      >   default_realm = EXAMPLE.ORG <http://EXAMPLE.ORG> <http://EXAMPLE.ORG>
>      >
>      > [domain_realm]
>      > arecord.mysite.com <http://arecord.mysite.com> <http://arecord.mysite.com> = EXAMPLE.ORG <http://EXAMPLE.ORG> <http://EXAMPLE.ORG>
>      >
>      > [realms]
>      > EXAMPLE.ORG <http://EXAMPLE.ORG> <http://EXAMPLE.ORG> = {
>      >    admin_server = ip address of dc/kdc
>      >    kdc = ip address of dc/kdc
>      >   }
>      >
>      > Our webserver uses virtual hosts to host multiple websites.  So the httpd.conf file uses "include" to reference to another site specific .conf file where I added this mod_auth_kerb logic:
>      >
>      >   LoadModule auth_kerb_module /path/to/modules/mod_auth_kerb.so
>      >   <Location /user/login/sso>
>      >      AuthType Kerberos
>      >      KrbAuthRealms EXAMPLE.ORG <http://EXAMPLE.ORG> <http://EXAMPLE.ORG>
>      >      KrbMethodNegotiate on
>      >      KrbMethodK5Passwd off
>      >      KrbServiceName HTTP/are...@EX... <mailto:are...@EX...> <mailto:are...@EX... <mailto:are...@EX...>>
>
>        The KrbServiceName is the "service" part of a principal, name, so would only be HTTP
>        not the full principal name. The default is HTTP and should work.
>        It could also be "Any" to allow the user of any entry in the keytab.
>          See: http://comments.gmane.org/gmane.comp.apache.mod-auth-kerb.general/2465
>        (This is most likely your problem.)
>
>
> Thanks for sending that post. When Henry says: "Also make sure that Apache has its own keytab file and isn't using the system default one." Does that just mean point to the keytab file exported from
> Active Directory in the Apache .conf file?
>
> I've tried updating the .conf file as you describe:
> When I change KrbServiceName to just HTTP:
>
> 1) Browser error is 500 Internal Server
>
> 2) Apache error logs:
>
> [info] Subsequent (No.2) HTTPS request received for child 1 (server arecord.mysite.com:80 <http://arecord.mysite.com:80>)
>
> [debug] src/mod_auth_kerb.c(1628): [client <ip address>] kerb_authenticate_user entered with user (NULL) and auth_type Kerberos
>
> [debug] src/mod_auth_kerb.c(1240): [client <ip address>] Acquiring creds for HT...@ar...

This is normal. GSS-API uses the term <service>@<instance>. When Kerberos is the underlying GSS mechanisum
this is then translated into a Kerberos Principal: <service>/<instance>@<realm>  where realm is determined by
the Kkerberos library, and kerb5.conf.

>
> [debug] src/mod_auth_kerb.c(1101): [client <ip address>] GSS-API major_status:000d0000, minor_status:000186a4

  Same error message, in Hex that is 10004
>
> [error] [client <ip address>] gss_acquire_cred() failed: Unspecified GSS failure.  Minor code may provide more information (, )
>
> [debug] ssl_engine_kernel.c(1889): OpenSSL: Write: SSL negotiation finished successfully
>
> The only difference I see between this and using "HTTP/arecord.mysite.com  as the KrbServiceName in the .conf file is that it logs "Acquiring creds for
> HT...@ar..." instead of "Acquiring creds for HTTP/arecord.mysite.com"  -  the difference being "@" vs "/"
>
>
> When I change KrbServiceName to Any:
>
> 1) Browser error is 401 Authorization Required
>
> 2) Apache error logs:
>
> [info] Subsequent (No.2) HTTPS request received for child 0 (server arecord.mysite.com:80 <http://arecord.mysite.com:80>)
>
> [debug] src/mod_auth_kerb.c(1628): [client <ip address>] kerb_authenticate_user entered with user (NULL) and auth_type Kerberos
>
> [debug] src/mod_auth_kerb.c(1385): [client <ip address>] Verifying client data using KRB5 GSS-API
> [debug] src/mod_auth_kerb.c(1401): [client <ip address>] Client didn't delegate us their credential
>
> [debug] src/mod_auth_kerb.c(1420): [client <ip address>] GSS-API token of length 9 bytes will be sent back
>
> [debug] src/mod_auth_kerb.c(1101): [client <ip address>] GSS-API major_status:000d0000, minor_status:000186a1

Same error message, still can not access the keytab file.

>
> [error] [client <ip address>] gss_accept_sec_context() failed: Unspecified GSS failure.  Minor code may provide more information (, )
>
>
> According to this: http://blog.stefan-macke.com/2011/04/19/single-sign-on-with-kerberos-using-debian-and-windows-server-2008-r2/, up until the part about the major_status / minor_status error, it
> showed signs of a successful login.  It seems strange that "Client didn't delegate us their credential" is part of a successful login process.

That is not required. User's should be very careful about what servers they delegate credentials.
The delegated credential is a full TGT for the user, in effect giving the server the user's identity
and ability to authneticate ast the user to other servers.

>
>      >      Krb5Keytab /etc/krb5.keytab
>
>     Usually you would have a separate keytab for the apache server, readable only by the apache server.
>     The above is used by the system, for the host/<fqdn>@<realm> keys.
>     (If the keytab is world readable, Kerberos will not use it.)
>
> I'm trying to understand the separate keytab for Apache.  Should there have been a default keytab at /etc/krb5.keytab before I exported one from the KDC and uploaded it?  Is the separate keytab for
> apache just a duplicate of the exact same keytab but in a different location?

The /etc/krb5.keytab has the host keys, used for Kerberos login at the console, or for ssh, telnet, ftp, rlogin...
or any other "login" service where you use Kerberos authentication. See: pam_krb5

(PuTTY 6.1 and above is a Windows SSH client that can do GSS-API authentication to a SSHD on linux.)
If the user logins to Windows using AD, PuTTY can use the underlying Kerberos, much like the browser does.
The Windows SSPI is GSS_API compatible.)

An /etc/krb5.keytab.apache would have the HTTP keys, to allow a user to authenticate to the web server.
Thats a lot different then a user login to the OS.

So yes they should be separate keytabs.

With AD as the KDC, the /mapuser account used for the host principal
*SHOULD* be different from the /mapuser account used for the HTTP principal.
So they each have a different password, and thus different keys.

>
> This seems to support separate keytabs: http://www.microhowto.info/howto/add_a_host_or_service_principal_to_a_keytab_using_mit_kerberos.html, but I'm still trying to wrap my head around the concept.
> Also good to know that world readable keytabs won't be used by Kerberos.
>
>      >      require valid-user
>      >    </Location>
>      >
>      > I worked with the IT staff that maintains the Active Directory Domain Controller (the KDC in this case) to create a keytab file using ktpass:
>      >
>      > <code>
>      > ktpass /pass <password for account> /mapuser <username for account> /out c:\location\of\file\output\krb5.keytab /princ HTTP/are...@EX...
>     <mailto:are...@EX...> <mailto:are...@EX... <mailto:are...@EX...>>
>      > /ptype KRB5_NT_PRINCIPAL /crypto RC4-HMAC-NT /Target EXAMPLE.ORG <http://EXAMPLE.ORG> <http://EXAMPLE.ORG>
>      > </code>
>
>     The /mapuser username is a "user" type account (but not an actual user)
>     which has a UPN, and SPNs. Even though it has multiple SPN, it has only one password that is used to generate
>     tickets on the fly. If you change the password, you will need to update the keytab. (I don't think this your problem.)
>     So there is usually a one-to-one between the /mapuser and a keytab.
>
> When creating the keytab, we referenced a username of an actual user from Active Directory.

This is were most people get confused.
The AD account is a "user" *TYPE* account, but not an actual user. The account represents the host or the web server.
Windows machines are joined to AD and uses a machine account, and from a kerberos prospective they both
a userPrincipalName, and multiple ServicePrincipalNames.

For AD account as refered to by ktpass /mapuser  come up with a naming convention, and location
in AD. The names need to unique across the forest. something like simpleHostNname-service
or in your case: arecord-host for the host entry that endup in /etc/krb5.keytab
and arecord-HTTP that end up in the /etc/krb5.keytab.apache

  This user isn't used for anything besides this and for service account binding via LDAP.

You could have a separate AD account for the service account binding via LDAP.

  The user's
> password should not be changing.  Should we not have created a keytab with the username of an actual AD user?  Can you tell me more detail about what this user type account is and how to create one?

This is from 1/2000, but is has the basics of Windows and uniux Kerberos interoperability:

  http://technet.microsoft.com/library/Bb742433

See section: "Support for Kerberos Services"

>     Also Google for msktutil
>
>
>   I was looking at this as an alternative to using ktpass to create the keytab.  If it's decided that the keytab is malformed, I'm definitely trying this.  I've read that transferring the keytab can
> sometimes introduce problems, so a more direct method like this is a great idea.
>
>
>      >
>      > I also asked them to use "setspn -q HTTP/arecord.mysite.com <http://arecord.mysite.com> <http://arecord.mysite.com>" on the Active Directory DC to check for duplicate SPNs and it returns
>     "Existing SPN Found!" which I believe
>      > means that it is okay.  There are two SPN's listed however: HTTP/are...@EX... <mailto:are...@EX...> <mailto:are...@EX...
>     <mailto:are...@EX...>> and HTTPS/are...@EX... <mailto:are...@EX...>
>      > <mailto:are...@EX... <mailto:are...@EX...>> because the first time we created the keytab, we didn't realize the host (HTTP in this case) referred to a
>     service class that encompasses both the HTTP and
>      > HTTPS protocols.
>
>      >
>      > Things that are done:
>      > -synched clocks between AD server and Apache server via NTP (Luigi The Cat's post was helpful:
>      > http://community.spiceworks.com/topic/143891-possible-to-synchronize-ntp-on-a-linux-server-to-a-windows-domain-controller)
>      > -had our IP provider set up a PTR record that points the webserver's IP to the A Record for the site.
>      >
>      > Things I've tried:
>      > -moving the keytab record higher up in the directory structure
>      > -reinstalling mod_auth_kerb from source: http://sourceforge.net/projects/modauthkerb/files/ and according to instructions: http://modauthkerb.sourceforge.net/install.html
>      > -increasing the permissions for keytab file
>      > -disabling SElinux
>      >
>      > My setup closely resembles this Drupal issue (We're using Drupal as well): https://drupal.org/node/1777528#comment-form
>      >
>      > I have been following advice from these guides:
>      > http://www.grolmsnet.de/kerbtut/
>      > http://acksyn.org/?p=460
>      >
>      > These have been helpful for debugging errors:
>      > http://publib.boulder.ibm.com/infocenter/tivihelp/v2r1/index.jsp?topic=%2Fcom.ibm.itame.doc_6.0%2Fam60_problem95.htm
>      > http://blog.stefan-macke.com/2011/04/19/single-sign-on-with-kerberos-using-debian-and-windows-server-2008-r2/
>      >
>      > Any advice is appreciated, please let me know if I need to provide additional information or test anything.  Thanks so much!
>
>     You appear to have obfuscated many of the names. So there may be some issues with REALM  name != AD DOMAIN name.
>
> Yes most of the names have been changed, but I tried to be as consistent and descriptive as possible to show what these names were representing.  I honestly thought the Realm and the AD Domain were
> synonymous, so thanks for pointing this out.

more or less. Kerberos principals are case sensitive AD is not. T Kerberos realm name is upper case.
So use the AD domain name as the Kerberos realm, but use it as upper case.)

  I think I'm using the realm name correctly according to this: http://technet.microsoft.com/en-us/library/cc731342(v=ws.10).aspx.  Is there way you'd like
> me to check or verify this?
>
>     Wireshark on the client would be helpful as it will show the unencrypted parts of the Kerberos protocols,
>     include kvno and principal names.
>
> I tried doing a tcpdump from the server and hitting https:cname.mysite.com/user/login/sso <http://cname.mysite.com/user/login/sso> while it was running.  Then sending the tcpdump file it created to my
> local machine via SCP and opening with Wireshark to analyze it.  However, I wasn't really seeing the HTTP requests that I was looking for.  This may be attributed to A) my lack of experience with
> Wireshark B) dropped packets, but I did try 3+ exports C) could the requests be hitting errors before helpful info is being captured?
>
> This post had some helpful info about what to look for with Wireshark: http://comments.gmane.org/gmane.comp.apache.mod-auth-kerb.general/2642
>
> Great beginners guide to tcpdump / opening in Wireshark: https://workaround.org/using-tcpdump-and-wireshark
>
> When I'm back onsite I will try downloading Wireshark onto a windows 7 workstation within the network to look at the live traffic between it and the webserver.
>
> Thanks so much for all your help Douglas! You've definitely gotten me thinking in new ways to try to solve this!
>
>      >
>      >
>      > ------------------------------------------------------------------------------
>      > Android is increasing in popularity, but the open development platform that
>      > developers love is also attractive to malware creators. Download this white
>      > paper to learn more about secure code signing practices that can help keep
>      > Android apps secure.
>      > http://pubads.g.doubleclick.net/gampad/clk?id=65839951&iu=/4140/ostg.clktrk
>      >
>      >
>      >
>      > _______________________________________________
>      > modauthkerb-help mailing list
>      > mod...@li... <mailto:mod...@li...>
>      > https://lists.sourceforge.net/lists/listinfo/modauthkerb-help
>      >
>
>     --
>
>        Douglas E. Engert  <DEE...@an... <mailto:DEE...@an...>>
>        Argonne National Laboratory
>        9700 South Cass Avenue
>        Argonne, Illinois  60439
>     (630) 252-5444 <tel:%28630%29%20252-5444>
>
>     ------------------------------------------------------------------------------
>     Android is increasing in popularity, but the open development platform that
>     developers love is also attractive to malware creators. Download this white
>     paper to learn more about secure code signing practices that can help keep
>     Android apps secure.
>     http://pubads.g.doubleclick.net/gampad/clk?id=65839951&iu=/4140/ostg.clktrk
>     _______________________________________________
>     modauthkerb-help mailing list
>     mod...@li... <mailto:mod...@li...>
>     https://lists.sourceforge.net/lists/listinfo/modauthkerb-help
>
>

-- 

  Douglas E. Engert  <DEE...@an...>
  Argonne National Laboratory
  9700 South Cass Avenue
  Argonne, Illinois  60439
  (630) 252-5444