DavMail POP/IMAP/SMTP/Caldav to Exchange / Bugs / #534 Kerberos Authentication doesn't seem to be work cross domain

Mickael Guessant - 2013-06-16

Note that the bug status is: Not an Issue ?!?

Anyway, DavMail relies on Java Kerberos implementation => there is not much I can do about this. You should probably try to create a simple test case with Java only code (see KerberosHelper for basic API) to reproduce your issue and submit an OpenJDK bug if needed.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Colin Simpson - 2013-06-18

I'm going to try and get a simpler http request from Java using Kerberos auth going. My Java is a bit rusty. I'll have limited time over the next two weeks but I'll try my best (esp after that). Looks like httpcomponents might be the way to test this.

I had a bit of a try yesterday, one issue that I saw was it complaining about lack of support for RC4 in the Java. Maybe davmail isn't trapping this, but pure speculation at this stage. I'll investigate and provide more feedback.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

You should probably try to make it work with no DavMail dependency, first create a Kerberos LoginContext and call native Http implementation:

Subject.doAs(loginContext.getSubject(), new PrivilegedAction() {

    public Object run() {
        try {
            URL url = new URL("https://testserver.company.com/ews/exchange.asmx");
            InputStream httpInputStream = (InputStream) url.getContent();
            byte[] buffer = new byte[2048];
            int count = httpInputStream.read(buffer);

            System.out.println(new String(buffer, 0, count));
        } catch (MalformedURLException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }
        return "test";
    }
});

Last edit: Mickael Guessant 2013-06-19

Colin Simpson - 2013-07-03

Okay I have managed to create (well largely borrowed from elsewhere) a Java fragment that successfully authenticates with Kerberos to my Exchange CAS boxes, to the same URL that davmail fails to (with Kerberos auth).

Our servers are in a so called resource domain. The output shows my fragment successfully ask for a cross realm ticket and speaks to the resource domains DC's (KDC's) for a service ticket for this and gets one. Later in the output from my script we get the HTML from here (not shown).

Attached is:

RunHttpSpnego.java - My test HTTP (well HTTPS) downloader that uses Kerberos. Excuse my crude Java.

My login.conf is:

com.sun.security.jgss.krb5.initiate {
com.sun.security.auth.module.Krb5LoginModule required
doNotPrompt=false useTicketCache=true ;
};

I'm launching with:

java -Djava.security.krb5.conf=/etc/krb5.conf
-Djava.security.auth.login.config=login.conf
-Djavax.security.auth.useSubjectCredsOnly=false
-Dsun.security.krb5.debug=true
RunHttpSpnego
'https://eu1casa.rd.iongeo.com/owa/#'

I'll post the outputs too.

RunHttpSpnego.java

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Colin Simpson - 2013-07-03

Here was the failure to this host with davmail. Note it doesn't even try to look for KDC's on the rd (resource domain).

davmailkrb2.txt

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Colin Simpson - 2013-07-03

Here is the output of my RunHttpSpnego, note it obtains a cross realm TGT and goes to the other domains KDC's for the service tickets.

mytest.txt

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Colin Simpson - 2013-07-24

This still occurs on 4.3.3. Any thoughts on this or how I might be able to help you with this one?

Thanks

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Mickael Guessant - 2013-07-25

Well, basically we need to find why DavMail kerberos configuration does not work in your case. Unfortunately I don't have the infrastructure to reproduce this here.

This may be an issue with KerberosLoginConfiguration set in KerberosHelper:
Security.setProperty("login.configuration.provider", "davmail.http.KerberosLoginConfiguration");

=> You may comment this out and rename your login context to spnego-client in login.conf

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Colin Simpson - 2013-09-27

I reviewed the GSSAPI davmail code and it all looks correct from my limited knowledge of Java. As I said before my test code succeeds but doesn't use any GSSAPI calls. So confused, I maybe need to re-jig my test code to use GSSAPI and try again. But if that fails not sure where to turn to except maybe online forums, maybe wireshark it.

Also tried on an AD Windows machine (all my previous tests are on Linux systems that are AD joined). It fails on Windows in the same way, which if fortunate in one sense.

Any other ideas as to how to debug?

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Colin Simpson - 2014-02-18

Ah cracked it!

Sadly I went as far as extracting all the GSSAPI code from Davmail into a separate standalone program to debug this.

But cutting to the chase, in KerberosHelper.java, inside the internalInitSecContext there is a line:

GSSName serverName = manager.createName(protocol + "/" + host, null);

Reading the documentation for this from here:

http://docs.oracle.com/javase/7/docs/technotes/guides/security/jgss/tutorials/BasicClientServer.html

It says:

You can also leave off the realm if a principal name will be converted to a
GSSName by a GSSManager createName method. For example, when you run
SampleClient, one of the arguments is the server service principal name. You
can specify the name without including the realm, because SampleClient
passes the name to such a createName method, which appends the default realm
as needed.

It is recommended that you always include realms when principal names are
used in login configuration files and policy files, because the behavior of
the parsers for such files may be implementation-dependent; they may or may
not append the default realm before such names are utilized and subsequent
actions may fail if there is no realm in the name."

So the way this is presently written it assumes that the host is always in the same realm as the default realm. This is possibly not the case, certainly not in our case. I verified that this fixes my cross-realm issue by hand hacking my domain into this line i.e

GSSName serverName = manager.createName(protocol + "/" + host + "@RD.IONGEO.COM", null);

This should this be properly fixed by looking up the realm for this host (I'm guessing). But I can't see an easy way to do that (again I'm no Java expert). These guys had the same issue, but they seemed to "fix it" by looking up the realm in krb5.conf which isn't cross platform:

https://issues.apache.org/jira/browse/HTTPCLIENT-1067

And they rejected this!

One simple way might be to allow the realm to be passed in the davmail optionsbut there must be a better way. My earlier java example (in this bug) did work cross-realm using "HttpsURLConnection", so this call must have some way of working out the realm in a cross platform way.

Or can a different method be used in createname that looks up the realm? Maybe:

GSSName name = manager.createName("service@host",
GSSName.NT_HOSTBASED_SERVICE);

Not sure, if this does this or not?

One hack is to specify the default realm on the command line e.g.
-Djava.security.krb5.realm=RD.IONGEO.COM

This though has a nasty limitation as per

http://docs.oracle.com/javase/7/docs/technotes/guides/security/jgss/tutorials/KerberosReq.html#SetProps

It says:

java.security.krb5.realm

java.security.krb5.kdc

If you set one of these properties you must set them both.

Why should that be necessary i.e. if you set the realm you must statically set the KDC for this domain, even though it can be found by DNS (or specified in an krb5.conf or krb5.ini file).

Any thoughts as to how best to fix?

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Mauro Cicognini - 2014-02-19
  
  Hi Colin, I have a glitch in the Kerberos authentication as well,
  perhaps you could share your program so that I can troubleshoot it?
  In my case, authentication works but I always get a prompt, although it
  is supposed to take the credentials I've logged on to the machine with.
  
  Thanks in advance
  Mauro
  
  --
  On 18/02/2014 21:12, Colin Simpson wrote:
  
  Ah cracked it!
  
  Sadly I went as far as extracting all the GSSAPI code from Davmail
  into a separate standalone program to debug this.
  
  But cutting to the chase, in KerberosHelper.java, inside the
  internalInitSecContext there is a line:
  
  GSSName serverName = manager.createName(protocol + "/" + host, null);
  
  Reading the documentation for this from here:
  
  http://docs.oracle.com/javase/7/docs/technotes/guides/security/jgss/tutorials/BasicClientServer.html
  
  It says:
  
  You can also leave off the realm if a principal name will be converted to a GSSName by a GSSManager createName method. For example, when you run SampleClient, one of the arguments is the server service principal name. You can specify the name without including the realm, because SampleClient passes the name to such a createName method, which appends the default realm as needed. It is recommended that you always include realms when principal names are used in login configuration files and policy files, because the behavior of the parsers for such files may be implementation-dependent; they may or may not append the default realm before such names are utilized and subsequent actions may fail if there is no realm in the name."
  
  So the way this is presently written it assumes that the host is
  always in the same realm as the default realm. This is possibly not
  the case, certainly not in our case. I verified that this fixes my
  cross-realm issue by hand hacking my domain into this line i.e
  
  GSSName serverName = manager.createName(protocol + "/" + host +
  "@RD.IONGEO.COM", null);
  
  This should this be properly fixed by looking up the realm for this
  host (I'm guessing). But I can't see an easy way to do that (again I'm
  no Java expert). These guys had the same issue, but they seemed to
  "fix it" by looking up the realm in krb5.conf which isn't cross platform:
  
  https://issues.apache.org/jira/browse/HTTPCLIENT-1067
  
  And they rejected this!
  
  One simple way might be to allow the realm to be passed in the davmail
  optionsbut there must be a better way. My earlier java example (in
  this bug) did work cross-realm using "HttpsURLConnection", so this
  call must have some way of working out the realm in a cross platform way.
  
  Or can a different method be used in createname that looks up the
  realm? Maybe:
  
  GSSName name = manager.createName("service@host",
  GSSName.NT_HOSTBASED_SERVICE);
  
  Not sure, if this does this or not?
  
  One hack is to specify the default realm on the command line e.g.
  -Djava.security.krb5.realm=RD.IONGEO.COM
  
  This though has a nasty limitation as per
  
  http://docs.oracle.com/javase/7/docs/technotes/guides/security/jgss/tutorials/KerberosReq.html#SetProps
  
  It says:
  
  java.security.krb5.realm java.security.krb5.kdc If you set one of these properties you must set them both.
  
  Why should that be necessary i.e. if you set the realm you must
  statically set the KDC for this domain, even though it can be found by
  DNS (or specified in an krb5.conf or krb5.ini file).
  
  Any thoughts as to how best to fix?
  
  [bugs:#534] http://sourceforge.net/p/davmail/bugs/534/ Kerberos
  Authentication doesn't seem to be work cross domain
  
  Status: open
  Created: Fri Jun 14, 2013 03:50 PM UTC by Colin Simpson
  Last Updated: Mon Feb 17, 2014 12:15 AM UTC
  Owner: nobody
  
  I have been trying to get Kerberos authentication working with
  davmail. In my case to the caldav. I think the issue might be this
  doesn't seem to work cross realm.
  
  Basically I'm connecting to an Exchange OWA server (on a cas box in
  Exchange 2010). This setup works using a password with Kerberos turned
  off. I'm using Thunderbird lightning to connect and when attempting to
  use Kerberos authentication I'm just putting gunk into the username
  and password prompt from Thunderbird (I think this is the correct
  thing to do as davmail is doing the Kerberos auth).
  
  I know the Kerberos authentication is working for this server (on our
  network), as if I connect to the URL of the Exchange cas in a browser
  (the same one davmail is also going to) I get connected passwordless,
  without an issue.
  
  The output from the debug on davmail is limited to the following:
  
  2013-06-14 15:42:32,708 DEBUG [CaldavConnection-60547]
  org.apache.commons.httpclient.auth.AuthChallengeProcessor - Using
  authentication scheme: Negotiate
  2013-06-14 15:42:32,708 DEBUG [CaldavConnection-60547]
  org.apache.commons.httpclient.auth.AuthChallengeProcessor -
  Authorization challenge processed
  2013-06-14 15:42:32,708 DEBUG [CaldavConnection-60547]
  org.apache.commons.httpclient.HttpMethodDirector - Authentication
  scope: NEGOTIATE @eu1casarray.rd.iongeo.com:443
  2013-06-14 15:42:32,708 DEBUG [CaldavConnection-60547]
  org.apache.commons.httpclient.HttpMethodDirector - Credentials required
  2013-06-14 15:42:32,708 DEBUG [CaldavConnection-60547]
  org.apache.commons.httpclient.HttpMethodDirector - Credentials
  provider not available
  2013-06-14 15:42:32,708 INFO [CaldavConnection-60547]
  org.apache.commons.httpclient.HttpMethodDirector - Failure
  authenticating with NEGOTIATE @eu1casarray.rd.iongeo.com:443
  
  Turning on Kerberos debug for Java "-Dsun.security.krb5.debug=true"
  gives more information. I have attached this output.
  
  Now my account is in a domain (REALM) EU.IONGEO.COM whereas the server
  is in the REALM RD.IONGEO.COM (there is a full trust in place). The
  reason I suspect a cross realm issue is that a bit of googling turns
  up the same error as we see at the end of this output:
  
  EType: sun.security.krb5.internal.crypto.ArcFourHmacEType KrbException: Message stream modified (41) at sun.security.krb5.KrbKdcRep.check(Unknown Source)
  
  http://bugs.sun.com/view_bug.do?bug_id=6727246
  
  But this looks fixed especially as I'm using Java 1.7.0-21
  
  Any ideas how I can debug further or if davmail isn't properly
  implementing cross realm Kerberos?
  
  Thanks
  
  Sent from sourceforge.net because you indicated interest in
  https://sourceforge.net/p/davmail/bugs/534/
  
  To unsubscribe from further messages, please visit
  https://sourceforge.net/auth/subscriptions/
  
  Related
  
  Bugs: ~~#534~~
  
  alternate
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Colin Simpson - 2014-02-19

It looks like replacing

GSSName serverName = manager.createName(protocol + "/" + host, null);

with

GSSName serverName = manager.createName("http@" + host, GSSName.NT_HOSTBASED_SERVICE);

, fixes the cross-realm authentication for me. I had a quick look at the mutt GSSAPI C code, and they call using this for IMAP:

/ get an IMAP service ticket for the server /
snprintf (buf1, sizeof (buf1), "imap@%s", idata->conn->account.host);
request_buf.value = buf1;
request_buf.length = strlen (buf1) + 1;
maj_stat = gss_import_name (&min_stat, &request_buf, gss_nt_service_name,
&target_name);

So I think this is correct.

I guess the other minor change will be in initSecurityContext to change the debug line if the above change is made:

LOGGER.debug("KerberosHelper.initSecurityContext " + protocol + "/" + host + " " + token.length + " bytes token");

, as in the above will no longer be accurate.

This should resolve the issue.

One other thing that I notice this code doesn't seem to do, is to canonicalize the hostname. Usually the service principal for HTTP is the hostname of the server and not the service i.e. if the service is a CNAME it will use the real hostname of the server for the principal. Not sure how common this is in an exchange setting.

So for example, if I say my server is:

http://blah/ and blah is a CNAME to "waffle.fqdn", then the service principal looked for is "http/waffle.fqdn". This is true on IE, Firefox. It allows you to move services around to servers without having to change (or move) the "http/" principals a server holds (you just change the CNAME). The only thing that doesn't seem to so this is Safari:

https://discussions.apple.com/thread/2414517

As I say, don't know if anyone ever does this with Exchange just thought I'd mention it.....

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Mickael Guessant - 2014-03-02

Ok, will need some time to investigate this.

As we rely on local Kerberos ticket, do you think we could retrieve realm from ticket ?

See KerberosHelper.clientLogin code for more details.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Colin Simpson - 2014-03-03

You can't retrieve the destination realm (of the host) from the ticket as it's not in there until something requests a ticket for the host. So chicken and egg.

The solution/patch I gave should be correct. The original way currently in the code is saying kind of you will do all the work of setting the name up for the underlying authenticator. My change says let GSSAPI do it. See the RFC on this:

http://tools.ietf.org/html/rfc5653#page-45
http://tools.ietf.org/html/rfc5653#page-44

You should be able to make this change and it should just past your tests and work cross-realm.

Thanks

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Mickael Guessant - 2014-03-10

Indeed, already patched local code, but need to restart Kerberos environment to check for regressions

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Colin Simpson - 2014-03-11

Good news.

@Mauro: Did any of my suggestions to fix your Kerberos work? Happy to help further on this.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Colin Simpson - 2014-07-22

This patch doesn't seem to have made it into a release yet? Any issue with it that you need some help to look at?

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Colin Simpson - 2014-07-23

Just if any help my change is still working correctly on 4.5.1-2303 i.e. change in KerberosHelper.java

GSSName serverName = manager.createName(protocol + '/' + host, null);

to

GSSName serverName = manager.createName(protocol + '@' + host, GSSName.NT_HOSTBASED_SERVICE);

This was tested on Linux haven't tested on Windows.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Colin Simpson - 2014-07-24

I have now tested this patch on Windows and it does work fine (with a suitable krb5.ini file).

Not sure why Windows needs that krb5.ini (I just a chunk of my Linux krb5.conf as the Windows krb5.ini). But I guess it's the domain_realm mappings it wants, I just have KDC lookup to use dns i.e "dns_lookup_kdc = true", though for here I have "dns_lookup_realm = false".

I thought it would be able to work out the realm from the domain of the URL, by defaulting to using the upper case version of the domain as the realm, but I guess that's just not what Java does even though other Windows programs seem to.

So my krb5.ini is (my machine is a member of eu.iongeo.com, the exchange servers are members of eu.ionngeo.com, and iongeo.com is the parent domain of the forest):

[libdefaults]
default_realm = EU.IONGEO.COM
forwardable = true
dns_lookup_realm = false
dns_lookup_kdc = true

[domain_realm]
.eu.iongeo.com = EU.IONGEO.COM
eu.iongeo.com = EU.IONGEO.COM
.rd.iongeo.com = RD.IONGEO.COM
rd.iongeo.com = RD.IONGEO.COM
.iongeo.com = IONGEO.COM
iongeo.com = IONGEO.COM

Hope this helps someone.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Mickael Guessant - 2014-07-28

Thanks for your feedback, didn't find time to check this myself.
=> your fix is available in subversion, can you please confirm ?

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Colin Simpson - 2014-07-29

The version pulled from SVN looks fine and is working cross realm for me.

As a minor detail I wonder if the debug line:

LOGGER.debug("KerberosHelper.initSecurityContext " + protocol + '/' + host + ' ' + token.length + " bytes token");

should now say

LOGGER.debug("KerberosHelper.initSecurityContext " + protocol + '@' + host + ' ' + token.length + " bytes token");

Not a big deal though....

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Mickael Guessant - 2014-07-31

Definitely, patch merged

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Mickael Guessant - 2014-07-31

status: open --> open-fixed
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Colin Simpson - 2015-02-16

The new version 4.6.0-2331 is working cross realm for me, as expected. So this big can be marked closed as far as I'm concerned.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Mickael Guessant - 2015-02-18

status: open-fixed --> closed-fixed
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Kerberos Authentication doesn't seem to be work cross domain

Group

Searches

Help

#534 Kerberos Authentication doesn't seem to be work cross domain

Related

Discussion

Related