dnsjava-users — The dnsjava mailing list

Re: UDP responses on Mac OS 10.7/java 1.6

[Brian W. <bwe...@xb...>]

On Aug 6, 2012, at 2:46 PM, Casey Deccio <ca...@de...> wrote:

> 
> On Mon, Aug 6, 2012 at 2:17 PM, Brian Wellington <bwe...@xb...> wrote:
> 
> I suspect the problem might be firewall-related; when java is set to "Block incoming connections", I see timeouts with the same stack trace.  I don't know what dnsjava is doing differently with respect to the network and firewall rules, but you might want to try allowing incoming connections for java to see if that fixes it.
> 
> 
> Ugh.  Yes, that was it.  Thanks!  Apparently I had forgotten that OS X's firewall considers processes, not just ports, which is why I didn't see the issue with ISC's dig.
> 
> I'm curious why UDP responses for dnsjava are looked at as "incoming connections", whereas UDP responses for ISC's dig, for example, don't have this problem.

I don't know.  Both should be doing similar operations, but there's obviously some difference (unless there's something else giving dig extra permissions, but I don't see it).

> Would a simple UDP client/server test work with DatagramPacket?  I assume using the lower level interfaces in dnsjava is to obtain source port randomization?  Is it possible that makes it behave differently, in terms of state?

It's using the interfaces it does because of source port randomization, but opening a UDP socket by address and port shouldn't be any different, security wise, from opening a UDP socket with a given address and the wildcard port.

Brian

Re: UDP responses on Mac OS 10.7/java 1.6

[Casey D. <ca...@de...>]

On Mon, Aug 6, 2012 at 2:17 PM, Brian Wellington <bwe...@xb...> wrote:

>
> I suspect the problem might be firewall-related; when java is set to
> "Block incoming connections", I see timeouts with the same stack trace.  I
> don't know what dnsjava is doing differently with respect to the network
> and firewall rules, but you might want to try allowing incoming connections
> for java to see if that fixes it.
>
>
Ugh.  Yes, that was it.  Thanks!  Apparently I had forgotten that OS X's
firewall considers processes, not just ports, which is why I didn't see the
issue with ISC's dig.

I'm curious why UDP responses for dnsjava are looked at as "incoming
connections", whereas UDP responses for ISC's dig, for example, don't have
this problem.

Would a simple UDP client/server test work with DatagramPacket?  I assume
using the lower level interfaces in dnsjava is to obtain source port
randomization?  Is it possible that makes it behave differently, in terms
of state?

Casey

Re: UDP responses on Mac OS 10.7/java 1.6

[Brian W. <bwe...@xb...>]

On Aug 6, 2012, at 12:32 PM, Casey Deccio <ca...@de...> wrote:

> Hi,
> 
> I'm using dnsjava-2.1.3 on Mac OS 10.7 Lion and java 1.6:
> 
> $ java -version
> java version "1.6.0_33"
> Java(TM) SE Runtime Environment (build 1.6.0_33-b03-424-11M3720)
> Java HotSpot(TM) 64-Bit Server VM (build 20.8-b03-424, mixed mode)
> 
> I'm unable to receive UDP responses.  For example, using the provided dig class:
> 
> $ java dig @192.0.2.1 example.com
> Exception in thread "main" java.net.SocketTimeoutException
>     at org.xbill.DNS.Client.blockUntil(Client.java:43)
>     at org.xbill.DNS.UDPClient.recv(UDPClient.java:124)
>     at org.xbill.DNS.UDPClient.sendrecv(UDPClient.java:150)
>     at org.xbill.DNS.SimpleResolver.send(SimpleResolver.java:257)
>     at dig.main(dig.java:196)
> 
> I get the same result if I use the library within my own class as well.  I can see the packets arriving on the wire using tcpdump, but the client isn't picking them up.  However, if I use resolver.setTCP(true), then it works fine.  It also works fine with ISC's DiG:
> 
> $ dig @192.0.2.1 example.com
> 
> ; <<>> DiG 9.7.3-P3 <<>> @192.0.2.1 example.com
> ; (1 server found)
> ;; global options: +cmd
> ;; Got answer:
> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 37887
> ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 0
> 
> ;; QUESTION SECTION:
> ;example.com.            IN    A
> 
> ;; ANSWER SECTION:
> example.com.        2170    IN    A    192.0.43.10
> 
> ;; AUTHORITY SECTION:
> example.com.        171370    IN    NS    a.iana-servers.net.
> example.com.        171370    IN    NS    b.iana-servers.net.
> 
> ;; Query time: 2 msec
> ;; SERVER: 192.0.2.1#53(192.0.2.1)
> ;; WHEN: Mon Aug  6 12:26:58 2012
> ;; MSG SIZE  rcvd: 93
> 
> Any ideas?

Until last week, my primary development platform was OS X Lion, and I never saw this.

I suspect the problem might be firewall-related; when java is set to "Block incoming connections", I see timeouts with the same stack trace.  I don't know what dnsjava is doing differently with respect to the network and firewall rules, but you might want to try allowing incoming connections for java to see if that fixes it.

Brian

UDP responses on Mac OS 10.7/java 1.6

[Casey D. <ca...@de...>]

Hi,

I'm using dnsjava-2.1.3 on Mac OS 10.7 Lion and java 1.6:

$ java -version
java version "1.6.0_33"
Java(TM) SE Runtime Environment (build 1.6.0_33-b03-424-11M3720)
Java HotSpot(TM) 64-Bit Server VM (build 20.8-b03-424, mixed mode)

I'm unable to receive UDP responses.  For example, using the provided dig
class:

$ java dig @192.0.2.1 example.com
Exception in thread "main" java.net.SocketTimeoutException
    at org.xbill.DNS.Client.blockUntil(Client.java:43)
    at org.xbill.DNS.UDPClient.recv(UDPClient.java:124)
    at org.xbill.DNS.UDPClient.sendrecv(UDPClient.java:150)
    at org.xbill.DNS.SimpleResolver.send(SimpleResolver.java:257)
    at dig.main(dig.java:196)

I get the same result if I use the library within my own class as well.  I
can see the packets arriving on the wire using tcpdump, but the client
isn't picking them up.  However, if I use resolver.setTCP(true), then it
works fine.  It also works fine with ISC's DiG:

$ dig @192.0.2.1 example.com

; <<>> DiG 9.7.3-P3 <<>> @192.0.2.1 example.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 37887
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 0

;; QUESTION SECTION:
;example.com.            IN    A

;; ANSWER SECTION:
example.com.        2170    IN    A    192.0.43.10

;; AUTHORITY SECTION:
example.com.        171370    IN    NS    a.iana-servers.net.
example.com.        171370    IN    NS    b.iana-servers.net.

;; Query time: 2 msec
;; SERVER: 192.0.2.1#53(192.0.2.1)
;; WHEN: Mon Aug  6 12:26:58 2012
;; MSG SIZE  rcvd: 93

Any ideas?

Casey

RE: Record.byteArrayToString escaping characters

[Abby K. <Abb...@xc...>]

Inline.

Abby.

From: Brian Wellington [mailto:bwe...@xb...]
Sent: Tuesday, March 20, 2012 22:22 PM
To: Abby Kehat
Cc: dns...@li...
Subject: Re: Record.byteArrayToString escaping characters


On Mar 20, 2012, at 5:09 AM, Abby Kehat wrote:


Brian -

Actually, it's more of a problem that retrieving the regexp field.

The purpose of the NAPTRRecord is to support regular expression based rewriting. The rewriting is often done using regular expression backreferences to refer to the original phone number in the query. You can see the specification of the regular expression field syntax in section 3.2 of this<http://rfc-ref.org/RFC-TEXTS/3402/chapter3.html#sub2> RFC, including an example of using backreferences. Basically the first part (between the first two '!') is an extended regular expression which is used to capture [part of] the original phone number. The second part (between the second two '!') is a substitution expression which is applied to the text captured by the first part.

For example, the regular expression field !^(.+)$!0\1! means: capture the entire phone number (^(.+)$). Prefix the phone number with a zero (0\1).

When written as a Java String, !^(.+)$!0\1! becomes "!^(.+)$!0\\1!" since the backslash has to be escaped in Java strings. If the backslash is not escaped, it has a totally different meaning: replace the original phone number with the string '0\001', where \001 represents the unprintable character with byte value of 1.

Right.  Java uses backslashes for escaping in Strings.


However, constructing a NAPTRRecord with a regular expression containing backreferences throws an IllegalArgumentException!

The following code attempts to create a NAPTRRecord with a regular expression mentioned above, !^(.+)$!0\1!.

NAPTRRecord rec = new NAPTRRecord(name,                     // Name
DClass.IN,              // Class
0,                      // TTL
0,                      // Order
0,                      // Preference
"u",                    // Flags
"E2U+sip",              // Service
"!^(.+)$!0\\1!",        // Regular expression.
Name.root);             // Replacement

Exception in thread "main" java.lang.IllegalArgumentException: bad escape
      at org.xbill.DNS.NAPTRRecord.<init>(NAPTRRecord.java:55)

The problem is the backreference in the replacement part of the regular expression; the string '0\1'. ThebyteArrayFromString(String) method used by the NAPTRRecord constructor throws the exception when it parses the '\1' as it doesn't know how to handle a backslash followed by less than 3 digits (see theRecord.byteArrayFromString(String s) method,  lines 374-375).

That's DNS master file escaping.  One could probably argue that since this isn't a master file, master file escaping isn't necessary, but that's how dnsjava has worked for over 10 years, and changing it now would likely break other users.

Indeed, this is not a master file, so escaping should not be necessary - nor should it be necessary in the other record classes, either. I understand the importance of backwards compatibility so as not to break existing code.

As backreferences are quite often needed in NAPTR records, it would be great if you could fix this issue. Right now we have a workaround doubling and singling backslashes when accessing the regular expression, but this is causing quite a bit of confusion in our code and our logs.

I don't know what the right answer is for this.  Adding a new NAPTRRecord constructor that took byte arrays instead of Strings might solve the problem, but it's ugly (since you probably have Strings, and would need to convert them into byte arrays).  Adding a new constructor that took Strings, but interpreted them differently, would be weird.  Adding some sort of global property for this might work, but may cause other problems.

Brian

It seems to me that that a constructor that doesn't accept a legal Java String is a problem, especially since the JavaDoc says nothing about constraints on the string contents.

But more importantly, the specific purpose of the NAPTR record is for regular expression rewriting. An application that processes rewrite rules should be able to retrieve the regular expression as it was written, without escaping backreferences, or it will not be processed properly. Right now we are working around this problem by doubling backslashes before passing it to the constructor, and 'singling' backslashes when using the getregexp() method.

A simple solution is to override the Record class's byteArrayFromString and byteArrayToString methods in the NAPTRRecord class, so that no escaping is done. This would correctly allow one to create a regular expression with backreferences and retrieve it properly. I imagine that anyone using the NAPTRRecord for regular expression rewriting must be having a similar problem, unless they have a similar workaround, and would welcome such a fix. On the other hand, if one does not have backreferences, there is no problem and this change shouldn't affect them. This should probably be done with any string fields in the record classes. Naturally, the change must be well documented in the changelog.

With the need to preserve backwards compatibility, I'm not sure what the correct answer is, either. Most solutions are going to be ugly, since the issue is in the basic representation of the record. Though ugly, you might add another argument to the constructor indicating that the regular expression is to be stored without escaping it. In order that we should not have to do so - again, though ugly - you can have another argument to the getregexp() method indicating that it should be returned as is, without dealing with escapes. We still have the issue of the toString() method incorrectly displaying two backslashes, but as long as the content is correct, that is a lower priority issue, though quite confusing. And then, again, you could provide a toString method with an argument. Ugly, but it works.

I'd be very curious to hear if other users are having similar issues.

Thanks,
Abby.

Re: Record.byteArrayToString escaping characters

[Brian W. <bwe...@xb...>]

On Mar 20, 2012, at 5:09 AM, Abby Kehat wrote:

> Brian –
>  
> Actually, it’s more of a problem that retrieving the regexp field.
>  
> The purpose of the NAPTRRecord is to support regular expression based rewriting. The rewriting is often done using regular expression backreferences to refer to the original phone number in the query. You can see the specification of the regular expression field syntax in section 3.2 of this RFC, including an example of using backreferences. Basically the first part (between the first two ‘!’) is an extended regular expression which is used to capture [part of] the original phone number. The second part (between the second two ‘!’) is a substitution expression which is applied to the text captured by the first part.
>  
> For example, the regular expression field !^(.+)$!0\1! means: capture the entire phone number (^(.+)$). Prefix the phone number with a zero (0\1).
>  
> When written as a Java String, !^(.+)$!0\1! becomes "!^(.+)$!0\\1!" since the backslash has to be escaped in Java strings. If the backslash is not escaped, it has a totally different meaning: replace the original phone number with the string ‘0\001’, where \001 represents the unprintable character with byte value of 1.

Right.  Java uses backslashes for escaping in Strings.

> However, constructing a NAPTRRecord with a regular expression containing backreferences throws an IllegalArgumentException!
>  
> The following code attempts to create a NAPTRRecord with a regular expression mentioned above, !^(.+)$!0\1!.
>  
> NAPTRRecord rec = new NAPTRRecord(name,                     // Name
> DClass.IN,              // Class
> 0,                      // TTL
> 0,                      // Order
> 0,                      // Preference
> "u",                    // Flags
> "E2U+sip",              // Service
> "!^(.+)$!0\\1!",        // Regular expression.
> Name.root);             // Replacement
>  
> Exception in thread "main" java.lang.IllegalArgumentException: bad escape
>       at org.xbill.DNS.NAPTRRecord.<init>(NAPTRRecord.java:55)
>  
> The problem is the backreference in the replacement part of the regular expression; the string ‘0\1’. ThebyteArrayFromString(String) method used by the NAPTRRecord constructor throws the exception when it parses the ‘\1’ as it doesn’t know how to handle a backslash followed by less than 3 digits (see theRecord.byteArrayFromString(String s) method,  lines 374-375).

That's DNS master file escaping.  One could probably argue that since this isn't a master file, master file escaping isn't necessary, but that's how dnsjava has worked for over 10 years, and changing it now would likely break other users.

> As backreferences are quite often needed in NAPTR records, it would be great if you could fix this issue. Right now we have a workaround doubling and singling backslashes when accessing the regular expression, but this is causing quite a bit of confusion in our code and our logs.

I don't know what the right answer is for this.  Adding a new NAPTRRecord constructor that took byte arrays instead of Strings might solve the problem, but it's ugly (since you probably have Strings, and would need to convert them into byte arrays).  Adding a new constructor that took Strings, but interpreted them differently, would be weird.  Adding some sort of global property for this might work, but may cause other problems.

Brian

RE: Record.byteArrayToString escaping characters

[Abby K. <Abb...@xc...>]

Brian -

Actually, it's more of a problem that retrieving the regexp field.

The purpose of the NAPTRRecord is to support regular expression based rewriting. The rewriting is often done using regular expression backreferences to refer to the original phone number in the query. You can see the specification of the regular expression field syntax in section 3.2 of this<http://rfc-ref.org/RFC-TEXTS/3402/chapter3.html#sub2> RFC, including an example of using backreferences. Basically the first part (between the first two '!') is an extended regular expression which is used to capture [part of] the original phone number. The second part (between the second two '!') is a substitution expression which is applied to the text captured by the first part.

For example, the regular expression field !^(.+)$!0\1! means: capture the entire phone number (^(.+)$). Prefix the phone number with a zero (0\1).

When written as a Java String, !^(.+)$!0\1! becomes "!^(.+)$!0\\1!" since the backslash has to be escaped in Java strings. If the backslash is not escaped, it has a totally different meaning: replace the original phone number with the string '0\001', where \001 represents the unprintable character with byte value of 1.

However, constructing a NAPTRRecord with a regular expression containing backreferences throws an IllegalArgumentException!

The following code attempts to create a NAPTRRecord with a regular expression mentioned above, !^(.+)$!0\1!.

NAPTRRecord rec = new NAPTRRecord(name,                     // Name
DClass.IN,              // Class
0,                      // TTL
0,                      // Order
0,                      // Preference
"u",                    // Flags
"E2U+sip",              // Service
"!^(.+)$!0\\1!",        // Regular expression.
Name.root);             // Replacement

Exception in thread "main" java.lang.IllegalArgumentException: bad escape
      at org.xbill.DNS.NAPTRRecord.<init>(NAPTRRecord.java:55)

The problem is the backreference in the replacement part of the regular expression; the string '0\1'. The byteArrayFromString(String) method used by the NAPTRRecord constructor throws the exception when it parses the '\1' as it doesn't know how to handle a backslash followed by less than 3 digits (see the Record.byteArrayFromString(String s) method,  lines 374-375).

As backreferences are quite often needed in NAPTR records, it would be great if you could fix this issue. Right now we have a workaround doubling and singling backslashes when accessing the regular expression, but this is causing quite a bit of confusion in our code and our logs.

Thanks very much,
Abby.

From: Abby Kehat
Sent: Thursday, March 15, 2012 15:02 PM
To: 'Brian Wellington'
Subject: RE: Record.byteArrayToString escaping characters

Brian -

Thanks for your reply. Actually, we had patched a previous release of dnsjava to include a getRegexpAsByteArray() method as a workaround - most probably after seeing something similar in the TXTRecord - so it's funny that you suggested precisely the same thing as a solution.

Might you be able to add this additional method in a formal release? We work with Maven so we do not want to stray too far from formal releases.

If you did add this method, what would the timeframe be?

Thanks very much for your assistance,
Abby.

From: Brian Wellington [mailto:bwe...@xb...]
Sent: Wednesday, March 14, 2012 20:49 PM
To: Abby Kehat
Cc: dns...@li...<mailto:dns...@li...>
Subject: Re: Record.byteArrayToString escaping characters


On Mar 14, 2012, at 9:34 AM, Abby Kehat wrote:

I am using dnsjava-2.1.3. When converting a record to a String, the Record.byteArrayToString method escapes some characters ('\' and '"') with a '\'. Unfortunately, this means that the resulting string doesn't accurately represent the contents of the message and can cause incorrect results when the string is used as is in other processing.

For example, suppose I am handling NAPTR records.

NAPTRRecord naptrRecord = ... ;
String regExp = naptrRecord.getRegexp();

Suppose the NAPTR record has a regular expression of "!^(.+)$!0\1!". This quite standard regular expression means that the replacement part (between the second two '!') is to be applied to the string matched by the first part (between the first two '!'). In this simple example, we expect the string captured by the first part of the expression,^(.+)$ - which, in our case, captures the entire string - to be prefixed with the digit 0.

However, the NAPTR getRegexp() method calls the byteArrayToString method, thereby escaping the '\', giving the String "!^(.+)$!0\\1!". If used in regular expression processing the additional \ will cause the result to be "0\1", which is incorrect.

I could manually replace the escaped characters myself, but since the toString method of the dnsjava classes uses the byteArrayToString method, I wouldn't be able to do this in all uses of the byteArrayToString method. Additionally, there would be some inconsistency in what is displayed in a log file, for example, and what is actually in the wire format of the object.

One was of solving this is by providing a byteArray method, which simply returns the wire format of a field. The String(byte []) constructor can be used to convert this to a String.
Another option is to simply use the String(byte[]) constructor in the byteArrayToString method. Obviously, this would might problems in displaying unprintable characters, but at theast the String would be an accurate representation of what's in the message.

Any assistance would be appreciated.

I don't know what the right answer is.  One solution could involve adding a parallel method to getRegexp(), which returns the data as a byte array; this is somewhat similar to TXTRecord's getStringsAsByteArrays().  But I don't think regexp's work with byte arrays, and converting the byte arrays into strings using String(byte[]) is going to have problems with unprintable characters and/or decode multibyte characters.  In any case, it would be really hard for dnsjava to do "the right thing" here, because the right thing isn't too clear, and probably won't be the same for everyone.

If it turns out that getRegexpAsByteArray() (or a better name) is useful, it could be added.

Brian

Re: Record.byteArrayToString escaping characters

[Brian W. <bwe...@xb...>]

On Mar 14, 2012, at 9:34 AM, Abby Kehat wrote:

> I am using dnsjava-2.1.3. When converting a record to a String, the Record.byteArrayToString method escapes some characters (‘\’ and ‘”’) with a ‘\’. Unfortunately, this means that the resulting string doesn’t accurately represent the contents of the message and can cause incorrect results when the string is used as is in other processing.
>  
> For example, suppose I am handling NAPTR records.
>  
> NAPTRRecord naptrRecord = … ;
> String regExp = naptrRecord.getRegexp();
>  
> Suppose the NAPTR record has a regular expression of “!^(.+)$!0\1!”. This quite standard regular expression means that the replacement part (between the second two ‘!’) is to be applied to the string matched by the first part (between the first two ‘!’). In this simple example, we expect the string captured by the first part of the expression,^(.+)$ - which, in our case, captures the entire string - to be prefixed with the digit 0.
>  
> However, the NAPTR getRegexp() method calls the byteArrayToString method, thereby escaping the ‘\’, giving the String “!^(.+)$!0\\1!”. If used in regular expression processing the additional \ will cause the result to be “0\1”, which is incorrect.
>  
> I could manually replace the escaped characters myself, but since the toString method of the dnsjava classes uses the byteArrayToString method, I wouldn’t be able to do this in all uses of the byteArrayToString method. Additionally, there would be some inconsistency in what is displayed in a log file, for example, and what is actually in the wire format of the object.
>  
> One was of solving this is by providing a byteArray method, which simply returns the wire format of a field. The String(byte []) constructor can be used to convert this to a String.
> Another option is to simply use the String(byte[]) constructor in the byteArrayToString method. Obviously, this would might problems in displaying unprintable characters, but at theast the String would be an accurate representation of what’s in the message.
>  
> Any assistance would be appreciated.

I don't know what the right answer is.  One solution could involve adding a parallel method to getRegexp(), which returns the data as a byte array; this is somewhat similar to TXTRecord's getStringsAsByteArrays().  But I don't think regexp's work with byte arrays, and converting the byte arrays into strings using String(byte[]) is going to have problems with unprintable characters and/or decode multibyte characters.  In any case, it would be really hard for dnsjava to do "the right thing" here, because the right thing isn't too clear, and probably won't be the same for everyone.

If it turns out that getRegexpAsByteArray() (or a better name) is useful, it could be added.

Brian

Record.byteArrayToString escaping characters

[Abby K. <Abb...@xc...>]

I am using dnsjava-2.1.3. When converting a record to a String, the Record.byteArrayToString method escapes some characters ('\' and '"') with a '\'. Unfortunately, this means that the resulting string doesn't accurately represent the contents of the message and can cause incorrect results when the string is used as is in other processing.

For example, suppose I am handling NAPTR records.

NAPTRRecord naptrRecord = ... ;
String regExp = naptrRecord.getRegexp();

Suppose the NAPTR record has a regular expression of "!^(.+)$!0\1!". This quite standard regular expression means that the replacement part (between the second two '!') is to be applied to the string matched by the first part (between the first two '!'). In this simple example, we expect the string captured by the first part of the expression, ^(.+)$ - which, in our case, captures the entire string - to be prefixed with the digit 0.

However, the NAPTR getRegexp() method calls the byteArrayToString method, thereby escaping the '\', giving the String "!^(.+)$!0\\1!". If used in regular expression processing the additional \ will cause the result to be "0\1", which is incorrect.

I could manually replace the escaped characters myself, but since the toString method of the dnsjava classes uses the byteArrayToString method, I wouldn't be able to do this in all uses of the byteArrayToString method. Additionally, there would be some inconsistency in what is displayed in a log file, for example, and what is actually in the wire format of the object.

One was of solving this is by providing a byteArray method, which simply returns the wire format of a field. The String(byte []) constructor can be used to convert this to a String.
Another option is to simply use the String(byte[]) constructor in the byteArrayToString method. Obviously, this would might problems in displaying unprintable characters, but at theast the String would be an accurate representation of what's in the message.

Any assistance would be appreciated.

Thanks,
Abby.

Re: DNSSSEC verification example

[Adam F. <af...@br...>]

A link would be welcome, but I mostly just wanted to let you know your
detailed responses weren't falling on deaf ears.

The only tricky part about third parties posting to Sonatype is that
they require you to control the domain you publish under. I could do
it under org.littleshoot, for example, but can't do it under
org.xbill. It would also be somewhat challenging to stay in sync, as I
don't follow dnsjava on a daily basis so am not always aware of the
latest release right away.

I could easily publish a placeholder under org.littleshoot if you
wanted, however -- DNSSEC4J would just grab that one, and others could
as well. I could shoot you the super simple pom.xml that would be
useful in any case.

-Adam


On Tue, Feb 21, 2012 at 6:58 PM, Brian Wellington <bwe...@xb...> wrote:
>
> On Feb 20, 2012, at 1:52 PM, Adam Fisk wrote:
>
>> I belatedly wanted to thank you for all these details, Brian. It's
>> still an early version that needs more extensive testing in the wild,
>> but the steps you described are integrated into DNSSEC4J at:
>>
>> https://github.com/adamfisk/DNSSEC4J
>
> Thanks!  Do you want me to add a link to this from dnsjava.org?
>
>> I know others have mentioned it, but I'll throw in my +1 for putting
>> dnsjava up at Sonatype for auto-syncing with Maven central -- would
>> make it much easier for third parties to integrate things like
>> DNSSEC4J as well. It's actually fairly easy to do and totally worth it
>> -- described at:
>>
>> https://docs.sonatype.org/display/Repository/Sonatype+OSS+Maven+Repository+Usage+Guide
>
> I don't use Maven, so I'm not planning to do this.  If anyone else wants to do this on a regular basis, I'm happy to let them.
>
> Brian
>
>
>
>> On Wed, Nov 23, 2011 at 4:32 PM, Brian Wellington <bwe...@xb...> wrote:
>>>
>>> On Nov 22, 2011, at 8:38 PM, Adam Fisk wrote:
>>>
>>>> Hi Brian- How do I modify the initial request message to request the
>>>> DNSSEC extended attributes? Should I be able to get those attributes
>>>> from a query to say gmail.com -- i.e can I just fire off queries to
>>>> bunch of the major sites at this point and expect DNSSEC responses?
>>>
>>> If you want to set it on all queries, you can call:
>>>
>>> Resolver.setEDNS(0, 0, ExtendedFlags.DO, null);
>>>
>>> on the resolver being used.  To set it on an individual query:
>>>
>>> opt = new OPTRecord(1280, 0, 1, ExtendedFlags.DO , null);
>>> query.addRecord(opt, Section.ADDITIONAL);
>>>
>>> Note that unless you send the query directly to an authoritative server, the response will only contain DNSSEC information if the caching server you're pointing to supports DNSSEC.  Whether this is true depends on how your ISP (or other DNS provider) has configured it.
>>>
>>> There are definitely major sites using DNSSEC, but gmail isn't one of them.  I'm sure that someone's maintaining a list, but I haven't looked.
>>>
>>> Brian
>>>
>

Re: DNSSSEC verification example

[Brian W. <bwe...@xb...>]

On Feb 20, 2012, at 1:52 PM, Adam Fisk wrote:

> I belatedly wanted to thank you for all these details, Brian. It's
> still an early version that needs more extensive testing in the wild,
> but the steps you described are integrated into DNSSEC4J at:
> 
> https://github.com/adamfisk/DNSSEC4J

Thanks!  Do you want me to add a link to this from dnsjava.org?

> I know others have mentioned it, but I'll throw in my +1 for putting
> dnsjava up at Sonatype for auto-syncing with Maven central -- would
> make it much easier for third parties to integrate things like
> DNSSEC4J as well. It's actually fairly easy to do and totally worth it
> -- described at:
> 
> https://docs.sonatype.org/display/Repository/Sonatype+OSS+Maven+Repository+Usage+Guide

I don't use Maven, so I'm not planning to do this.  If anyone else wants to do this on a regular basis, I'm happy to let them.

Brian



> On Wed, Nov 23, 2011 at 4:32 PM, Brian Wellington <bwe...@xb...> wrote:
>> 
>> On Nov 22, 2011, at 8:38 PM, Adam Fisk wrote:
>> 
>>> Hi Brian- How do I modify the initial request message to request the
>>> DNSSEC extended attributes? Should I be able to get those attributes
>>> from a query to say gmail.com -- i.e can I just fire off queries to
>>> bunch of the major sites at this point and expect DNSSEC responses?
>> 
>> If you want to set it on all queries, you can call:
>> 
>> Resolver.setEDNS(0, 0, ExtendedFlags.DO, null);
>> 
>> on the resolver being used.  To set it on an individual query:
>> 
>> opt = new OPTRecord(1280, 0, 1, ExtendedFlags.DO , null);
>> query.addRecord(opt, Section.ADDITIONAL);
>> 
>> Note that unless you send the query directly to an authoritative server, the response will only contain DNSSEC information if the caching server you're pointing to supports DNSSEC.  Whether this is true depends on how your ISP (or other DNS provider) has configured it.
>> 
>> There are definitely major sites using DNSSEC, but gmail isn't one of them.  I'm sure that someone's maintaining a list, but I haven't looked.
>> 
>> Brian
>> 

Re: DNSSSEC verification example

[Adam F. <af...@br...>]

I belatedly wanted to thank you for all these details, Brian. It's
still an early version that needs more extensive testing in the wild,
but the steps you described are integrated into DNSSEC4J at:

https://github.com/adamfisk/DNSSEC4J

I know others have mentioned it, but I'll throw in my +1 for putting
dnsjava up at Sonatype for auto-syncing with Maven central -- would
make it much easier for third parties to integrate things like
DNSSEC4J as well. It's actually fairly easy to do and totally worth it
-- described at:

https://docs.sonatype.org/display/Repository/Sonatype+OSS+Maven+Repository+Usage+Guide

All the Best,

-Adam


On Wed, Nov 23, 2011 at 4:32 PM, Brian Wellington <bwe...@xb...> wrote:
>
> On Nov 22, 2011, at 8:38 PM, Adam Fisk wrote:
>
>> Hi Brian- How do I modify the initial request message to request the
>> DNSSEC extended attributes? Should I be able to get those attributes
>> from a query to say gmail.com -- i.e can I just fire off queries to
>> bunch of the major sites at this point and expect DNSSEC responses?
>
> If you want to set it on all queries, you can call:
>
> Resolver.setEDNS(0, 0, ExtendedFlags.DO, null);
>
> on the resolver being used.  To set it on an individual query:
>
> opt = new OPTRecord(1280, 0, 1, ExtendedFlags.DO , null);
> query.addRecord(opt, Section.ADDITIONAL);
>
> Note that unless you send the query directly to an authoritative server, the response will only contain DNSSEC information if the caching server you're pointing to supports DNSSEC.  Whether this is true depends on how your ISP (or other DNS provider) has configured it.
>
> There are definitely major sites using DNSSEC, but gmail isn't one of them.  I'm sure that someone's maintaining a list, but I haven't looked.
>
> Brian
>

Re: Problem with zone transfers from UltraDNS

[Deven P. <dev...@gm...>]

Brian,

     Sorry, I was away on vacation for a while. I will try to see if 
running the AXFR first will resolve the issue for Ultra and let you know 
the result.

Deven

On 01/05/2012 06:41 PM, Brian Wellington wrote:
> On Jan 5, 2012, at 3:32 PM, Deven Phillips wrote:
>
>> The first is the IXFR from dnsjava, which as you said is rejected. The second is the AXFR fallback from dnsjava and that too fails. The final one is the AXFR initiated from "dig" and it works fine... The strange part is that I do zone transfers from BIND and MSDNS and others every day with the same code. What's different about Ultra?
> The IXFR isn't being rejected by the authoritative server.  I'm not sure what the client's doing with it, though.  The AXFR is being rejected.
>
> My guess is that what's happening is that the UltraDNS server is closing the connection after the IXFR completes, rather than leaving the connection open for more requests (as it should do).  This should be easy to test, by attempting to do an AXFR in the first place, rather than attempting to do an IXFR and falling back to AXFR.  I (obviously) can't test this theory without being able to send queries to the UltraDNS server.  If this is the case, it should be possible to change the code to establish a new connection when falling back to AXFR.
>
>> Thanks for looking into this!!
> Let's hope there's a solution…
>
> Brian
>
>
>> On 01/05/2012 12:55 AM, Brian Wellington wrote:
>>> On Jan 4, 2012, at 2:31 PM, Deven Phillips wrote:
>>>
>>>> Brian,
>>>>
>>>>     Unfortunately, the zone I am testing against belongs to a client and I cannot give you access to perform the XFR on that zone. The best I can do is the pcap. I have attached it here.
>>> Hmm.  I just wrote some code that imports the data from the pcap file, converts it into a list of DNS messages, and uses that list of messages instead of contacting the authoritative server.  The AXFR completed with no errors.
>>>
>>> Looking at the pcap, there appear to be 3 zone transfer attempts.  First, an IXFR.  Second, an AXFR that is immediately rejected.  Third, another AXFR.  I was using the responses from the third AXFR attempt.  Is that correct?
>>>
>>> Brian
>>>
>>>
>>>> On 01/04/2012 05:25 PM, Brian Wellington wrote:
>>>>> On Jan 4, 2012, at 2:20 PM, Deven Phillips wrote:
>>>>>
>>>>>> Well, the same code is working flawlessly against BIND servers, unbound servers, and MSDNS servers. So it definitely appears to be something specific to UltraDNS and their XFR implementation. If I were to send you a pcap, do you think you could see if there is something I am missing?
>>>>> I can definitely believe there's something wrong with UltraDNS's servers.  A pcap would probably help, but if you could provide a specific zone and IP address which are exhibiting the problem, that would be even better.
>>>>>
>>>>> Brian
>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Deven
>>>>>>
>>>>>> On 01/04/2012 02:10 PM, Brian Wellington wrote:
>>>>>>> On Jan 4, 2012, at 8:41 AM, Deven Phillips wrote:
>>>>>>>
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>>       I am attempting to use dnsjava to perform a zone transfer from
>>>>>>>> UltraDNS. I have verified that the zone transfers are allowed and
>>>>>>>> working from Ultra by using:
>>>>>>>>
>>>>>>>> dig @<Ultra IP>     mydomain.com. AXFR
>>>>>>>>
>>>>>>>> I get good results from the dig command, but when I try to use the
>>>>>>>> dnsjava library, I get the following:
>>>>>>>>
>>>>>>>> java.io.EOFException
>>>>>>>>          at org.xbill.DNS.TCPClient._recv(Unknown Source)
>>>>>>>>          at org.xbill.DNS.TCPClient.recv(Unknown Source)
>>>>>>>>          at org.xbill.DNS.ZoneTransferIn.doxfr(Unknown Source)
>>>>>>>>          at org.xbill.DNS.ZoneTransferIn.doxfr(Unknown Source)
>>>>>>>>          at org.xbill.DNS.ZoneTransferIn.run(Unknown Source)
>>>>>>>>          at com.zanclus.dns.InboundWorker.run(InboundWorker.java:432)
>>>>>>>>          at
>>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>>>>>>>>          at
>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>>>>>>>>          at java.lang.Thread.run(Thread.java:636)
>>>>>>>>
>>>>>>>> As best I can tell, the zone transfer connection is getting closed from
>>>>>>>> Ultra's side before the byte count has been fulfilled. The data is
>>>>>>>> complete, but I cannot access it because the Exception does not allow it
>>>>>>>> to be stored and returned in the ZoneTransferIn object. Any suggestions
>>>>>>>> on how to work around Ultra's broken protocol?
>>>>>>> If the problem was that not all the bytes were delivered, then the dig command would also fail, since dig also attempts to read full DNS messages before parsing them.  So would any other authoritative server attempting to act as a secondary, so I'm dubious that that's the problem you're seeing.
>>>>>>>
>>>>>>> You could try using the callback-based interface to AXFR in dnsjava 2.1.3 and see if you receive the entire contents before an Exception is raised; if that still doesn't work, I'm not sure what to tell you.  If UltraDNS's servers aren't obeying the protocol, I don't see how any AXFR client would work, as dnsjava isn't doing anything all that different.
>>>>>>>
>>>>>>> Brian
>>>>>> ------------------------------------------------------------------------------
>>>>>> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
>>>>>> infrastructure or vast IT resources to deliver seamless, secure access to
>>>>>> virtual desktops. With this all-in-one solution, easily deploy virtual
>>>>>> desktops for less than the cost of PCs and save 60% on VDI infrastructure
>>>>>> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox_______________________________________________
>>>>>> dnsjava-users mailing list
>>>>>> dns...@li...
>>>>>> https://lists.sourceforge.net/lists/listinfo/dnsjava-users
>>>> <ultradns.pcap>
>>

Re: Problem with zone transfers from UltraDNS

[Brian W. <bwe...@xb...>]

On Jan 5, 2012, at 3:32 PM, Deven Phillips wrote:

> The first is the IXFR from dnsjava, which as you said is rejected. The second is the AXFR fallback from dnsjava and that too fails. The final one is the AXFR initiated from "dig" and it works fine... The strange part is that I do zone transfers from BIND and MSDNS and others every day with the same code. What's different about Ultra?

The IXFR isn't being rejected by the authoritative server.  I'm not sure what the client's doing with it, though.  The AXFR is being rejected.

My guess is that what's happening is that the UltraDNS server is closing the connection after the IXFR completes, rather than leaving the connection open for more requests (as it should do).  This should be easy to test, by attempting to do an AXFR in the first place, rather than attempting to do an IXFR and falling back to AXFR.  I (obviously) can't test this theory without being able to send queries to the UltraDNS server.  If this is the case, it should be possible to change the code to establish a new connection when falling back to AXFR.

> Thanks for looking into this!!

Let's hope there's a solution…

Brian


> On 01/05/2012 12:55 AM, Brian Wellington wrote:
>> On Jan 4, 2012, at 2:31 PM, Deven Phillips wrote:
>> 
>>> Brian,
>>> 
>>>    Unfortunately, the zone I am testing against belongs to a client and I cannot give you access to perform the XFR on that zone. The best I can do is the pcap. I have attached it here.
>> Hmm.  I just wrote some code that imports the data from the pcap file, converts it into a list of DNS messages, and uses that list of messages instead of contacting the authoritative server.  The AXFR completed with no errors.
>> 
>> Looking at the pcap, there appear to be 3 zone transfer attempts.  First, an IXFR.  Second, an AXFR that is immediately rejected.  Third, another AXFR.  I was using the responses from the third AXFR attempt.  Is that correct?
>> 
>> Brian
>> 
>> 
>>> On 01/04/2012 05:25 PM, Brian Wellington wrote:
>>>> On Jan 4, 2012, at 2:20 PM, Deven Phillips wrote:
>>>> 
>>>>> Well, the same code is working flawlessly against BIND servers, unbound servers, and MSDNS servers. So it definitely appears to be something specific to UltraDNS and their XFR implementation. If I were to send you a pcap, do you think you could see if there is something I am missing?
>>>> I can definitely believe there's something wrong with UltraDNS's servers.  A pcap would probably help, but if you could provide a specific zone and IP address which are exhibiting the problem, that would be even better.
>>>> 
>>>> Brian
>>>> 
>>>>> Thanks,
>>>>> 
>>>>> Deven
>>>>> 
>>>>> On 01/04/2012 02:10 PM, Brian Wellington wrote:
>>>>>> On Jan 4, 2012, at 8:41 AM, Deven Phillips wrote:
>>>>>> 
>>>>>>> Hello,
>>>>>>> 
>>>>>>>      I am attempting to use dnsjava to perform a zone transfer from
>>>>>>> UltraDNS. I have verified that the zone transfers are allowed and
>>>>>>> working from Ultra by using:
>>>>>>> 
>>>>>>> dig @<Ultra IP>    mydomain.com. AXFR
>>>>>>> 
>>>>>>> I get good results from the dig command, but when I try to use the
>>>>>>> dnsjava library, I get the following:
>>>>>>> 
>>>>>>> java.io.EOFException
>>>>>>>         at org.xbill.DNS.TCPClient._recv(Unknown Source)
>>>>>>>         at org.xbill.DNS.TCPClient.recv(Unknown Source)
>>>>>>>         at org.xbill.DNS.ZoneTransferIn.doxfr(Unknown Source)
>>>>>>>         at org.xbill.DNS.ZoneTransferIn.doxfr(Unknown Source)
>>>>>>>         at org.xbill.DNS.ZoneTransferIn.run(Unknown Source)
>>>>>>>         at com.zanclus.dns.InboundWorker.run(InboundWorker.java:432)
>>>>>>>         at
>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>>>>>>>         at
>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>>>>>>>         at java.lang.Thread.run(Thread.java:636)
>>>>>>> 
>>>>>>> As best I can tell, the zone transfer connection is getting closed from
>>>>>>> Ultra's side before the byte count has been fulfilled. The data is
>>>>>>> complete, but I cannot access it because the Exception does not allow it
>>>>>>> to be stored and returned in the ZoneTransferIn object. Any suggestions
>>>>>>> on how to work around Ultra's broken protocol?
>>>>>> If the problem was that not all the bytes were delivered, then the dig command would also fail, since dig also attempts to read full DNS messages before parsing them.  So would any other authoritative server attempting to act as a secondary, so I'm dubious that that's the problem you're seeing.
>>>>>> 
>>>>>> You could try using the callback-based interface to AXFR in dnsjava 2.1.3 and see if you receive the entire contents before an Exception is raised; if that still doesn't work, I'm not sure what to tell you.  If UltraDNS's servers aren't obeying the protocol, I don't see how any AXFR client would work, as dnsjava isn't doing anything all that different.
>>>>>> 
>>>>>> Brian
>>>>> ------------------------------------------------------------------------------
>>>>> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
>>>>> infrastructure or vast IT resources to deliver seamless, secure access to
>>>>> virtual desktops. With this all-in-one solution, easily deploy virtual
>>>>> desktops for less than the cost of PCs and save 60% on VDI infrastructure
>>>>> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox_______________________________________________
>>>>> dnsjava-users mailing list
>>>>> dns...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/dnsjava-users
>>> <ultradns.pcap>
> 
> 

Re: Problem with zone transfers from UltraDNS

[Deven P. <dev...@gm...>]

The first is the IXFR from dnsjava, which as you said is rejected. The 
second is the AXFR fallback from dnsjava and that too fails. The final 
one is the AXFR initiated from "dig" and it works fine... The strange 
part is that I do zone transfers from BIND and MSDNS and others every 
day with the same code. What's different about Ultra?

Thanks for looking into this!!

Deven

On 01/05/2012 12:55 AM, Brian Wellington wrote:
> On Jan 4, 2012, at 2:31 PM, Deven Phillips wrote:
>
>> Brian,
>>
>>     Unfortunately, the zone I am testing against belongs to a client and I cannot give you access to perform the XFR on that zone. The best I can do is the pcap. I have attached it here.
> Hmm.  I just wrote some code that imports the data from the pcap file, converts it into a list of DNS messages, and uses that list of messages instead of contacting the authoritative server.  The AXFR completed with no errors.
>
> Looking at the pcap, there appear to be 3 zone transfer attempts.  First, an IXFR.  Second, an AXFR that is immediately rejected.  Third, another AXFR.  I was using the responses from the third AXFR attempt.  Is that correct?
>
> Brian
>
>
>> On 01/04/2012 05:25 PM, Brian Wellington wrote:
>>> On Jan 4, 2012, at 2:20 PM, Deven Phillips wrote:
>>>
>>>> Well, the same code is working flawlessly against BIND servers, unbound servers, and MSDNS servers. So it definitely appears to be something specific to UltraDNS and their XFR implementation. If I were to send you a pcap, do you think you could see if there is something I am missing?
>>> I can definitely believe there's something wrong with UltraDNS's servers.  A pcap would probably help, but if you could provide a specific zone and IP address which are exhibiting the problem, that would be even better.
>>>
>>> Brian
>>>
>>>> Thanks,
>>>>
>>>> Deven
>>>>
>>>> On 01/04/2012 02:10 PM, Brian Wellington wrote:
>>>>> On Jan 4, 2012, at 8:41 AM, Deven Phillips wrote:
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>>       I am attempting to use dnsjava to perform a zone transfer from
>>>>>> UltraDNS. I have verified that the zone transfers are allowed and
>>>>>> working from Ultra by using:
>>>>>>
>>>>>> dig @<Ultra IP>    mydomain.com. AXFR
>>>>>>
>>>>>> I get good results from the dig command, but when I try to use the
>>>>>> dnsjava library, I get the following:
>>>>>>
>>>>>> java.io.EOFException
>>>>>>          at org.xbill.DNS.TCPClient._recv(Unknown Source)
>>>>>>          at org.xbill.DNS.TCPClient.recv(Unknown Source)
>>>>>>          at org.xbill.DNS.ZoneTransferIn.doxfr(Unknown Source)
>>>>>>          at org.xbill.DNS.ZoneTransferIn.doxfr(Unknown Source)
>>>>>>          at org.xbill.DNS.ZoneTransferIn.run(Unknown Source)
>>>>>>          at com.zanclus.dns.InboundWorker.run(InboundWorker.java:432)
>>>>>>          at
>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>>>>>>          at
>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>>>>>>          at java.lang.Thread.run(Thread.java:636)
>>>>>>
>>>>>> As best I can tell, the zone transfer connection is getting closed from
>>>>>> Ultra's side before the byte count has been fulfilled. The data is
>>>>>> complete, but I cannot access it because the Exception does not allow it
>>>>>> to be stored and returned in the ZoneTransferIn object. Any suggestions
>>>>>> on how to work around Ultra's broken protocol?
>>>>> If the problem was that not all the bytes were delivered, then the dig command would also fail, since dig also attempts to read full DNS messages before parsing them.  So would any other authoritative server attempting to act as a secondary, so I'm dubious that that's the problem you're seeing.
>>>>>
>>>>> You could try using the callback-based interface to AXFR in dnsjava 2.1.3 and see if you receive the entire contents before an Exception is raised; if that still doesn't work, I'm not sure what to tell you.  If UltraDNS's servers aren't obeying the protocol, I don't see how any AXFR client would work, as dnsjava isn't doing anything all that different.
>>>>>
>>>>> Brian
>>>> ------------------------------------------------------------------------------
>>>> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
>>>> infrastructure or vast IT resources to deliver seamless, secure access to
>>>> virtual desktops. With this all-in-one solution, easily deploy virtual
>>>> desktops for less than the cost of PCs and save 60% on VDI infrastructure
>>>> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox_______________________________________________
>>>> dnsjava-users mailing list
>>>> dns...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/dnsjava-users
>> <ultradns.pcap>

Re: Problem with zone transfers from UltraDNS

[Brian W. <bwe...@xb...>]

On Jan 4, 2012, at 2:31 PM, Deven Phillips wrote:

> Brian,
> 
>    Unfortunately, the zone I am testing against belongs to a client and I cannot give you access to perform the XFR on that zone. The best I can do is the pcap. I have attached it here.

Hmm.  I just wrote some code that imports the data from the pcap file, converts it into a list of DNS messages, and uses that list of messages instead of contacting the authoritative server.  The AXFR completed with no errors.

Looking at the pcap, there appear to be 3 zone transfer attempts.  First, an IXFR.  Second, an AXFR that is immediately rejected.  Third, another AXFR.  I was using the responses from the third AXFR attempt.  Is that correct?

Brian


> On 01/04/2012 05:25 PM, Brian Wellington wrote:
>> On Jan 4, 2012, at 2:20 PM, Deven Phillips wrote:
>> 
>>> Well, the same code is working flawlessly against BIND servers, unbound servers, and MSDNS servers. So it definitely appears to be something specific to UltraDNS and their XFR implementation. If I were to send you a pcap, do you think you could see if there is something I am missing?
>> I can definitely believe there's something wrong with UltraDNS's servers.  A pcap would probably help, but if you could provide a specific zone and IP address which are exhibiting the problem, that would be even better.
>> 
>> Brian
>> 
>>> Thanks,
>>> 
>>> Deven
>>> 
>>> On 01/04/2012 02:10 PM, Brian Wellington wrote:
>>>> On Jan 4, 2012, at 8:41 AM, Deven Phillips wrote:
>>>> 
>>>>> Hello,
>>>>> 
>>>>>      I am attempting to use dnsjava to perform a zone transfer from
>>>>> UltraDNS. I have verified that the zone transfers are allowed and
>>>>> working from Ultra by using:
>>>>> 
>>>>> dig @<Ultra IP>   mydomain.com. AXFR
>>>>> 
>>>>> I get good results from the dig command, but when I try to use the
>>>>> dnsjava library, I get the following:
>>>>> 
>>>>> java.io.EOFException
>>>>>         at org.xbill.DNS.TCPClient._recv(Unknown Source)
>>>>>         at org.xbill.DNS.TCPClient.recv(Unknown Source)
>>>>>         at org.xbill.DNS.ZoneTransferIn.doxfr(Unknown Source)
>>>>>         at org.xbill.DNS.ZoneTransferIn.doxfr(Unknown Source)
>>>>>         at org.xbill.DNS.ZoneTransferIn.run(Unknown Source)
>>>>>         at com.zanclus.dns.InboundWorker.run(InboundWorker.java:432)
>>>>>         at
>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>>>>>         at
>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>>>>>         at java.lang.Thread.run(Thread.java:636)
>>>>> 
>>>>> As best I can tell, the zone transfer connection is getting closed from
>>>>> Ultra's side before the byte count has been fulfilled. The data is
>>>>> complete, but I cannot access it because the Exception does not allow it
>>>>> to be stored and returned in the ZoneTransferIn object. Any suggestions
>>>>> on how to work around Ultra's broken protocol?
>>>> If the problem was that not all the bytes were delivered, then the dig command would also fail, since dig also attempts to read full DNS messages before parsing them.  So would any other authoritative server attempting to act as a secondary, so I'm dubious that that's the problem you're seeing.
>>>> 
>>>> You could try using the callback-based interface to AXFR in dnsjava 2.1.3 and see if you receive the entire contents before an Exception is raised; if that still doesn't work, I'm not sure what to tell you.  If UltraDNS's servers aren't obeying the protocol, I don't see how any AXFR client would work, as dnsjava isn't doing anything all that different.
>>>> 
>>>> Brian
>>> 
>>> ------------------------------------------------------------------------------
>>> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
>>> infrastructure or vast IT resources to deliver seamless, secure access to
>>> virtual desktops. With this all-in-one solution, easily deploy virtual
>>> desktops for less than the cost of PCs and save 60% on VDI infrastructure
>>> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox_______________________________________________
>>> dnsjava-users mailing list
>>> dns...@li...
>>> https://lists.sourceforge.net/lists/listinfo/dnsjava-users
> 
> <ultradns.pcap>

Re: Problem with zone transfers from UltraDNS

[Deven P. <dev...@gm...>]

Brian,

     Unfortunately, the zone I am testing against belongs to a client 
and I cannot give you access to perform the XFR on that zone. The best I 
can do is the pcap. I have attached it here.

Deven

On 01/04/2012 05:25 PM, Brian Wellington wrote:
> On Jan 4, 2012, at 2:20 PM, Deven Phillips wrote:
>
>> Well, the same code is working flawlessly against BIND servers, unbound servers, and MSDNS servers. So it definitely appears to be something specific to UltraDNS and their XFR implementation. If I were to send you a pcap, do you think you could see if there is something I am missing?
> I can definitely believe there's something wrong with UltraDNS's servers.  A pcap would probably help, but if you could provide a specific zone and IP address which are exhibiting the problem, that would be even better.
>
> Brian
>
>> Thanks,
>>
>> Deven
>>
>> On 01/04/2012 02:10 PM, Brian Wellington wrote:
>>> On Jan 4, 2012, at 8:41 AM, Deven Phillips wrote:
>>>
>>>> Hello,
>>>>
>>>>       I am attempting to use dnsjava to perform a zone transfer from
>>>> UltraDNS. I have verified that the zone transfers are allowed and
>>>> working from Ultra by using:
>>>>
>>>> dig @<Ultra IP>   mydomain.com. AXFR
>>>>
>>>> I get good results from the dig command, but when I try to use the
>>>> dnsjava library, I get the following:
>>>>
>>>> java.io.EOFException
>>>>          at org.xbill.DNS.TCPClient._recv(Unknown Source)
>>>>          at org.xbill.DNS.TCPClient.recv(Unknown Source)
>>>>          at org.xbill.DNS.ZoneTransferIn.doxfr(Unknown Source)
>>>>          at org.xbill.DNS.ZoneTransferIn.doxfr(Unknown Source)
>>>>          at org.xbill.DNS.ZoneTransferIn.run(Unknown Source)
>>>>          at com.zanclus.dns.InboundWorker.run(InboundWorker.java:432)
>>>>          at
>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>>>>          at
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>>>>          at java.lang.Thread.run(Thread.java:636)
>>>>
>>>> As best I can tell, the zone transfer connection is getting closed from
>>>> Ultra's side before the byte count has been fulfilled. The data is
>>>> complete, but I cannot access it because the Exception does not allow it
>>>> to be stored and returned in the ZoneTransferIn object. Any suggestions
>>>> on how to work around Ultra's broken protocol?
>>> If the problem was that not all the bytes were delivered, then the dig command would also fail, since dig also attempts to read full DNS messages before parsing them.  So would any other authoritative server attempting to act as a secondary, so I'm dubious that that's the problem you're seeing.
>>>
>>> You could try using the callback-based interface to AXFR in dnsjava 2.1.3 and see if you receive the entire contents before an Exception is raised; if that still doesn't work, I'm not sure what to tell you.  If UltraDNS's servers aren't obeying the protocol, I don't see how any AXFR client would work, as dnsjava isn't doing anything all that different.
>>>
>>> Brian
>>
>> ------------------------------------------------------------------------------
>> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
>> infrastructure or vast IT resources to deliver seamless, secure access to
>> virtual desktops. With this all-in-one solution, easily deploy virtual
>> desktops for less than the cost of PCs and save 60% on VDI infrastructure
>> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox_______________________________________________
>> dnsjava-users mailing list
>> dns...@li...
>> https://lists.sourceforge.net/lists/listinfo/dnsjava-users

Re: Problem with zone transfers from UltraDNS

[Brian W. <bwe...@xb...>]

On Jan 4, 2012, at 2:20 PM, Deven Phillips wrote:

> Well, the same code is working flawlessly against BIND servers, unbound servers, and MSDNS servers. So it definitely appears to be something specific to UltraDNS and their XFR implementation. If I were to send you a pcap, do you think you could see if there is something I am missing?

I can definitely believe there's something wrong with UltraDNS's servers.  A pcap would probably help, but if you could provide a specific zone and IP address which are exhibiting the problem, that would be even better.

Brian

> Thanks,
> 
> Deven
> 
> On 01/04/2012 02:10 PM, Brian Wellington wrote:
>> On Jan 4, 2012, at 8:41 AM, Deven Phillips wrote:
>> 
>>> Hello,
>>> 
>>>      I am attempting to use dnsjava to perform a zone transfer from
>>> UltraDNS. I have verified that the zone transfers are allowed and
>>> working from Ultra by using:
>>> 
>>> dig @<Ultra IP>  mydomain.com. AXFR
>>> 
>>> I get good results from the dig command, but when I try to use the
>>> dnsjava library, I get the following:
>>> 
>>> java.io.EOFException
>>>         at org.xbill.DNS.TCPClient._recv(Unknown Source)
>>>         at org.xbill.DNS.TCPClient.recv(Unknown Source)
>>>         at org.xbill.DNS.ZoneTransferIn.doxfr(Unknown Source)
>>>         at org.xbill.DNS.ZoneTransferIn.doxfr(Unknown Source)
>>>         at org.xbill.DNS.ZoneTransferIn.run(Unknown Source)
>>>         at com.zanclus.dns.InboundWorker.run(InboundWorker.java:432)
>>>         at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>>>         at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>>>         at java.lang.Thread.run(Thread.java:636)
>>> 
>>> As best I can tell, the zone transfer connection is getting closed from
>>> Ultra's side before the byte count has been fulfilled. The data is
>>> complete, but I cannot access it because the Exception does not allow it
>>> to be stored and returned in the ZoneTransferIn object. Any suggestions
>>> on how to work around Ultra's broken protocol?
>> If the problem was that not all the bytes were delivered, then the dig command would also fail, since dig also attempts to read full DNS messages before parsing them.  So would any other authoritative server attempting to act as a secondary, so I'm dubious that that's the problem you're seeing.
>> 
>> You could try using the callback-based interface to AXFR in dnsjava 2.1.3 and see if you receive the entire contents before an Exception is raised; if that still doesn't work, I'm not sure what to tell you.  If UltraDNS's servers aren't obeying the protocol, I don't see how any AXFR client would work, as dnsjava isn't doing anything all that different.
>> 
>> Brian
> 
> 
> ------------------------------------------------------------------------------
> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
> infrastructure or vast IT resources to deliver seamless, secure access to
> virtual desktops. With this all-in-one solution, easily deploy virtual 
> desktops for less than the cost of PCs and save 60% on VDI infrastructure 
> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox_______________________________________________
> dnsjava-users mailing list
> dns...@li...
> https://lists.sourceforge.net/lists/listinfo/dnsjava-users

Re: Problem with zone transfers from UltraDNS

[Deven P. <dev...@gm...>]

Well, the same code is working flawlessly against BIND servers, unbound 
servers, and MSDNS servers. So it definitely appears to be something 
specific to UltraDNS and their XFR implementation. If I were to send you 
a pcap, do you think you could see if there is something I am missing?

Thanks,

Deven

On 01/04/2012 02:10 PM, Brian Wellington wrote:
> On Jan 4, 2012, at 8:41 AM, Deven Phillips wrote:
>
>> Hello,
>>
>>       I am attempting to use dnsjava to perform a zone transfer from
>> UltraDNS. I have verified that the zone transfers are allowed and
>> working from Ultra by using:
>>
>> dig @<Ultra IP>  mydomain.com. AXFR
>>
>> I get good results from the dig command, but when I try to use the
>> dnsjava library, I get the following:
>>
>> java.io.EOFException
>>          at org.xbill.DNS.TCPClient._recv(Unknown Source)
>>          at org.xbill.DNS.TCPClient.recv(Unknown Source)
>>          at org.xbill.DNS.ZoneTransferIn.doxfr(Unknown Source)
>>          at org.xbill.DNS.ZoneTransferIn.doxfr(Unknown Source)
>>          at org.xbill.DNS.ZoneTransferIn.run(Unknown Source)
>>          at com.zanclus.dns.InboundWorker.run(InboundWorker.java:432)
>>          at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>>          at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>>          at java.lang.Thread.run(Thread.java:636)
>>
>> As best I can tell, the zone transfer connection is getting closed from
>> Ultra's side before the byte count has been fulfilled. The data is
>> complete, but I cannot access it because the Exception does not allow it
>> to be stored and returned in the ZoneTransferIn object. Any suggestions
>> on how to work around Ultra's broken protocol?
> If the problem was that not all the bytes were delivered, then the dig command would also fail, since dig also attempts to read full DNS messages before parsing them.  So would any other authoritative server attempting to act as a secondary, so I'm dubious that that's the problem you're seeing.
>
> You could try using the callback-based interface to AXFR in dnsjava 2.1.3 and see if you receive the entire contents before an Exception is raised; if that still doesn't work, I'm not sure what to tell you.  If UltraDNS's servers aren't obeying the protocol, I don't see how any AXFR client would work, as dnsjava isn't doing anything all that different.
>
> Brian

Re: Problem with zone transfers from UltraDNS

[Brian W. <bwe...@xb...>]

On Jan 4, 2012, at 8:41 AM, Deven Phillips wrote:

> Hello,
> 
>      I am attempting to use dnsjava to perform a zone transfer from 
> UltraDNS. I have verified that the zone transfers are allowed and 
> working from Ultra by using:
> 
> dig @<Ultra IP> mydomain.com. AXFR
> 
> I get good results from the dig command, but when I try to use the 
> dnsjava library, I get the following:
> 
> java.io.EOFException
>         at org.xbill.DNS.TCPClient._recv(Unknown Source)
>         at org.xbill.DNS.TCPClient.recv(Unknown Source)
>         at org.xbill.DNS.ZoneTransferIn.doxfr(Unknown Source)
>         at org.xbill.DNS.ZoneTransferIn.doxfr(Unknown Source)
>         at org.xbill.DNS.ZoneTransferIn.run(Unknown Source)
>         at com.zanclus.dns.InboundWorker.run(InboundWorker.java:432)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:636)
> 
> As best I can tell, the zone transfer connection is getting closed from 
> Ultra's side before the byte count has been fulfilled. The data is 
> complete, but I cannot access it because the Exception does not allow it 
> to be stored and returned in the ZoneTransferIn object. Any suggestions 
> on how to work around Ultra's broken protocol?

If the problem was that not all the bytes were delivered, then the dig command would also fail, since dig also attempts to read full DNS messages before parsing them.  So would any other authoritative server attempting to act as a secondary, so I'm dubious that that's the problem you're seeing.

You could try using the callback-based interface to AXFR in dnsjava 2.1.3 and see if you receive the entire contents before an Exception is raised; if that still doesn't work, I'm not sure what to tell you.  If UltraDNS's servers aren't obeying the protocol, I don't see how any AXFR client would work, as dnsjava isn't doing anything all that different.

Brian

Problem with zone transfers from UltraDNS

[Deven P. <dev...@gm...>]

Hello,

      I am attempting to use dnsjava to perform a zone transfer from 
UltraDNS. I have verified that the zone transfers are allowed and 
working from Ultra by using:

dig @<Ultra IP> mydomain.com. AXFR

I get good results from the dig command, but when I try to use the 
dnsjava library, I get the following:

java.io.EOFException
         at org.xbill.DNS.TCPClient._recv(Unknown Source)
         at org.xbill.DNS.TCPClient.recv(Unknown Source)
         at org.xbill.DNS.ZoneTransferIn.doxfr(Unknown Source)
         at org.xbill.DNS.ZoneTransferIn.doxfr(Unknown Source)
         at org.xbill.DNS.ZoneTransferIn.run(Unknown Source)
         at com.zanclus.dns.InboundWorker.run(InboundWorker.java:432)
         at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
         at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
         at java.lang.Thread.run(Thread.java:636)

As best I can tell, the zone transfer connection is getting closed from 
Ultra's side before the byte count has been fulfilled. The data is 
complete, but I cannot access it because the Exception does not allow it 
to be stored and returned in the ZoneTransferIn object. Any suggestions 
on how to work around Ultra's broken protocol?

Thanks,

Deven

RE: Avoid server lookups for localhost in SPI and PTR queries

[Bauersachs I. <ing...@fh...>]

Thanks for looking into it!

>> I'm trying to integrate DNSJAVA's SPI in Jitsi and had a few issues:
>> - Some JVM internals (seed generation for randomness) call
>> InetAddress.getLocalHost. This causes a lookup to the configured DNS server,
>> which usually doesn't know of localhost (or on Windows, my machine name).
> 
> I'm not sure if this is usually the case.  It's the case sometimes, but it
> depends on a lot of things.

Windows, Ubuntu, OpenSuse and Fedora in default installations all return the hostname (not localhost), so this is where my "usually" came from. I don't know about other (non-Sun/OpenJDK) VMs.

>> - SSL/TLS sockets try to obtain the hostname from InetAddress objects,
>> which causes PTR query when the InetAdresses object was not pre-initialized
>> with the hostname.
>> 
>> Attached is a patch to DNSJAVA that solves these issues. Would you mind
>> applying it to trunk?
> 
> I think some of it can be applied, but not all.
> 
> The changes to ARecord/AAAARecord look ok.  I believe someone else suggested
> similar changes at some point, and I don't think there are any downsides.
> I'll commit this part.

That would be nice, thanks! :-)

> The change in Address might be ok, but I don't know why setting the hostname
> to a literal address representation is an improvement over not being set.  If
> it is a good idea, then all the other places in Address doing the same things
> should also be changed.

The benefit over not specifying it, is avoiding a PTR query. If someone indeed does want a PTR query it could be sent through the library or obtained with InetAddress.getCanonicalHostName().
It of course might depend on the point of view of whether avoiding a PTR query is an advantage or not. As there are sometimes problems with unnecessary delays due to these PTRs, I see more advantage than disadvantage.
I agree that the other methods should then also be changed.
 
> The SPI changes are not backwards compatible in the case where a lookup of
> localhost would work, which is almost always the case on non-Windows machine,
> and often the case on Windows machines.  Specifically, an application looking
> up "localhost" might get any combination of 127.0.0.1, ::1, or a routable
> address, depending on the configuration of the machine and the local DNS
> server.  This also uses undocumented internal classes, and I have no way of
> knowing whether the code would work at all across the set of supported
> platforms and Java versions.

That is indeed true :(
Would it be an option to make this workaround configurable through a system property?

Problem is: without such a check there are huge delays and the SPI becomes almost unusable. I updated the change (attached) so that it uses the default addresses (those returned without dnsjava as an SPI) for a query to localhost, so the results should cover the variety of possible localhost answers.

I'm not sure whether it's much of a problem to use those internal classes. Generally it's bad of course, but the whole SPI implementation is already based on something Sun/Oracle declared as internal (sun.* namespace and "These properties may not be supported in future releases.") A failure to access those internals would only result in the default behavior of asking the server.

>> More of a general question: The project uses a very unusual file-layout for
>> Java projects and the Ant buildscript also needed some modifications to get
>> the stuff running. Would it be an option to bring that in a more "standard"-
>> compliant layout? I'd help of course if you're interested. :-)
> 
> The dnsjava code base precedes whichever standard you're talking about, and
> in the 13 years or so that I've been developing it, there have been more
> standards than I can count.  I don't see any reason to change everything to
> comply with the current one.

I guess there is no real standard, that's why I put it into quotes. Forget it :-)

> Brian

Ingo

Re: Avoid server lookups for localhost in SPI and PTR queries

[Brian W. <bwe...@xb...>]

On Nov 29, 2011, at 5:41 AM, Bauersachs Ingo wrote:

> Hey Brian
> 
> I'm trying to integrate DNSJAVA's SPI in Jitsi and had a few issues:
> - Some JVM internals (seed generation for randomness) call InetAddress.getLocalHost. This causes a lookup to the configured DNS server, which usually doesn't know of localhost (or on Windows, my machine name).

I'm not sure if this is usually the case.  It's the case sometimes, but it depends on a lot of things.

> - SSL/TLS sockets try to obtain the hostname from InetAddress objects, which causes PTR query when the InetAdresses object was not pre-initialized with the hostname.
> 
> Attached is a patch to DNSJAVA that solves these issues. Would you mind applying it to trunk?

I think some of it can be applied, but not all.

The changes to ARecord/AAAARecord look ok.  I believe someone else suggested similar changes at some point, and I don't think there are any downsides.  I'll commit this part.

The change in Address might be ok, but I don't know why setting the hostname to a literal address representation is an improvement over not being set.  If it is a good idea, then all the other places in Address doing the same things should also be changed.

The SPI changes are not backwards compatible in the case where a lookup of localhost would work, which is almost always the case on non-Windows machine, and often the case on Windows machines.  Specifically, an application looking up "localhost" might get any combination of 127.0.0.1, ::1, or a routable address, depending on the configuration of the machine and the local DNS server.  This also uses undocumented internal classes, and I have no way of knowing whether the code would work at all across the set of supported platforms and Java versions.

> The unit tests all ran fine (apart from DNSSECWithLunaProviderTest which I excluded because of the dependency; and an IP of a colorado.edu server changed).
> 
> More of a general question: The project uses a very unusual file-layout for Java projects and the Ant buildscript also needed some modifications to get the stuff running. Would it be an option to bring that in a more "standard"-compliant layout? I'd help of course if you're interested. :-)

The dnsjava code base precedes whichever standard you're talking about, and in the 13 years or so that I've been developing it, there have been more standards than I can count.  I don't see any reason to change everything to comply with the current one.

Brian

Avoid server lookups for localhost in SPI and PTR queries

[Bauersachs I. <ing...@fh...>]

Hey Brian

I'm trying to integrate DNSJAVA's SPI in Jitsi and had a few issues:
- Some JVM internals (seed generation for randomness) call InetAddress.getLocalHost. This causes a lookup to the configured DNS server, which usually doesn't know of localhost (or on Windows, my machine name).
- SSL/TLS sockets try to obtain the hostname from InetAddress objects, which causes PTR query when the InetAdresses object was not pre-initialized with the hostname.

Attached is a patch to DNSJAVA that solves these issues. Would you mind applying it to trunk?
The unit tests all ran fine (apart from DNSSECWithLunaProviderTest which I excluded because of the dependency; and an IP of a colorado.edu server changed).

More of a general question: The project uses a very unusual file-layout for Java projects and the Ant buildscript also needed some modifications to get the stuff running. Would it be an option to bring that in a more "standard"-compliant layout? I'd help of course if you're interested. :-)

Regards,
Ingo

Re: DNSSSEC verification example

[Brian W. <bwe...@xb...>]

On Nov 22, 2011, at 8:38 PM, Adam Fisk wrote:

> Hi Brian- How do I modify the initial request message to request the
> DNSSEC extended attributes? Should I be able to get those attributes
> from a query to say gmail.com -- i.e can I just fire off queries to
> bunch of the major sites at this point and expect DNSSEC responses?

If you want to set it on all queries, you can call:

Resolver.setEDNS(0, 0, ExtendedFlags.DO, null);

on the resolver being used.  To set it on an individual query:

opt = new OPTRecord(1280, 0, 1, ExtendedFlags.DO , null);
query.addRecord(opt, Section.ADDITIONAL);

Note that unless you send the query directly to an authoritative server, the response will only contain DNSSEC information if the caching server you're pointing to supports DNSSEC.  Whether this is true depends on how your ISP (or other DNS provider) has configured it.

There are definitely major sites using DNSSEC, but gmail isn't one of them.  I'm sure that someone's maintaining a list, but I haven't looked.

Brian