#119 CacheInstance leaks sockets when DNS fails

[Janet]

Hi Steve,

When DNS fails, CacheInstance appears to be opening a socket, failing, and leaving it open. This is recurring many times, exhausting file descriptors.

(Its exceptions just go to /dev/null instead of a log file?)

Extracts from strace

socket(PF_INET6, SOCK_STREAM, IPPROTO_IP) = 647
write(1, "java.rmi.RemoteException: VI SDK"..., 99) = 99
write(1, "\n", 1) = 1
write(1, "\tat com.vmware.vim25.ws.WSClient"..., 58) = 58
write(1, "\n", 1) = 1
write(1, "\tat com.vmware.vim25.ws.WSClient"..., 58) = 58
write(1, "\n", 1) = 1
write(1, "\tat com.vmware.vim25.ws.VimStub."..., 63) = 63
write(1, "\n", 1) = 1
write(1, "\tat com.vmware.vim25.mo.Property"..., 84) = 84
write(1, "\n", 1) = 1
write(1, "\tat com.vmware.vim.cf.ManagedObj"..., 76) = 76
write(1, "\n", 1) = 1
write(1, "\tat java.lang.Thread.run(Unknown"..., 40) = 40
write(1, "\n", 1) = 1

Some socket numbers do get reused... (the garbage collector must be freeing them?) but eventually we do run out, sometimes only temporarily.

79164:socket(PF_INET6, SOCK_STREAM, IPPROTO_IP) = 1018
79179:socket(PF_INET6, SOCK_STREAM, IPPROTO_IP) = 1019
79194:socket(PF_INET6, SOCK_STREAM, IPPROTO_IP) = 1020
79209:socket(PF_INET6, SOCK_STREAM, IPPROTO_IP) = 1021
79224:socket(PF_INET6, SOCK_STREAM, IPPROTO_IP) = 1022
79239:socket(PF_INET6, SOCK_STREAM, IPPROTO_IP) = 1023
79254:socket(PF_INET6, SOCK_STREAM, IPPROTO_IP) = -1 EMFILE (Too many open files)
79269:socket(PF_INET6, SOCK_STREAM, IPPROTO_IP) = -1 EMFILE (Too many open files)
79284:socket(PF_INET6, SOCK_STREAM, IPPROTO_IP) = -1 EMFILE (Too many open files)
79299:socket(PF_INET6, SOCK_STREAM, IPPROTO_IP) = -1 EMFILE (Too many open files)
...
79648:socket(PF_INET6, SOCK_STREAM, IPPROTO_IP) = -1 EMFILE (Too many open files)
79663:socket(PF_INET6, SOCK_STREAM, IPPROTO_IP) = 267
79678:socket(PF_INET6, SOCK_STREAM, IPPROTO_IP) = 450
79693:socket(PF_INET6, SOCK_STREAM, IPPROTO_IP) = 806
79708:socket(PF_INET6, SOCK_STREAM, IPPROTO_IP) = 998
79723:socket(PF_INET6, SOCK_STREAM, IPPROTO_IP) = 989
79738:socket(PF_INET6, SOCK_STREAM, IPPROTO_IP) = 980

Thanks
Janet

[Steve]

This a bit tough bug to fix because the socket is not exposed to HttpsURLConnection. Do you have stack trace?
Also, you can probably work around this by using IP address directly.

Steve