Re: [Simpleweb-Support] simple-4.1.10 is leaking file handles

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Hi,

If you do not pipe lsof through wc -l you will see the file handles point to libraries loaded by the JVM. These are not open sockets. I ran your test on Mac OS X and it stops at 58. These are the various shared libraries. It will not grow or leak. Thanks for the feedback though, its appreciated.

Thanks,
Niall

--- On Fri, 4/3/09, Christophe Roudet <cr...@ac...> wrote:

> From: Christophe Roudet <cr...@ac...>
> Subject: Re: [Simpleweb-Support] simple-4.1.10 is leaking file handles
> To: "'Simple support and user issues'" <sim...@li...>
> Date: Friday, April 3, 2009, 12:44 PM
> I don't see any problem when watching
> netstat but lsof reports ever growing
> number of file handles.
> 
> I have attached a small example
> java -cp bin:lib/simple-4.1.9.jar
> org.simpleframework.ServerTest
> will launch the server on port 900
> 
> java -cp bin org.simpleframework.ClientTest <host>
> will run a client
> 
> initially for the server, I have:
> >lsof -p 3040 | wc -l
> 42
> 
> Then each time I run the client test
> >lsof -p 3040 | wc -l
> 49
> next
> >lsof -p 3040 | wc -l
> 55
> 
> And so on.
> 
> It doesn't look like a connection problem.
> 
> 
> > -----Original Message-----
> > From: Niall Gallagher [mailto:gal...@ya...]
> > Sent: Friday, April 03, 2009 2:16 PM
> > To: Simple support and user issues
> > Subject: Re: [Simpleweb-Support] simple-4.1.10 is
> leaking file handles
> > 
> > 
> > Hi,
> > 
> > I really don't think there is a file leak here. I have
> just performed
> > 10,000,000 requests with apache bench using HTTP/1.0
> (non persistent)
> > connections. and 100,000,000 persistent connections
> with apache bench.
> > Everything was fine. On both Linux and Windows.
> > 
> > I think you are either not closing the connection or
> not expecting
> > "Transfer-Encoding: chunked" responses which will be
> maintained. Such
> > connections should be closed from the client side.
> Also, Simple will close
> > all dormant connections after two minutes. Regardless,
> so all resources
> > will be cleaned up eventually.
> > 
> > If you can prove me wrong, by means of a unit test. I
> would be very
> > grateful. However, I really doubt there is a
> connection leak. In
> > particular the changes between 4.1.9 and 4.1.10 do not
> really deal with
> > connections.
> > 
> > Niall
> > 
> > --- On Fri, 4/3/09, Nia...@ub...
> <Nia...@ub...>
> > wrote:
> > 
> > > From: Nia...@ub...
> <Nia...@ub...>
> > > Subject: Re: [Simpleweb-Support] simple-4.1.10 is
> leaking file handles
> > > To: sim...@li...
> > > Date: Friday, April 3, 2009, 10:14 AM
> > > Hi,
> > >
> > > Can you try "netstat -P tcp" to get the actual
> TCP
> > > settings, perhaps
> > > grep for the server port to match the
> connections. I have
> > > just run a
> > > test with a couple of thousand connections and it
> worked
> > > fine. However,
> > > I am not doubting there is a problem. Also I have
> just
> > > introduced a new
> > > test to 4.1.10 that uses the java.net.URL (which
> delegates
> > > to
> > > java.net.HttpUrlConnection) and it runs through a
> couple of
> > > hundred
> > > connectiosn per sec without issue.
> > >
> > > I'll run some more substantial tests shortly. I
> would
> > > really need to see
> > > the TCP state of the leaking connections to know
> what is
> > > happening.
> > >
> > > Thanks,
> > > Niall
> > >
> > >
> > > -----Original Message-----
> > > From: Christophe Roudet [mailto:cr...@ac...]
> > >
> > > Sent: 03 April 2009 18:03
> > > To: 'Simple support and user issues'
> > > Subject: Re: [Simpleweb-Support] simple-4.1.10 is
> leaking
> > > file handles
> > >
> > > The connections are closed.
> > > I am using HttpUrlConnection to connect to the
> server.
> > > In the lsof command output I see a lot of:
> > >
> > > java    12776 root   53u
> > > sock        0,5
> > >     214006695 can't
> > > identify
> > > protocol
> > > java    12776 root   54u
> > > sock        0,5
> > >     214007226 can't
> > > identify
> > > protocol
> > > java    12776 root   55u
> > > sock        0,5
> > >     214006688 can't
> > > identify
> > > protocol
> > > java    12776 root   56u
> > > sock        0,5
> > >     214005947 can't
> > > identify
> > > protocol
> > > java    12776 root   57u
> > > sock        0,5
> > >     214005790 can't
> > > identify
> > > protocol
> > > java    12776 root   58u
> > > sock        0,5
> > >     214010186 can't
> > > identify
> > > protocol
> > > java    12776 root   59u
> > > sock        0,5
> > >     214016384 can't
> > > identify
> > > protocol
> > > java    12776 root   60u
> > > sock        0,5
> > >     214007875 can't
> > > identify
> > > protocol
> > > java    12776 root   61u
> > > sock        0,5
> > >     214006161 can't
> > > identify
> > > protocol
> > > java    12776 root   62u
> > > sock        0,5
> > >     214016081 can't
> > > identify
> > > protocol
> > > java    12776 root   63u
> > > sock        0,5
> > >     214007880 can't
> > > identify
> > > protocol
> > > java    12776 root   64u
> > > sock        0,5
> > >     214015056 can't
> > > identify
> > > protocol
> > > java    12776 root   65u
> > > sock        0,5
> > >     214007332 can't
> > > identify
> > > protocol
> > >
> > > > -----Original Message-----
> > > > From: Nia...@ub...
> > > [mailto:Nia...@ub...]
> > > > Sent: Friday, April 03, 2009 12:50 PM
> > > > To: sim...@li...
> > > > Subject: Re: [Simpleweb-Support]
> simple-4.1.10 is
> > > leaking file handles
> > > >
> > > > Hi,
> > > >
> > > > What is connecting to the server, are you
> using a load
> > > tester such as
> > > > ApacheBench. Also, are you closing the
> connections?
> > > For example
> > > >
> > > > Response.close();
> > > >
> > > > Or
> > > >
> > > > OutputStream.close();
> > > >
> > > > Within the resource or container. Just
> looking at the
> > > code for the
> > > > change and it does not seem likely that a
> file leak
> > > could occur, but
> > > > I'd like to know exactly what you see.
> > > >
> > > > If you could I would appreciate any
> information on the
> > > TCP state of
> > > > the connections. Are they CLOSE_WAIT,
> TIME_WAIT or
> > > something like
> > > FIN_WAIT1?
> > > >
> > > > Thanks,
> > > > Niall
> > > >
> > > > -----Original Message-----
> > > > From: Christophe Roudet [mailto:cr...@ac...]
> > > > Sent: 03 April 2009 17:46
> > > > To: 'Simple support and user issues'
> > > > Subject: Re: [Simpleweb-Support]
> simple-4.1.10 is
> > > leaking file handles
> > > >
> > > > I have just seen my server crashed with 'too
> many open
> > > files'
> > > messages.
> > > > I monitored with lsof -p <pid> and I
> saw the
> > > number of file handles
> > > > growing.
> > > >
> > > > Chrisotphe
> > > >
> > > > > -----Original Message-----
> > > > > From: Nia...@ub...
> > > [mailto:Nia...@ub...]
> > > > > Sent: Friday, April 03, 2009 12:42 PM
> > > > > To: sim...@li...
> > > > > Subject: Re: [Simpleweb-Support]
> simple-4.1.10 is
> > > leaking file
> > > > > handles
> > > > >
> > > > > Hi,
> > > > >
> > > > > How do you mean leaking file handles?
> In what
> > > scenario and under
> > > > > what conditions? Are you using netstat
> or
> > > something?
> > > > >
> > > > > Thanks,
> > > > > Niall
> > > > >
> > > > > -----Original Message-----
> > > > > From: Christophe Roudet [mailto:cr...@ac...]
> > > > > Sent: 03 April 2009 17:13
> > > > > To: sim...@li...
> > > > > Subject: [Simpleweb-Support]
> simple-4.1.10 is
> > > leaking file handles
> > > > >
> > > > > Hi,
> > > > >
> > > > > It looks like simple-4.1.10.jar is
> leaking file
> > > handles, using
> > > > > simple-4.1.9.jar turns things back to
> normal.
> > > > >
> > > > > Christophe
> > > > >
> > > > >
> > > > >
> > >
> --------------------------------------------------------------------
> > > > > --
> > > > > --
> > > > > ------
> > > > >
> _______________________________________________
> > > > > Simpleweb-Support mailing list
> > > > > Sim...@li...
> > > > > https://lists.sourceforge.net/lists/listinfo/simpleweb-support
> > > > > Visit our website at http://www.ubs.com
> > > > >
> > > > > This message contains confidential
> information
> > > and is intended only
> > > > > for the individual named.  If you are
> not
> > > the named addressee you
> > > > > should not disseminate, distribute or
> copy this
> > > e-mail.  Please
> > > > > notify
> > > >
> > > > > the sender immediately by e-mail if you
> have
> > > received this e-mail by
> > >
> > > > > mistake and delete this e-mail from
> your system.
> > > > >
> > > > > E-mails are not encrypted and cannot
> be
> > > guaranteed to be secure or
> > > > > error-free as information could be
> intercepted,
> > > corrupted, lost,
> > > > > destroyed, arrive late or incomplete,
> or contain
> > > viruses.  The
> > > > > sender therefore does not accept
> liability for
> > > any errors or
> > > > > omissions in the
> > > >
> > > > > contents of this message which arise as
> a result
> > > of e-mail
> > > > transmission.
> > > > > If verification is required please
> request a
> > > hard-copy version.
> > > > > This message is provided for
> informational
> > > purposes and should not
> > > > > be construed as a solicitation or offer
> to buy or
> > > sell any
> > > > > securities or related financial
> instruments.
> > > > >
> > > > > UBS Limited is a company registered in
> England
> > > & Wales under company
> > >
> > > > > number 2035362, whose registered office
> is at 1
> > > Finsbury Avenue,
> > > > > London, EC2M 2PP, United Kingdom.
> > > > >
> > > > > UBS AG (London Branch) is registered as
> a branch
> > > of a foreign
> > > > > company under number BR004507, whose
> registered
> > > office is at
> > > > > 1 Finsbury Avenue, London, EC2M 2PP,
> United
> > > Kingdom.
> > > > >
> > > > > UBS Clearing and Execution Services
> Limited is a
> > > company registered
> > > > > in
> > > >
> > > > > England & Wales under company
> number
> > > 03123037, whose registered
> > > > > office
> > > >
> > > > > is at 1 Finsbury Avenue, London, EC2M
> 2PP, United
> > > Kingdom.
> > > > >
> > > > >
> > >
> --------------------------------------------------------------------
> > > > > --
> > > > > ----
> > > > > ----
> > > > >
> _______________________________________________
> > > > > Simpleweb-Support mailing list
> > > > > Sim...@li...
> > > > > https://lists.sourceforge.net/lists/listinfo/simpleweb-support
> > > >
> > > >
> > > >
> > >
> ----------------------------------------------------------------------
> > > > --
> > > > ------
> > > >
> _______________________________________________
> > > > Simpleweb-Support mailing list
> > > > Sim...@li...
> > > > https://lists.sourceforge.net/lists/listinfo/simpleweb-support
> > > > Visit our website at http://www.ubs.com
> > > >
> > > > This message contains confidential
> information and is
> > > intended only
> > > > for the individual named.  If you are not
> the
> > > named addressee you
> > > > should not disseminate, distribute or copy
> this
> > > e-mail.  Please notify
> > >
> > > > the sender immediately by e-mail if you have
> received
> > > this e-mail by
> > > > mistake and delete this e-mail from your
> system.
> > > >
> > > > E-mails are not encrypted and cannot be
> guaranteed to
> > > be secure or
> > > > error-free as information could be
> intercepted,
> > > corrupted, lost,
> > > > destroyed, arrive late or incomplete, or
> contain
> > > viruses.  The sender
> > > > therefore does not accept liability for any
> errors or
> > > omissions in the
> > >
> > > > contents of this message which arise as a
> result of
> > > e-mail
> > > transmission.
> > > > If verification is required please request a
> hard-copy
> > > version.  This
> > > > message is provided for informational
> purposes and
> > > should not be
> > > > construed as a solicitation or offer to buy
> or sell
> > > any securities or
> > > > related financial instruments.
> > > >
> > > > UBS Limited is a company registered in
> England &
> > > Wales under company
> > > > number 2035362, whose registered office is
> at 1
> > > Finsbury Avenue,
> > > > London, EC2M 2PP, United Kingdom.
> > > >
> > > > UBS AG (London Branch) is registered as a
> branch of a
> > > foreign company
> > > > under number BR004507, whose registered
> office is at
> > > > 1 Finsbury Avenue, London, EC2M 2PP, United
> Kingdom.
> > > >
> > > > UBS Clearing and Execution Services Limited
> is a
> > > company registered in
> > >
> > > > England & Wales under company number
> 03123037,
> > > whose registered office
> > >
> > > > is at 1 Finsbury Avenue, London, EC2M 2PP,
> United
> > > Kingdom.
> > > >
> > > >
> > >
> ----------------------------------------------------------------------
> > > > ----
> > > > ----
> > > >
> _______________________________________________
> > > > Simpleweb-Support mailing list
> > > > Sim...@li...
> > > > https://lists.sourceforge.net/lists/listinfo/simpleweb-support
> > >
> > >
> > >
> ------------------------------------------------------------------------
> > > ------
> > > _______________________________________________
> > > Simpleweb-Support mailing list
> > > Sim...@li...
> > > https://lists.sourceforge.net/lists/listinfo/simpleweb-support
> > > Visit our website at http://www.ubs.com
> > >
> > > This message contains confidential information
> and is
> > > intended only
> > > for the individual named.  If you are not the
> named
> > > addressee you
> > > should not disseminate, distribute or copy this
> > > e-mail.  Please
> > > notify the sender immediately by e-mail if you
> have
> > > received this
> > > e-mail by mistake and delete this e-mail from
> your system.
> > >
> > > E-mails are not encrypted and cannot be
> guaranteed to be
> > > secure or
> > > error-free as information could be intercepted,
> corrupted,
> > > lost,
> > > destroyed, arrive late or incomplete, or contain
> > > viruses.  The sender
> > > therefore does not accept liability for any
> errors or
> > > omissions in the
> > > contents of this message which arise as a result
> of e-mail
> > > transmission.
> > > If verification is required please request a
> hard-copy
> > > version.  This
> > > message is provided for informational purposes
> and should
> > > not be
> > > construed as a solicitation or offer to buy or
> sell any
> > > securities
> > > or related financial instruments.
> > >
> > > UBS Limited is a company registered in England
> & Wales
> > > under company
> > > number 2035362, whose registered office is at 1
> Finsbury
> > > Avenue,
> > > London, EC2M 2PP, United Kingdom.
> > >
> > > UBS AG (London Branch) is registered as a branch
> of a
> > > foreign company
> > > under number BR004507, whose registered office is
> at
> > > 1 Finsbury Avenue, London, EC2M 2PP, United
> Kingdom.
> > >
> > > UBS Clearing and Execution Services Limited is a
> company
> > > registered
> > > in England & Wales under company number
> 03123037, whose
> > > registered
> > > office is at 1 Finsbury Avenue, London, EC2M 2PP,
> United
> > > Kingdom.
> > >
> > >
> ------------------------------------------------------------------------
> > ------
> > > _______________________________________________
> > > Simpleweb-Support mailing list
> > > Sim...@li...
> > > https://lists.sourceforge.net/lists/listinfo/simpleweb-support
> > >
> > 
> > 
> > 
> > 
> >
> --------------------------------------------------------------------------
> > ----
> > _______________________________________________
> > Simpleweb-Support mailing list
> > Sim...@li...
> > https://lists.sourceforge.net/lists/listinfo/simpleweb-support
> 
> -----Inline Attachment Follows-----
> 
> ------------------------------------------------------------------------------
> 
> -----Inline Attachment Follows-----
> 
> _______________________________________________
> Simpleweb-Support mailing list
> Sim...@li...
> https://lists.sourceforge.net/lists/listinfo/simpleweb-support
> 