Re: [EsiGate-users] Connection thread leak

[Francois-Xavier B. <fra...@ce...>]

Could you send the complete stack trace for the UnknownHostException?
It would also be helpful to activate FetchLogging as explained here:
http://www.esigate.org/reference.html#FragmentLogging_and_FetchLogging_extensions

2016-07-12 11:46 GMT+02:00 Poupon Emilie <EP...@vo...>:

>
>
> 1/ When falling this morning (with no user), we had no error in our
> EsiGate logs.
>
> What we see now is that the Java Application responds with some 302 to the
> EsiGate (bloc inclusion) : we are searching around these responses if they
> could explain the instance “death”.
>
>
>
> 2/ When falling past night (with some users), we had one error message we
> didn’t explain yet :
>
> org.esigate.HttpErrorPage - Error retrieving URL
>
> java.net.UnknownHostException: www.transilien.com: unknown error
>
>             at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
>
>             at
> java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:907)
>
>             at
> java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1302)
>
>             at java.net.InetAddress.getAllByName0(InetAddress.java:1255)
>
>             at java.net.InetAddress.getAllByName(InetAddress.java:1171)
>
> The EsiGate should not access the www.transilien.com domain as an
> external domain, in fact it is the website itself.
>
>
>
>
>
> Emilie Poupon Adam
>
> Responsable Delivery Web Mobile
>
> VSC Technologies
>
>
>
> *De :* fx....@gm... [mailto:fx....@gm...] *De la part de*
> Francois-Xavier Bonnet
> *Envoyé :* mardi 12 juillet 2016 10:44
>
> *À :* Poupon Emilie <EP...@VO...>
> *Cc :* Francois-Xavier Bonnet <fra...@ce...>;
> Auffredou Thomas <TAu...@vo...>;
> web...@li...
> *Objet :* Re: [EsiGate-users] Connection thread leak
>
>
>
> Do you have some errors in the log files of EsiGate?
>
>
>
> 2016-07-12 9:33 GMT+02:00 Poupon Emilie <EP...@vo...>:
>
> Hi,
>
>
>
> I did suscribe yesterday (but after sendind my e-mail) !
>
>
>
> What is the value of {{socketTimeout}} ?
>
> socketTimeout = 45000
>
>
>
> I can see in your configuration that you let the default value for
> maxConnectionsPerHost which is 20.
>
> Yes, we saw that point and agreed to put an upper value (70) since the
> number of connections is limited at entry.
>
> We are going to test this today.
>
>
>
> When the problem occurs, do you have a lot of users on the application?
> How many hits/s?
>
> First time we saw the problem we had an several users connected, but
> yesterday problem occurred on a server not used at all (only life check was
> running).
>
> We didn't find out how make the problem occurs, we tried with several
> stress tests and not succeed.
>
>
>
> Does the problem fix itself when the load decreases or not?
>
> NO ! We had to reboot the servers, even after several hours, number of
> open thread didn’t decrease.
>
>
>
> Is EsiGate used also for static files (images, js, css) from the drupal
> backend? If so why did you disable the cache.
>
> EsiGate mix transactional content from a Java application and editorial
> content from a drupal backend.
>
> We are using a VARNISH cache between the EsiGate and the Drupal backend.
>
>
>
> Find here the JVM / Tomcat parameters set on our ESIGate server :
>
> connectionTimeout = 20000
>
> maxThreads = 70
>
> acceptCount = 50
>
> compression no value
>
> compressionMinSize no value
>
> compressableMimeType no value
>
> JDK = 1.8u45 64bits
>
> Tomcat = 7.0.53
>
> JVM params = Xms512m Xmx512m XX:+UseConcMarkSweepGC
>
>
>
> Thx,
>
> Emilie Poupon Adam
>
>
>
> *De :* fx....@gm... [mailto:fx....@gm...] *De la part de*
> Francois-Xavier Bonnet
> *Envoyé :* mardi 12 juillet 2016 08:36
> *À :* Poupon Emilie <EP...@VO...>
> *Cc :* Francois-Xavier Bonnet <fra...@ce...>;
> Auffredou Thomas <TAu...@vo...>;
> web...@li...
>
>
> *Objet :* Re: [EsiGate-users] Connection thread leak
>
>
>
> Hi,
>
>
>
> Please subscribe to the mailing list for faster answers.
>
>
>
> What is the value of {{socketTimeout}} ?
>
> I can see in your configuration that you let the default value for
> maxConnectionsPerHost which is 20. When the problem occurs, do you have a
> lot of users on the application? How many hits/s? Does the problem fix
> itself when the load decreases or not?
>
> Is EsiGate used also for static files (images, js, css) from the drupal
> backend? If so why did you disable the cache.
>
>
>
> 2016-07-11 20:02 GMT+02:00 Poupon Emilie <EP...@vo...>:
>
> Hi,
>
>
>
> I’m working with Thomas, he left few minutes ago (he is coming back on
> wednesday).
>
> We still have the issue and no idea about the causes.
>
>
>
> ESI Gate 5.1 + esigate.properties
>
> Thx,
>
> Emilie Poupon Adam
>
>
>
> *De :* fx....@gm... [mailto:fx....@gm...] *De la part de*
> Francois-Xavier Bonnet
> *Envoyé :* lundi 11 juillet 2016 18:22
> *À :* Auffredou Thomas <TAu...@vo...>
> *Cc :* web...@li...; Poupon Emilie <
> EP...@VO...>
> *Objet :* Re: [EsiGate-users] Connection thread leak
>
>
>
> Hi,
>
>
>
> In order to help me understand your issue, could you give more
> information: Which version of EsiGate are you using? Could you share your
> esigate.properties configuration file?
>
>
>
> Please subscribe to the mailing so that your future messages will be
> forwarded directly without requiring an administrator to validate them.
>
>
>
> 2016-07-11 17:36 GMT+02:00 Auffredou Thomas <
> TAu...@vo...>:
>
> Hi,
>
>
>
> I’m experimenting issues with the ESI clientPool.
>
> At some point (I can’t figure out what are the causes), threads start to
> grow until reaching the thread pool saturation.
>
> The leaks happens linearly when requesting a healthcheck url, and any
> other request accelerates the behavior.
>
>
>
> The following threaddump extract shows it hangs while acquiring the http
> client.
>
>
>
> "http-bio-10.98.48.138-51125-exec-1" #40 daemon prio=5 os_prio=0
> tid=0x00007fb760001800 nid=0x6ff4 waiting on condition [0x00007fb7dc507000]
>
>    java.lang.Thread.State: WAITING (parking)
>
>                 at sun.misc.Unsafe.park(Native Method)
>
>                 - parking to wait for  <0x00000000eb970880> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
>                 at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
>
>                 at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
>
>                 at
> org.apache.http.pool.PoolEntryFuture.await(PoolEntryFuture.java:133)
>
>                 at
> org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:282)
>
>                 at
> org.apache.http.pool.AbstractConnPool.access$000(AbstractConnPool.java:64)
>
>                 at
> org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:177)
>
>                 at
> org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:170)
>
>                 at
> org.apache.http.pool.PoolEntryFuture.get(PoolEntryFuture.java:102)
>
>                 at
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:244)
>
>                 at
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:231)
>
>                 at
> org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:173)
>
>                 at
> org.esigate.http.ProxyingHttpClientBuilder$1.execute(ProxyingHttpClientBuilder.java:105)
>
>                 at
> org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195)
>
>                 at
> org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:86)
>
>                 at
> org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108)
>
>                 at
> org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
>
>                 at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72)
>
>                 at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
>
>                 at
> org.esigate.http.HttpClientRequestExecutor.execute(HttpClientRequestExecutor.java:307)
>
>                 at org.esigate.Driver.render(Driver.java:188)
>
>                 at
> org.esigate.esi.IncludeElement.processPage(IncludeElement.java:193)
>
>                 at
> org.esigate.esi.IncludeElement.onTagEnd(IncludeElement.java:81)
>
>                 at
> org.esigate.parser.ParserContextImpl.endElement(ParserContextImpl.java:92)
>
>                 at org.esigate.parser.Parser.parse(Parser.java:76)
>
>                 at
> org.esigate.esi.EsiRenderer.render(EsiRenderer.java:122)
>
>                 at org.esigate.Driver.performRendering(Driver.java:401)
>
>                 at org.esigate.Driver.performRendering(Driver.java:360)
>
>                 at org.esigate.Driver.proxy(Driver.java:295)
>
>                 at org.esigate.DriverFactory.proxy(DriverFactory.java:403)
>
>                 at
> org.esigate.servlet.ProxyFilter.doFilter(ProxyFilter.java:60)
>
>                 at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
>
>                 at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
>
>                 at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
>
>                 at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
>
>                 at
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501)
>
>                 at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
>
>                 at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
>
>                 at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
>
>                 at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
>
>                 at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
>
>                 at
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1040)
>
>                 at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
>
>                 at
> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:313)
>
>                 - locked <0x00000000ec8c1e70> (a
> org.apache.tomcat.util.net.SocketWrapper)
>
>                 at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>
>                 at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>
>                 at java.lang.Thread.run(Thread.java:745)
>
>
>
> Did someone already encountered this issue ?
>
>
>
> Thanks,
>
> Thomas
>
>
>
> ------------------------------------------------------------------------------
> Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
> Francisco, CA to explore cutting-edge tech and listen to tech luminaries
> present their vision of the future. This family event has something for
> everyone, including kids. Get more information and register today.
> http://sdm.link/attshape
> _______________________________________________
> Webassembletool-users mailing list
> Web...@li...
> https://lists.sourceforge.net/lists/listinfo/webassembletool-users
>
>
>
>
>
>
>