Re: [EsiGate-users] Connection thread leak

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Hi,

I did suscribe yesterday (but after sendind my e-mail) !

What is the value of {{socketTimeout}} ?
socketTimeout = 45000

I can see in your configuration that you let the default value for maxConnectionsPerHost which is 20.
Yes, we saw that point and agreed to put an upper value (70) since the number of connections is limited at entry.
We are going to test this today.

When the problem occurs, do you have a lot of users on the application? How many hits/s?
First time we saw the problem we had an several users connected, but yesterday problem occurred on a server not used at all (only life check was running).
We didn't find out how make the problem occurs, we tried with several stress tests and not succeed.

Does the problem fix itself when the load decreases or not?
NO ! We had to reboot the servers, even after several hours, number of open thread didn’t decrease.

Is EsiGate used also for static files (images, js, css) from the drupal backend? If so why did you disable the cache.
EsiGate mix transactional content from a Java application and editorial content from a drupal backend.
We are using a VARNISH cache between the EsiGate and the Drupal backend.

Find here the JVM / Tomcat parameters set on our ESIGate server :
connectionTimeout = 20000
maxThreads = 70
acceptCount = 50
compression no value
compressionMinSize no value
compressableMimeType no value
JDK = 1.8u45 64bits
Tomcat = 7.0.53
JVM params = Xms512m Xmx512m XX:+UseConcMarkSweepGC

Thx,
Emilie Poupon Adam

De : fx....@gm... [mailto:fx....@gm...] De la part de Francois-Xavier Bonnet
Envoyé : mardi 12 juillet 2016 08:36
À : Poupon Emilie <EP...@VO...>
Cc : Francois-Xavier Bonnet <fra...@ce...>; Auffredou Thomas <TAu...@vo...>; web...@li...
Objet : Re: [EsiGate-users] Connection thread leak

Hi,

Please subscribe to the mailing list for faster answers.

What is the value of {{socketTimeout}} ?
I can see in your configuration that you let the default value for maxConnectionsPerHost which is 20. When the problem occurs, do you have a lot of users on the application? How many hits/s? Does the problem fix itself when the load decreases or not?
Is EsiGate used also for static files (images, js, css) from the drupal backend? If so why did you disable the cache.

2016-07-11 20:02 GMT+02:00 Poupon Emilie <EP...@vo...<mailto:EP...@vo...>>:
Hi,

I’m working with Thomas, he left few minutes ago (he is coming back on wednesday).
We still have the issue and no idea about the causes.

ESI Gate 5.1 + esigate.properties
Thx,
Emilie Poupon Adam

De : fx....@gm...<mailto:fx....@gm...> [mailto:fx....@gm...<mailto:fx....@gm...>] De la part de Francois-Xavier Bonnet
Envoyé : lundi 11 juillet 2016 18:22
À : Auffredou Thomas <TAu...@vo...<mailto:TAu...@vo...>>
Cc : web...@li...<mailto:web...@li...>; Poupon Emilie <EP...@VO...<mailto:EP...@VO...>>
Objet : Re: [EsiGate-users] Connection thread leak

Hi,

In order to help me understand your issue, could you give more information: Which version of EsiGate are you using? Could you share your esigate.properties configuration file?

Please subscribe to the mailing so that your future messages will be forwarded directly without requiring an administrator to validate them.

2016-07-11 17:36 GMT+02:00 Auffredou Thomas <TAu...@vo...<mailto:TAu...@vo...>>:
Hi,

I’m experimenting issues with the ESI clientPool.
At some point (I can’t figure out what are the causes), threads start to grow until reaching the thread pool saturation.
The leaks happens linearly when requesting a healthcheck url, and any other request accelerates the behavior.

The following threaddump extract shows it hangs while acquiring the http client.

"http-bio-10.98.48.138-51125-exec-1" #40 daemon prio=5 os_prio=0 tid=0x00007fb760001800 nid=0x6ff4 waiting on condition [0x00007fb7dc507000]
   java.lang.Thread.State: WAITING (parking)
                at sun.misc.Unsafe.park(Native Method)
                - parking to wait for  <0x00000000eb970880> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
                at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
                at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
                at org.apache.http.pool.PoolEntryFuture.await(PoolEntryFuture.java:133)
                at org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:282)
                at org.apache.http.pool.AbstractConnPool.access$000(AbstractConnPool.java:64)
                at org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:177)
                at org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:170)
                at org.apache.http.pool.PoolEntryFuture.get(PoolEntryFuture.java:102)
                at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:244)
                at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:231)
                at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:173)
                at org.esigate.http.ProxyingHttpClientBuilder$1.execute(ProxyingHttpClientBuilder.java:105)
                at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195)
                at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:86)
                at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108)
                at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
                at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72)
                at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
                at org.esigate.http.HttpClientRequestExecutor.execute(HttpClientRequestExecutor.java:307)
                at org.esigate.Driver.render(Driver.java:188)
                at org.esigate.esi.IncludeElement.processPage(IncludeElement.java:193)
                at org.esigate.esi.IncludeElement.onTagEnd(IncludeElement.java:81)
                at org.esigate.parser.ParserContextImpl.endElement(ParserContextImpl.java:92)
                at org.esigate.parser.Parser.parse(Parser.java:76)
                at org.esigate.esi.EsiRenderer.render(EsiRenderer.java:122)
                at org.esigate.Driver.performRendering(Driver.java:401)
                at org.esigate.Driver.performRendering(Driver.java:360)
                at org.esigate.Driver.proxy(Driver.java:295)
                at org.esigate.DriverFactory.proxy(DriverFactory.java:403)
                at org.esigate.servlet.ProxyFilter.doFilter(ProxyFilter.java:60)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
                at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
                at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
                at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501)
                at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
                at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
                at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
                at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
                at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
                at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1040)
                at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
                at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:313)
                - locked <0x00000000ec8c1e70> (a org.apache.tomcat.util.net.SocketWrapper)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
                at java.lang.Thread.run(Thread.java:745)

Did someone already encountered this issue ?

Thanks,
Thomas

------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
Webassembletool-users mailing list
Web...@li...<mailto:Web...@li...>
https://lists.sourceforge.net/lists/listinfo/webassembletool-users