From: Chih-Wei Yu <che...@gm...> - 2010-01-28 08:01:14
|
Hi, We're running YAWS-1.75 with 12B-5 with SSL. Platform is Solaris 10 T5220. The problems experienced is a build up of TCP sockets in IDLE state (unbound sockets). There is connections in and out but everynow and then IDLE sockets just build up. Now what is seen is that the ssl_esock process (driver) are not closing these sockets. The sockets is both created for incoming connections as well as connection between the ssl_esock and Erlang. Has anyone experience this problem. Another question is if YAWS will be using the new implementation of SSL in Erlang? Regards, Chih-Wei Yu |
From: Claes W. <kl...@ta...> - 2010-01-28 09:31:04
|
Chih-Wei Yu wrote: > Hi, > > We're running YAWS-1.75 with 12B-5 with SSL. Platform is Solaris > 10 T5220. The problems experienced is a build up of TCP sockets in IDLE > state (unbound sockets). There is connections in and out but everynow > and then IDLE sockets just build up. Now what is seen is that the > ssl_esock process (driver) are not closing these sockets. The sockets is > both created for incoming connections as well as connection between the > ssl_esock and Erlang. Has anyone experience this problem. Hmmm some bells ringing but I don't exactly recall. I'm certain I've heard this before but I don't remember the specifics. Maybey the Kreditor folks know ??? - they run an awful lot of ssl traffic. > > Another question is if YAWS will be using the new implementation of SSL > in Erlang? > I've tried the new ssl implementation a couple of times, and now redid that test using R13B03 - and I can't get it to work. I tried it the the first time over a a year ago. To enable again it's just --- a/src/yaws_server.erl +++ b/src/yaws_server.erl @@ -863,8 +863,8 @@ ssl_listen_opts(GC, SSL) -> if ?gc_use_old_ssl(GC) -> false; true -> - %%{ssl_imp, new} - still doesn't work (R13B) - false + {ssl_imp, new} %% - still doesn't work (R13B) + %false end This time it worked better though - but still no go. Not yet working. It would be great if it worked though since the esock thing has had several (hard to find bugs) over the years and it's also much more efficent to process the ssl proto inside the beam. /klacke |
From: Chih-Wei Yu <che...@gm...> - 2010-01-28 10:00:00
|
Hi, Processing the ssl proto inside erlang does sound like a better option and would make debugging easier as well. Problem currently is that its hogging file descriptors and there's no way to clean it up unless restarting the whole runtime. Regards, Chih-Wei Yu 2010/1/28 Claes Wikström <kl...@ta...> > Chih-Wei Yu wrote: > >> Hi, >> We're running YAWS-1.75 with 12B-5 with SSL. Platform is Solaris 10 >> T5220. The problems experienced is a build up of TCP sockets in IDLE state >> (unbound sockets). There is connections in and out but everynow and then >> IDLE sockets just build up. Now what is seen is that the ssl_esock process >> (driver) are not closing these sockets. The sockets is both created for >> incoming connections as well as connection between the ssl_esock and Erlang. >> Has anyone experience this problem. >> > > Hmmm some bells ringing but I don't exactly recall. I'm certain > I've heard this before but I don't remember the specifics. > Maybey the Kreditor folks know ??? - they run an awful lot of ssl > traffic. > > > Another question is if YAWS will be using the new implementation of SSL in >> Erlang? >> >> > > > I've tried the new ssl implementation a couple of times, and now redid > that test using R13B03 - and I can't get it to work. > > I tried it the the first time over a a year ago. To enable again it's just > > --- a/src/yaws_server.erl > +++ b/src/yaws_server.erl > @@ -863,8 +863,8 @@ ssl_listen_opts(GC, SSL) -> > if ?gc_use_old_ssl(GC) -> > false; > true -> > - %%{ssl_imp, new} - still doesn't work (R13B) > - false > + {ssl_imp, new} %% - still doesn't work (R13B) > + %false > end > > > This time it worked better though - but still no go. Not yet working. > It would be great if it worked though since the esock thing has had several > (hard to find bugs) over the years and it's also much more efficent to > process the ssl proto inside the beam. > > > /klacke > > > > > |
From: Claes W. <kl...@ta...> - 2010-01-28 10:09:34
|
Chih-Wei Yu wrote: > Hi, > > Processing the ssl proto inside erlang does sound like a better option > and would make debugging easier as well. Problem currently is that its > hogging file descriptors and there's no way to clean it up unless > restarting the whole runtime. Indeed - It'll require you to go to the latest OTP ssl code though. However, I couldn't get it to work with yaws when I tried it today. I'm not sure what went wrong - I just reapplied the patch I posted and tried. Maybe there is some other ssl trickery that needs to be applied in order to use the new ssl impl. I can't look at this exactly right now, so if you want immediate progress here you'll have to do some digging yourself. /klacke |
From: Chih-Wei Yu <che...@gm...> - 2010-01-28 15:51:46
|
Thank you for your assistance, i'll have to dig quite some bit before i get to the other side. =) Regards, Chih-Wei Yu 2010/1/28 Claes Wikström <kl...@ta...> > Chih-Wei Yu wrote: > >> Hi, >> >> Processing the ssl proto inside erlang does sound like a better option >> and would make debugging easier as well. Problem currently is that its >> hogging file descriptors and there's no way to clean it up unless restarting >> the whole runtime. >> > > Indeed - It'll require you to go to the latest OTP ssl code though. > However, I couldn't get it to work with yaws when I tried it today. > I'm not sure what went wrong - I just reapplied the patch I posted > and tried. Maybe there is some other ssl trickery that needs to be applied > in order to use the new ssl impl. > > I can't look at this exactly right now, so if you want immediate progress > here you'll have to do some digging yourself. > > /klacke > > |
From: Klas J. <kla...@gm...> - 2010-01-28 19:55:30
|
Hi, We experienced the problems Tomas described in this post (started by you it seems?): http://thread.gmane.org/gmane.comp.lang.erlang.general/41881 I'm not aware of any solution though, other than that the problem (at least partially) initiated the implementation of the new ssl. What you're seeing now, is that the same issue (enoproxysocket) outlined in the above post? Myself, I'd see if I could get the new ssl up to speed with yaws, since it'd be a lot easier to troubleshoot and build upon (when/if needed). Unfortunately I don't have that much high volume, stress test experience to say how it fares in a production (like) environment though. I'll leave that for someone else to comment upon. Kind Regards, Klas 2010/1/28 Chih-Wei Yu <che...@gm...>: > Thank you for your assistance, i'll have to dig quite some bit before i get > to the other side. =) > > Regards, > Chih-Wei Yu > > 2010/1/28 Claes Wikström <kl...@ta...> >> >> Chih-Wei Yu wrote: >>> >>> Hi, >>> Processing the ssl proto inside erlang does sound like a better option >>> and would make debugging easier as well. Problem currently is that its >>> hogging file descriptors and there's no way to clean it up unless restarting >>> the whole runtime. >> >> Indeed - It'll require you to go to the latest OTP ssl code though. >> However, I couldn't get it to work with yaws when I tried it today. >> I'm not sure what went wrong - I just reapplied the patch I posted >> and tried. Maybe there is some other ssl trickery that needs to be applied >> in order to use the new ssl impl. >> >> I can't look at this exactly right now, so if you want immediate progress >> here you'll have to do some digging yourself. >> >> /klacke >> > > > ------------------------------------------------------------------------------ > The Planet: dedicated and managed hosting, cloud storage, colocation > Stay online with enterprise data centers and the best network in the > business > Choose flexible plans and management services without long-term contracts > Personal 24x7 support from experience hosting pros just a phone call away. > http://p.sf.net/sfu/theplanet-com > _______________________________________________ > Erlyaws-list mailing list > Erl...@li... > https://lists.sourceforge.net/lists/listinfo/erlyaws-list > > |
From: Claes W. <kl...@ta...> - 2010-01-28 21:38:44
|
Klas Johansson wrote: > Hi, > > We experienced the problems Tomas described in this post (started by > you it seems?): > http://thread.gmane.org/gmane.comp.lang.erlang.general/41881 > > I'm not aware of any solution though, other than that the problem (at > least partially) initiated the implementation of the new ssl. What > you're seeing now, is that the same issue (enoproxysocket) outlined in > the above post? Good input - thanks > > Myself, I'd see if I could get the new ssl up to speed with yaws, > since it'd be a lot easier to troubleshoot and build upon Agree completely - but as I said - I don't have the time to investigate this now. /klacke |
From: Claes W. <kl...@ta...> - 2010-02-01 08:20:07
Attachments:
f
|
Chih-Wei Yu wrote: > Thank you for your assistance, i'll have to dig quite some bit before i > get to the other side. =) > Good news - I did some digging here, and commit: http://github.com/klacke/yaws/commit/d4fec792ddc73f475514fbf41974a05d65a801aa Seems to make the new ssl implementation work. It probably needs spme more testing with things like error codes on bad certs client certs etc - but the basics work. You should be able to patch the above into slightly old yaws releases as well. I attach a regular diff as well Good luck /klacke |
From: Chih-Wei Yu <che...@gm...> - 2010-02-01 16:13:28
|
Hi, I did raise the same problem in an earlier thread last year and just wanted to find out if anyone has started using the new SSL implementation in Erlang or came across what we were experiencing. Thanks for the patch. I assume this will be in the next release of YAWS. Regards, Chih-Wei Yu On Mon, Feb 1, 2010 at 10:19 AM, Claes Wikstrom <kl...@ta...> wrote: > Chih-Wei Yu wrote: > >> Thank you for your assistance, i'll have to dig quite some bit before i >> get to the other side. =) >> >> > > Good news - I did some digging here, and commit: > > > http://github.com/klacke/yaws/commit/d4fec792ddc73f475514fbf41974a05d65a801aa > > Seems to make the new ssl implementation work. > It probably needs spme more testing with things like > error codes on bad certs client certs etc - but the basics work. > > You should be able to patch the above into slightly old yaws > releases as well. I attach a regular diff as well > > > Good luck > > /klacke > > diff --git a/man/yaws.conf.5 b/man/yaws.conf.5 > index efc45b7..2382960 100644 > --- a/man/yaws.conf.5 > +++ b/man/yaws.conf.5 > @@ -92,7 +92,11 @@ to the yaws start script, this value is automatically > set to 0. > This enables traffic or http tracing. Tracing is also possible to enable > with > a command line flag to yaws. Default is false. > > - > +.TP > +\fBuse_old_ssl = true | false\fR > +This re-enables the old OTP ssl implementation. By default we use the > +new ssl implementation. > + > .TP > \fB auth_log = true | false\fR > Enable or disable the auth log. Default is true. > diff --git a/scripts/yaws.conf.template b/scripts/yaws.conf.template > index c526a4c..b97cc96 100644 > --- a/scripts/yaws.conf.template > +++ b/scripts/yaws.conf.template > @@ -37,8 +37,9 @@ max_connections = nolimit > > trace = false > > - > - > +# Enable this if we want to use the old OTP ssl implementation > +# OTP R13B03 is known to work with this flag set to false > (default) > +use_old_ssl = false > > > # it is possible to have yaws start additional > @@ -85,7 +86,10 @@ auth_log = true > # name. Yaws will write a number of runtime files under > # ${HOME}/.yaws/yaws/${id} > # The default value is "default" > - > +# If we're not planning to run multiple webservers on the > +# same host it's mych better to leave this value unset since > +# then all the ctl function (--stop et.el) work without having > +# to supply the id. > > # id = myname > > @@ -95,6 +99,7 @@ auth_log = true > # header doesn't match any name on any Host > # This is often nice in testing environments but not > # acceptable in real live hosting scenarios > +# think http://suckmydick.bigcompany.com > > pick_first_virthost_on_nomatch = true > > @@ -106,7 +111,9 @@ pick_first_virthost_on_nomatch = true > # a privileged port. > # If we use this feature, it requires fdsrv to be properly installed. > # Doesn't yet work with SSL. > -# Read http://yaws.hyber.org/privbind.yaws for more info > +# Read http://yaws.hyber.org/privbind.yaws for more info and a better > +# solution than fd_srv > + > use_fdsrv = false > > > @@ -149,6 +156,7 @@ use_fdsrv = false > <ssl> > keyfile = %certdir%/yaws-key.pem > certfile = %certdir%/yaws-cert.pem > + depth = 0 > </ssl> > </server> > > diff --git a/src/yaws.erl b/src/yaws.erl > index 79e47bb..323803f 100644 > --- a/src/yaws.erl > +++ b/src/yaws.erl > @@ -12,8 +12,6 @@ > -include("../include/yaws_api.hrl"). > -include("yaws_debug.hrl"). > > - > - > -include_lib("kernel/include/file.hrl"). > -export([start/0, stop/0, hup/1, restart/0, modules/0, load/0]). > -export([start_embedded/1, start_embedded/2, start_embedded/3, > @@ -1774,7 +1772,6 @@ do_recv(Sock, Num, nossl) -> > do_recv(Sock, Num, ssl) -> > ssl:recv(Sock, Num, ?READ_TIMEOUT). > > - > cli_recv(S, Num, SslBool) -> > Res = do_recv(S, Num, SslBool), > cli_recv_trace((get(gc))#gconf.trace, Res), > @@ -1883,7 +1880,6 @@ setopts(Sock, Opts, ssl) -> > ok = ssl:setopts(Sock, Opts). > > do_http_get_headers(CliSock, SSL) -> > - setopts(CliSock, [{packet, http}], SSL), > case http_recv_request(CliSock,SSL) of > bad_request -> > {#http_request{method=bad_request, version={0,9}}, > @@ -1897,6 +1893,7 @@ do_http_get_headers(CliSock, SSL) -> > > > http_recv_request(CliSock, SSL) -> > + setopts(CliSock, [{packet, http}], SSL), > case do_recv(CliSock, 0, SSL) of > {ok, R} when is_record(R, http_request) -> > R; > @@ -1912,13 +1909,12 @@ http_recv_request(CliSock, SSL) -> > closed; > {error, timeout} -> closed; > _Other -> > - ?Debug("Got ~p~n", [_Other]), > + error_logger:format("Unhandled reply fr. do_recv() ~p~n", > [_Other]), > exit(normal) > end. > > - > - > http_collect_headers(CliSock, Req, H, SSL, Count) when Count < 1000 -> > + setopts(CliSock, [{packet, httph}], SSL), > Recv = do_recv(CliSock, 0, SSL), > case Recv of > {ok, {http_header, _Num, 'Host', _, Host}} -> > diff --git a/src/yaws_config.erl b/src/yaws_config.erl > index ea492ef..dab1817 100644 > --- a/src/yaws_config.erl > +++ b/src/yaws_config.erl > @@ -1103,7 +1103,7 @@ fload(FD, ssl, GC, C, Cs, Lno, Chars) -> > end; > ["depth", '=', Val0] -> > Val = (catch list_to_integer(Val0)), > - case lists:member(Val, [1,2,3,4,5,6,7]) of > + case lists:member(Val, [0, 1,2,3,4,5,6,7]) of > true when is_record(C#sconf.ssl, ssl) -> > C2 = C#sconf{ssl = (C#sconf.ssl)#ssl{depth = Val}}, > fload(FD, ssl, GC, C2, Cs, Lno+1, Next); > @@ -1111,7 +1111,7 @@ fload(FD, ssl, GC, C, Cs, Lno, Chars) -> > {error, ?F("Need to set option ssl to true before line > ~w", > [Lno])}; > _ -> > - {error, ?F("Expect reasonable integer at line ~w", > [Lno])} > + {error, ?F("Expect integer 0..7 at line ~w", [Lno])} > end; > ["password", '=', Val] -> > if > diff --git a/src/yaws_server.erl b/src/yaws_server.erl > index e99d36e..c97c1ff 100644 > --- a/src/yaws_server.erl > +++ b/src/yaws_server.erl > @@ -860,11 +860,15 @@ ssl_listen_opts(GC, SSL) -> > true -> > false > end, > + if SSL#ssl.depth /= undefined -> > + {depth, SSL#ssl.depth}; > + true -> > + false > + end, > if ?gc_use_old_ssl(GC) -> > false; > true -> > - %%{ssl_imp, new} - still doesn't work (R13B) > - false > + {ssl_imp, new} > end > ], > filter_false(L). > > Hi K |
From: Claes W. <kl...@ta...> - 2010-02-01 21:19:39
|
Chih-Wei Yu wrote: > > Thanks for the patch. I assume this will be in the next release of YAWS. yes, /klacke |