ocamlnet-devel — Developer discussion

Re: [Ocamlnet-devel] Two issues with Netplex

[Gerd S. <in...@ge...>]

Am Donnerstag, den 07.11.2013, 19:34 +0100 schrieb Thomas Calderon:
> Hi there,
> 
> 
> I am having two issues with netplex at the moment.
> 
> 
> The first one is related to rpc-ssl support. The netplex service is
> configured with SSL enabled (TLSv1) and if you are connecting with a
> client that forces newer TLS versions (openssl s_client -connect
> host:port -tls1_1), an Ssl exception is raised.

Well, the ocaml-ssl bindings only offer TLS-1.0, even if openssl
supports more. That's basically a problem of the bindings. (Fyi, I'm
currently implementing complete bindings for GnuTLS to overcome all
limitations with ocaml-ssl.)

> However, the container process still continues to run but is not able
> to process any data, netplex then spawn a new child process to handle
> subsequent connections. 
> From the client side, the socket is closed and we end up not knowing
> in which state the "zombie" container is.
> We looked for a proper way to fix this in our code, but the exception
> is caught from within rpc_netplex.ml (line 183), is there a proper way
> to handle this case from our code or is it only a fix you can
> provide ?

You could close fd, and then call when_done() from the is_error
callback. This is how it is indicated that the task is done.

Probably this is really the best thing you can do here. When I
programmed this I could probably not imagine what kind of things can go
wrong at this spot, and hence opted to do nothing.

> The second issue is related the function
> "Netplex_cenv.system_shutdown()". This function can be used to
> completely shutdown the service, I would like to know if it is
> possible to restrict this function to be only callable from within the
> controller instance ? This way a container process could not call this
> function and stop the service and all the other containers from
> running.

Without modifying the sources, there is probably no way to get this
effect.

What's actually happening is that the controller accepts requests on the
admin socket (which is a Unix domain socket visible in the filesystem),
and if the container wants to do a system shutdown, is just connects to
this socket, and invokes the right RPC. I don't see how you could
prevent the container from doing so even if the connect code were
removed from the netplex sources. Maybe the only way is to set a
password on this service, and not telling the container the password.

Out of curiosity, what's your concern with system shutdowns? 

Gerd

> 
> Thanks.
> 
> 
> Thomas C.
> 
> 
> ------------------------------------------------------------------------------
> November Webinars for C, C++, Fortran Developers
> Accelerate application performance with scalable programming models. Explore
> techniques for threading, error checking, porting, and tuning. Get the most 
> from the latest Intel processors and coprocessors. See abstracts and register
> http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk
> _______________________________________________ Ocamlnet-devel mailing list Oca...@li... https://lists.sourceforge.net/lists/listinfo/ocamlnet-devel

-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
My OCaml site:          http://www.camlcity.org
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------

[Ocamlnet-devel] Two issues with Netplex

[Thomas C. <cal...@gm...>]

Hi there,

I am having two issues with netplex at the moment.

The first one is related to rpc-ssl support. The netplex service is
configured with SSL enabled (TLSv1) and if you are connecting with a client
that forces newer TLS versions (openssl s_client -connect host:port
-tls1_1), an Ssl exception is raised.
However, the container process still continues to run but is not able to
process any data, netplex then spawn a new child process to handle
subsequent connections.
>From the client side, the socket is closed and we end up not knowing in
which state the "zombie" container is.
We looked for a proper way to fix this in our code, but the exception is
caught from within rpc_netplex.ml (line 183), is there a proper way to
handle this case from our code or is it only a fix you can provide ?

The second issue is related the function "Netplex_cenv.system_shutdown()".
This function can be used to completely shutdown the service, I would like
to know if it is possible to restrict this function to be only callable
from within the controller instance ? This way a container process could
not call this function and stop the service and all the other containers
from running.

Thanks.

Thomas C.

Re: [Ocamlnet-devel] installation error

[Ashish A. <aga...@gm...>]

Thanks. I'll apply the patch. Unfortunately, upgrading the kernel is not in
my hands. It's a cluster used by hundreds of people. The sys admin's policy
is to never upgrade software.


On Thu, Oct 10, 2013 at 10:34 AM, Gerd Stolpmann <in...@ge...>wrote:

> Am Donnerstag, den 10.10.2013, 10:09 -0400 schrieb Ashish Agarwal:
> > With opam, on RHEL, I'm getting the error below. I've installed
> > ocamlnet on the same computer many times before, and don't think I
> > changed anything. Any ideas?
>
> It's a bug. Fix:
>
>
> https://godirepo.camlcity.org/wwwsvn/trunk/code/src/netsys/netsys_c.c?r1=1886&r2=1918&p1=trunk/code/src/netsys/netsys_c.c&p2=trunk/code/src/netsys/netsys_c.c&root=lib-ocamlnet2
>
> But honestly, you are using an ancient kernel (without CLOEXEC flag,
> i.e. older than 2.6.23). Consider upgrading the system.
>
> Gerd
>
> >
> > ===== ERROR while installing ocamlnet.3.7.3 =====
> > # opam-version 1.1.0-beta (c381a2daa6751e5b649d96a6e2b4226c8ebe4366)
> > # os           linux
> > # command      make all
> > # path         /home/aa144/.opam/4.01.0/build/ocamlnet.3.7.3
> > # compiler     4.01.0
> > # exit-code    2
> > # env-file
> >   /home/aa144/.opam/4.01.0/build/ocamlnet.3.7.3/ocamlnet-22544-ffb3fd.env
> > # stdout-file
> >  /home/aa144/.opam/4.01.0/build/ocamlnet.3.7.3/ocamlnet-22544-ffb3fd.out
> > # stderr-file
> >  /home/aa144/.opam/4.01.0/build/ocamlnet.3.7.3/ocamlnet-22544-ffb3fd.err
> > ### stdout ###
> > # ...[truncated]
> > # ocamlfind ocamlc -g   -package "unix unix" -c  netsys_xdr.mli
> > # ocamlfind ocamlc -g   -package "unix unix"  -c  netsys_xdr.ml
> > # ocamlfind ocamlc -g   -package "unix unix" -c  netsys_tmp.mli
> > # ocamlfind ocamlc -g   -package "unix unix"  -c  netsys_tmp.ml
> > # ocamlfind ocamlc -g   -package "unix unix" -c  netsys_sem.mli
> > # ocamlfind ocamlc -g   -package "unix unix"  -c  netsys_sem.ml
> > # ocamlfind ocamlc -g   -package "unix unix" -c  netsys_pmanage.mli
> > # ocamlfind ocamlc -g   -package "unix unix"  -c  netsys_pmanage.ml
> > # ../../tools/stubcc -ocamlc ocamlc -ccopt -O   netsys_c.c
> > # make[1]: Leaving directory
> > `/home/aa144/.opam/4.01.0/build/ocamlnet.3.7.3/src/netsys'
> > ### stderr ###
> > # + gcc -fno-defer-pop -Wall -D_FILE_OFFSET_BITS=64 -D_REENTRANT -fPIC
> > -c  -O  -I'/home/aa144/.opam/4.01.0/lib/ocaml' 'netsys_c.c'
> > # netsys_c.c: In function 'netsys_openat':
> > # netsys_c.c:482: error: 'fd' undeclared (first use in this function)
> > # netsys_c.c:482: error: (Each undeclared identifier is reported only
> > once
> > # netsys_c.c:482: error: for each function it appears in.)
> > # make[1]: *** [netsys_c.o] Error 2
> > # make: *** [all] Error 2
> >
> >
> >
> ------------------------------------------------------------------------------
> > October Webinars: Code for Performance
> > Free Intel webinars can help you accelerate application performance.
> > Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
> from
> > the latest Intel processors and coprocessors. See abstracts and register
> >
> >
> http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk
> > _______________________________________________ Ocamlnet-devel mailing
> list Oca...@li...
> https://lists.sourceforge.net/lists/listinfo/ocamlnet-devel
>
> --
> ------------------------------------------------------------
> Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
> Creator of GODI and camlcity.org.
> Contact details:        http://www.camlcity.org/contact.html
> Company homepage:       http://www.gerd-stolpmann.de
> ------------------------------------------------------------
>

Re: [Ocamlnet-devel] installation error

[Gerd S. <in...@ge...>]

Am Donnerstag, den 10.10.2013, 10:09 -0400 schrieb Ashish Agarwal:
> With opam, on RHEL, I'm getting the error below. I've installed
> ocamlnet on the same computer many times before, and don't think I
> changed anything. Any ideas?

It's a bug. Fix:

https://godirepo.camlcity.org/wwwsvn/trunk/code/src/netsys/netsys_c.c?r1=1886&r2=1918&p1=trunk/code/src/netsys/netsys_c.c&p2=trunk/code/src/netsys/netsys_c.c&root=lib-ocamlnet2

But honestly, you are using an ancient kernel (without CLOEXEC flag,
i.e. older than 2.6.23). Consider upgrading the system.

Gerd

> 
> ===== ERROR while installing ocamlnet.3.7.3 =====
> # opam-version 1.1.0-beta (c381a2daa6751e5b649d96a6e2b4226c8ebe4366)
> # os           linux
> # command      make all
> # path         /home/aa144/.opam/4.01.0/build/ocamlnet.3.7.3
> # compiler     4.01.0
> # exit-code    2
> # env-file
>   /home/aa144/.opam/4.01.0/build/ocamlnet.3.7.3/ocamlnet-22544-ffb3fd.env
> # stdout-file
>  /home/aa144/.opam/4.01.0/build/ocamlnet.3.7.3/ocamlnet-22544-ffb3fd.out
> # stderr-file
>  /home/aa144/.opam/4.01.0/build/ocamlnet.3.7.3/ocamlnet-22544-ffb3fd.err
> ### stdout ###
> # ...[truncated]
> # ocamlfind ocamlc -g   -package "unix unix" -c  netsys_xdr.mli
> # ocamlfind ocamlc -g   -package "unix unix"  -c  netsys_xdr.ml
> # ocamlfind ocamlc -g   -package "unix unix" -c  netsys_tmp.mli
> # ocamlfind ocamlc -g   -package "unix unix"  -c  netsys_tmp.ml
> # ocamlfind ocamlc -g   -package "unix unix" -c  netsys_sem.mli
> # ocamlfind ocamlc -g   -package "unix unix"  -c  netsys_sem.ml
> # ocamlfind ocamlc -g   -package "unix unix" -c  netsys_pmanage.mli
> # ocamlfind ocamlc -g   -package "unix unix"  -c  netsys_pmanage.ml
> # ../../tools/stubcc -ocamlc ocamlc -ccopt -O   netsys_c.c
> # make[1]: Leaving directory
> `/home/aa144/.opam/4.01.0/build/ocamlnet.3.7.3/src/netsys'
> ### stderr ###
> # + gcc -fno-defer-pop -Wall -D_FILE_OFFSET_BITS=64 -D_REENTRANT -fPIC
> -c  -O  -I'/home/aa144/.opam/4.01.0/lib/ocaml' 'netsys_c.c'
> # netsys_c.c: In function 'netsys_openat':
> # netsys_c.c:482: error: 'fd' undeclared (first use in this function)
> # netsys_c.c:482: error: (Each undeclared identifier is reported only
> once
> # netsys_c.c:482: error: for each function it appears in.)
> # make[1]: *** [netsys_c.o] Error 2
> # make: *** [all] Error 2
> 
> 
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from 
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk
> _______________________________________________ Ocamlnet-devel mailing list Oca...@li... https://lists.sourceforge.net/lists/listinfo/ocamlnet-devel

-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
Creator of GODI and camlcity.org.
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------

[Ocamlnet-devel] installation error

[Ashish A. <aga...@gm...>]

With opam, on RHEL, I'm getting the error below. I've installed ocamlnet on
the same computer many times before, and don't think I changed anything.
Any ideas?

===== ERROR while installing ocamlnet.3.7.3 =====
# opam-version 1.1.0-beta (c381a2daa6751e5b649d96a6e2b4226c8ebe4366)
# os           linux
# command      make all
# path         /home/aa144/.opam/4.01.0/build/ocamlnet.3.7.3
# compiler     4.01.0
# exit-code    2
# env-file
/home/aa144/.opam/4.01.0/build/ocamlnet.3.7.3/ocamlnet-22544-ffb3fd.env
# stdout-file
 /home/aa144/.opam/4.01.0/build/ocamlnet.3.7.3/ocamlnet-22544-ffb3fd.out
# stderr-file
 /home/aa144/.opam/4.01.0/build/ocamlnet.3.7.3/ocamlnet-22544-ffb3fd.err
### stdout ###
# ...[truncated]
# ocamlfind ocamlc -g   -package "unix unix" -c  netsys_xdr.mli
# ocamlfind ocamlc -g   -package "unix unix"  -c  netsys_xdr.ml
# ocamlfind ocamlc -g   -package "unix unix" -c  netsys_tmp.mli
# ocamlfind ocamlc -g   -package "unix unix"  -c  netsys_tmp.ml
# ocamlfind ocamlc -g   -package "unix unix" -c  netsys_sem.mli
# ocamlfind ocamlc -g   -package "unix unix"  -c  netsys_sem.ml
# ocamlfind ocamlc -g   -package "unix unix" -c  netsys_pmanage.mli
# ocamlfind ocamlc -g   -package "unix unix"  -c  netsys_pmanage.ml
# ../../tools/stubcc -ocamlc ocamlc -ccopt -O   netsys_c.c
# make[1]: Leaving directory
`/home/aa144/.opam/4.01.0/build/ocamlnet.3.7.3/src/netsys'
### stderr ###
# + gcc -fno-defer-pop -Wall -D_FILE_OFFSET_BITS=64 -D_REENTRANT -fPIC -c
 -O  -I'/home/aa144/.opam/4.01.0/lib/ocaml' 'netsys_c.c'
# netsys_c.c: In function 'netsys_openat':
# netsys_c.c:482: error: 'fd' undeclared (first use in this function)
# netsys_c.c:482: error: (Each undeclared identifier is reported only once
# netsys_c.c:482: error: for each function it appears in.)
# make[1]: *** [netsys_c.o] Error 2
# make: *** [all] Error 2

Re: [Ocamlnet-devel] Ftp_client.FTP_protocol_violation "Unexpected control message".

[Sylvain Le G. <sy...@le...>]

On 02-10-2013, Sylvain Le Gall <sy...@le...> wrote:
> Hello,
>
> I was trying to use ocamlnet to access the FTP server of a my home NAS.
> It turns out that it is a very strange hardware and that you get
> automaticaly logged in when you connect.
>
> Here is the transcript (debug on):
> # client # exec (connect_method ~host:"wndr4500" ());;
> [Wed Oct  2 21:59:55 2013] [debug] [1052:0] Ftp_client: connecting to wndr4500:21
> [Wed Oct  2 21:59:55 2013] [debug] [1052:0] Ftp_client: connected to wndr4500:21
> [Wed Oct  2 21:59:55 2013] [debug] [1052:0] Ftp_client: ctrl received: 230 User logged in.
> Exception: Ftp_client.FTP_protocol_violation "Unexpected control message".
>
> I am using ocamlnet 3.5.1 (provided with Debian).
>
> Is there any workaround? Is it a bug?

FTR (with curl):
# perform t;;
Exception:
Curl.CurlException
 (CURLE_FTP_WEIRD_SERVER_REPLY, 8, "CURLE_FTP_WEIRD_SERVER_REPLY").


I found the way to setup a password on the share and now it is working,
as intended.

Cheers,
Sylvain Le Gall
-- 
Website:     http://sylvain.le-gall.net/
OCaml forge: http://forge.ocamlcore.org
OCaml blogs: http://planet.ocaml.org

[Ocamlnet-devel] Ftp_client.FTP_protocol_violation "Unexpected control message".

[Sylvain Le G. <sy...@le...>]

Hello,

I was trying to use ocamlnet to access the FTP server of a my home NAS.
It turns out that it is a very strange hardware and that you get
automaticaly logged in when you connect.

Here is the transcript (debug on):
# client # exec (connect_method ~host:"wndr4500" ());;
[Wed Oct  2 21:59:55 2013] [debug] [1052:0] Ftp_client: connecting to wndr4500:21
[Wed Oct  2 21:59:55 2013] [debug] [1052:0] Ftp_client: connected to wndr4500:21
[Wed Oct  2 21:59:55 2013] [debug] [1052:0] Ftp_client: ctrl received: 230 User logged in.
Exception: Ftp_client.FTP_protocol_violation "Unexpected control message".

I am using ocamlnet 3.5.1 (provided with Debian).

Is there any workaround? Is it a bug?

Thanks,
Sylvain Le Gall
-- 
Website:     http://sylvain.le-gall.net/
OCaml forge: http://forge.ocamlcore.org
OCaml blogs: http://planet.ocaml.org

Re: [Ocamlnet-devel] Shared Variable - Exception Invalid_argument("Marshal.from_size")

[Gerd S. <in...@ge...>]

Am Donnerstag, den 26.09.2013, 17:52 +0200 schrieb Davide Ramaglietta:
> 	Hi,
> 
> I'm sorry to bother you again. 
> I eventually got my asynchronous processor, thanks to your precious hints. 
> 
> I have now a new problem on which I'm stuck. I want to store a global state between all my processes (just an integer counter). 
> I followed the instructions that are in the "Netplex_advanced" documentation but when I try to get the value with Stats_var.get I get the following exception: 
> 
> Exception Invalid_argument("Marshal.from_size")
> 
> I think the problem arises because the variable misses the initialization. 
> If the cause of the exception is actually this, my problem is that I don't know how to access to the variable.
> 
> Any hint would be great.

Yes, this looks like a missing initialization. You should ensure that
Netplex_sharedvar.create_var is called once in every process (with param
enc:true), before you try to get or set values.


Gerd

> 
> Thank you for your time and consideration,
> Davide.
> 
> 
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from 
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
> _______________________________________________
> Ocamlnet-devel mailing list
> Oca...@li...
> https://lists.sourceforge.net/lists/listinfo/ocamlnet-devel
> 

-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
My OCaml site:          http://www.camlcity.org
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------

Re: [Ocamlnet-devel] [PATCH] Persistent connections and HTTPS

[Gerd S. <in...@ge...>]

Hi,

I finally fixed this problem the clean way, namely by storing the SSL
socket in the connection cache. This solution is now committed to trunk.

Gerd


Am Samstag, den 21.09.2013, 22:58 +0300 schrieb Török Edwin:
> Hi,
> 
> Attached patch implements a workaround to allow persistent connections to work with HTTPS,
> avoiding bugs in Https_client#continue. See below for the long explanation.
> 
> Persistent HTTP connections work if I enable the aggressive connection cache with Http_client.
> However with Https_client I noticed that it always closed and reopened the Ssl connections. 
> While this doesn't influence the application's correctness (thanks to the retry mechanisms in Http_client!), it heavily influences its performance: the latency is very bad due to repeated reopened connections, and repeated SSL handshakes ... even on localhost.
> 
> Enabling Netlog debugging showed that some Ssl exceptions were thrown each time around read/write/shutdown after the 1st query completed. Further investigation revealed that the problem might be with Https_client#continue:
> 
>        method continue fd cb tmo tmo_x host port esys =
> -	let mplex =
> -	  Uq_ssl.create_ssl_multiplex_controller
> -	    ~close_inactive_descr:true
> -	    ~preclose:(preclose fd)
> -	    ~initial_state:`Client
> -	    ~timeout:(tmo, tmo_x)
> -	    fd ctx esys in
> -	(mplex :> Uq_engines.multiplex_controller)
> 
> Problems with this code:
>   * creating a new ssl multiplex controller will create a new Ssl.socket (and share just the Ssl.context and fd).
>   * the new Ssl.socket doesn't have a defined state (SSL_set_connected_state was not called, and there was no prior handshake either), causing further operations on it to raise errors (if I add some debugging code
> to Uq_ssl to print Ssl.get_error_string):
> [Thu Sep 19 14:57:29 2013] [debug] [7534:11] Uq_ssl: SSL write error: error:140D0114:SSL routines:SSL_write:uninitialized
> [Thu Sep 19 14:57:29 2013] [debug] [7534:11] Uq_ssl: SSL read error: error:140DF114:SSL routines:SSL_read:uninitialized
> [Thu Sep 19 14:57:29 2013] [debug] [7534:11] Uq_ssl: SSL shutdown error: error:140E0114:SSL routines:SSL_shutdown:uninitialized
>   * the new Ssl.socket is probably missing the state of the handshake (session keys, etc.)
> 
>     
> The attached workaround implements a very simple workaround: if the esys is still the same
> just reuse the previous multiplex controller, I'm guessing this is what the Hashtbl was meant for anyway.
> 
> A better solution would be to have something like create_ssl_multiplex_controller_for_existing_session that would take the existing Ssl.socket instead of creating a new one, and then Https_client#continue wouldn't have to drop the connection if esys or tmo changes.
> 
> Best regards,
> --Edwin
> 
> ------------------------------------------------------------------------------
> LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99!
> 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint
> 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes
> Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/22/13. 
> http://pubads.g.doubleclick.net/gampad/clk?id=64545871&iu=/4140/ostg.clktrk
> _______________________________________________ Ocamlnet-devel mailing list Oca...@li... https://lists.sourceforge.net/lists/listinfo/ocamlnet-devel

-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
My OCaml site:          http://www.camlcity.org
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------

Re: [Ocamlnet-devel] Shared Variable - Exception Invalid_argument("Marshal.from_size")

[Davide R. <dav...@im...>]

Ok, I'm sure now. It is the Stats_ver.get operation that causes the exception. 



On Sep 26, 2013, at 5:52 PM, Davide Ramaglietta <dav...@im...> wrote:

> 	Hi,
> 
> I'm sorry to bother you again. 
> I eventually got my asynchronous processor, thanks to your precious hints. 
> 
> I have now a new problem on which I'm stuck. I want to store a global state between all my processes (just an integer counter). 
> I followed the instructions that are in the "Netplex_advanced" documentation but when I try to get the value with Stats_var.get I get the following exception: 
> 
> Exception Invalid_argument("Marshal.from_size")
> 
> I think the problem arises because the variable misses the initialization. 
> If the cause of the exception is actually this, my problem is that I don't know how to access to the variable.
> 
> Any hint would be great.
> 
> Thank you for your time and consideration,
> Davide.
> 
> 
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from 
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
> _______________________________________________
> Ocamlnet-devel mailing list
> Oca...@li...
> https://lists.sourceforge.net/lists/listinfo/ocamlnet-devel

[Ocamlnet-devel] Shared Variable - Exception Invalid_argument("Marshal.from_size")

[Davide R. <dav...@im...>]

	Hi,

I'm sorry to bother you again. 
I eventually got my asynchronous processor, thanks to your precious hints. 

I have now a new problem on which I'm stuck. I want to store a global state between all my processes (just an integer counter). 
I followed the instructions that are in the "Netplex_advanced" documentation but when I try to get the value with Stats_var.get I get the following exception: 

Exception Invalid_argument("Marshal.from_size")

I think the problem arises because the variable misses the initialization. 
If the cause of the exception is actually this, my problem is that I don't know how to access to the variable.

Any hint would be great.

Thank you for your time and consideration,
Davide.

Re: [Ocamlnet-devel] [Caml-list] Re: Multithreaded https requests in ocamlnet netclient

[Török E. <edw...@et...>]

On 11/28/2012 03:43 AM, Mike Lin wrote:
> Thanks, Gerd!
> 
> FWIW I could not reproduce the crash by using ocaml-ssl's blocking operations directly.
> 
> https://gist.github.com/4152047#file_ssl_threads.ml
> 
> This works fine- so, perhaps something nasty arises from using nonblocking I/O on ssl sockets from multiple threads. I'm sure if there is any other critical difference with how netclient/equeue-ssl and my example use ocaml-ssl.
> 
> I also don't have time to pursue this much further, so I will try to put all of my http operations on one thread as your example suggests.

I was debugging a similar bug, and found this old thread with a testcase.
I figured a way to fix it, see the patch below (it didn't acquire the OCaml master lock before raising an 
exception, causing Ocaml code to be executed in parallel with other OCaml code ... leading to all sorts of nasty situations).

Debugging thread-related bugs is hard, especially that none of the usual tools help here.
I modified st_posix.h a bit by adding an m->owner field and checking it against pthread_self() to make sure
a thread attempts to release only a lock it acquired itself, but there is more that could be done
(check in raise_with_arg/etc. that we do hold the master lock, check after returning from each C call that
we do hold the lock, same for C callbacks, etc.). 

Would it be possible to do add checks like this with '-runtime-variant d', i.e. can the thread implementation be changed to a "checking" one in that case?

The patch:
Index: src/equeue-ssl/ssl_exts_stubs.c
===================================================================
--- src/equeue-ssl/ssl_exts_stubs.c     (revision 1913)
+++ src/equeue-ssl/ssl_exts_stubs.c     (working copy)
@@ -27,6 +27,7 @@
   caml_enter_blocking_section();
   ret = SSL_shutdown(ssl);                                    
   if (ret == -1) {                                            
+      caml_leave_blocking_section();                          
       raise_with_arg(*caml_named_value("ssl_exn_shutdown_error"), 
                     Val_int(SSL_get_error(ssl, ret)));        
   };   

Re: [Ocamlnet-devel] [PATCH] Persistent connections and HTTPS

[Gerd S. <in...@ge...>]

Am Montag, den 23.09.2013, 18:10 +0300 schrieb Török Edwin:
> On 09/23/2013 01:56 PM, Gerd Stolpmann wrote:
> > Hi,
> > 
> > thanks for the patch. If I don't find another solution, I'll apply it.
> > 
> > Currently, I'm revising SSL anyway - OCamlnet 4.0 will have its own
> > binding for GnuTLS, and this will give us a lot more freedom in the
> > design. In particular, the management of the TCP connection and the TLS
> > tunnel can be completely decoupled (i.e. TLS will appear as a separate
> > layer on top of the multiplex controller).
> > 
> > (If you are curious:
> > https://godirepo.camlcity.org/wwwsvn/branches/onet4/code/src/nettls-gnutls/?root=lib-ocamlnet2)
> 
> Interesting, does this mean that an application could choose between OpenSSL and GnuTLS by simply
> linking with nettls.gnutls vs nettls.openssl? [*]
> 
> [*] Or why not someone could contribute a nettls.nss, as Fedora seems to prefer it over GnuTLS (Debian
> on the other hand seems to default to GnuTLS in curl for example):
> https://fedoraproject.org/wiki/FedoraCryptoConsolidation

There will be just a module type TLS_provider that can be packed as
value and passed around, and it will be possible to have several
implementations for it (switchable even at runtime). GnuTLS serves
"only" as the implementation that is included. I don't know yet whether
I personally want to create another one - the existing binding for
OpenSSL is so minimal that it is hardly usable for anything else than a
simple client, so it would be some work to add all the missing
functionality. But if somebody else did it, I'd happily include it in
Ocamlnet. Same for NSS (which is probably easier to do).

Independently of the TLS implementation, there will also be a parser for
X.509 certificates. This is already written (directly in OCaml), and
will give you complete access to all the details.


Gerd


> Best regards,
> --Edwin
> 
> 

-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
Creator of GODI and camlcity.org.
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------

Re: [Ocamlnet-devel] [PATCH] Persistent connections and HTTPS

[Török E. <edw...@et...>]

On 09/23/2013 01:56 PM, Gerd Stolpmann wrote:
> Hi,
> 
> thanks for the patch. If I don't find another solution, I'll apply it.
> 
> Currently, I'm revising SSL anyway - OCamlnet 4.0 will have its own
> binding for GnuTLS, and this will give us a lot more freedom in the
> design. In particular, the management of the TCP connection and the TLS
> tunnel can be completely decoupled (i.e. TLS will appear as a separate
> layer on top of the multiplex controller).
> 
> (If you are curious:
> https://godirepo.camlcity.org/wwwsvn/branches/onet4/code/src/nettls-gnutls/?root=lib-ocamlnet2)

Interesting, does this mean that an application could choose between OpenSSL and GnuTLS by simply
linking with nettls.gnutls vs nettls.openssl? [*]

[*] Or why not someone could contribute a nettls.nss, as Fedora seems to prefer it over GnuTLS (Debian
on the other hand seems to default to GnuTLS in curl for example):
https://fedoraproject.org/wiki/FedoraCryptoConsolidation

Best regards,
--Edwin

Re: [Ocamlnet-devel] How to implement an asynchronous processor without RPC

[Gerd S. <in...@ge...>]

Am Freitag, den 20.09.2013, 16:08 +0200 schrieb Davide Ramaglietta:
> Hi,
> 
> is there exist any possible way to implement a server using only Netplex (avoiding the RPC layer implication),
> which provides a asynchronous processor such that can handles several connection in parallel? 

Yes, of course. There is no example in the distribution, but look at
examples/netplex/hello_world.ml: The method [process] gets a file
descriptor fd, and it can do anything with it. fd is by default in
non-blocking mode. When you are done with fd, call [when_done].

You'll have to use the event_system of the Netplex container for
asynchronous processing, i.e. container#event_system, where container =
Netplex_cenv.self_cont() if you don't have it at hand otherwise.

Gerd


> Best regards,
> --Davide.
> ------------------------------------------------------------------------------
> LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99!
> 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint
> 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes
> Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. 
> http://pubads.g.doubleclick.net/gampad/clk?id=58041151&iu=/4140/ostg.clktrk
> _______________________________________________
> Ocamlnet-devel mailing list
> Oca...@li...
> https://lists.sourceforge.net/lists/listinfo/ocamlnet-devel
> 

-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
My OCaml site:          http://www.camlcity.org
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------

Re: [Ocamlnet-devel] [PATCH] Persistent connections and HTTPS

[Gerd S. <in...@ge...>]

Hi,

thanks for the patch. If I don't find another solution, I'll apply it.

Currently, I'm revising SSL anyway - OCamlnet 4.0 will have its own
binding for GnuTLS, and this will give us a lot more freedom in the
design. In particular, the management of the TCP connection and the TLS
tunnel can be completely decoupled (i.e. TLS will appear as a separate
layer on top of the multiplex controller).

(If you are curious:
https://godirepo.camlcity.org/wwwsvn/branches/onet4/code/src/nettls-gnutls/?root=lib-ocamlnet2)

Gerd



Am Samstag, den 21.09.2013, 22:58 +0300 schrieb Török Edwin:
> Hi,
> 
> Attached patch implements a workaround to allow persistent connections to work with HTTPS,
> avoiding bugs in Https_client#continue. See below for the long explanation.
> 
> Persistent HTTP connections work if I enable the aggressive connection cache with Http_client.
> However with Https_client I noticed that it always closed and reopened the Ssl connections. 
> While this doesn't influence the application's correctness (thanks to the retry mechanisms in Http_client!), it heavily influences its performance: the latency is very bad due to repeated reopened connections, and repeated SSL handshakes ... even on localhost.
> 
> Enabling Netlog debugging showed that some Ssl exceptions were thrown each time around read/write/shutdown after the 1st query completed. Further investigation revealed that the problem might be with Https_client#continue:
> 
>        method continue fd cb tmo tmo_x host port esys =
> -	let mplex =
> -	  Uq_ssl.create_ssl_multiplex_controller
> -	    ~close_inactive_descr:true
> -	    ~preclose:(preclose fd)
> -	    ~initial_state:`Client
> -	    ~timeout:(tmo, tmo_x)
> -	    fd ctx esys in
> -	(mplex :> Uq_engines.multiplex_controller)
> 
> Problems with this code:
>   * creating a new ssl multiplex controller will create a new Ssl.socket (and share just the Ssl.context and fd).
>   * the new Ssl.socket doesn't have a defined state (SSL_set_connected_state was not called, and there was no prior handshake either), causing further operations on it to raise errors (if I add some debugging code
> to Uq_ssl to print Ssl.get_error_string):
> [Thu Sep 19 14:57:29 2013] [debug] [7534:11] Uq_ssl: SSL write error: error:140D0114:SSL routines:SSL_write:uninitialized
> [Thu Sep 19 14:57:29 2013] [debug] [7534:11] Uq_ssl: SSL read error: error:140DF114:SSL routines:SSL_read:uninitialized
> [Thu Sep 19 14:57:29 2013] [debug] [7534:11] Uq_ssl: SSL shutdown error: error:140E0114:SSL routines:SSL_shutdown:uninitialized
>   * the new Ssl.socket is probably missing the state of the handshake (session keys, etc.)
> 
>     
> The attached workaround implements a very simple workaround: if the esys is still the same
> just reuse the previous multiplex controller, I'm guessing this is what the Hashtbl was meant for anyway.
> 
> A better solution would be to have something like create_ssl_multiplex_controller_for_existing_session that would take the existing Ssl.socket instead of creating a new one, and then Https_client#continue wouldn't have to drop the connection if esys or tmo changes.
> 
> Best regards,
> --Edwin
> 
> ------------------------------------------------------------------------------
> LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99!
> 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint
> 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes
> Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/22/13. 
> http://pubads.g.doubleclick.net/gampad/clk?id=64545871&iu=/4140/ostg.clktrk
> _______________________________________________ Ocamlnet-devel mailing list Oca...@li... https://lists.sourceforge.net/lists/listinfo/ocamlnet-devel

-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
My OCaml site:          http://www.camlcity.org
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------

[Ocamlnet-devel] [PATCH] Persistent connections and HTTPS

[Török E. <edw...@et...>]

Hi,

Attached patch implements a workaround to allow persistent connections to work with HTTPS,
avoiding bugs in Https_client#continue. See below for the long explanation.

Persistent HTTP connections work if I enable the aggressive connection cache with Http_client.
However with Https_client I noticed that it always closed and reopened the Ssl connections. 
While this doesn't influence the application's correctness (thanks to the retry mechanisms in Http_client!), it heavily influences its performance: the latency is very bad due to repeated reopened connections, and repeated SSL handshakes ... even on localhost.

Enabling Netlog debugging showed that some Ssl exceptions were thrown each time around read/write/shutdown after the 1st query completed. Further investigation revealed that the problem might be with Https_client#continue:

       method continue fd cb tmo tmo_x host port esys =
-	let mplex =
-	  Uq_ssl.create_ssl_multiplex_controller
-	    ~close_inactive_descr:true
-	    ~preclose:(preclose fd)
-	    ~initial_state:`Client
-	    ~timeout:(tmo, tmo_x)
-	    fd ctx esys in
-	(mplex :> Uq_engines.multiplex_controller)

Problems with this code:
  * creating a new ssl multiplex controller will create a new Ssl.socket (and share just the Ssl.context and fd).
  * the new Ssl.socket doesn't have a defined state (SSL_set_connected_state was not called, and there was no prior handshake either), causing further operations on it to raise errors (if I add some debugging code
to Uq_ssl to print Ssl.get_error_string):
[Thu Sep 19 14:57:29 2013] [debug] [7534:11] Uq_ssl: SSL write error: error:140D0114:SSL routines:SSL_write:uninitialized
[Thu Sep 19 14:57:29 2013] [debug] [7534:11] Uq_ssl: SSL read error: error:140DF114:SSL routines:SSL_read:uninitialized
[Thu Sep 19 14:57:29 2013] [debug] [7534:11] Uq_ssl: SSL shutdown error: error:140E0114:SSL routines:SSL_shutdown:uninitialized
  * the new Ssl.socket is probably missing the state of the handshake (session keys, etc.)

    
The attached workaround implements a very simple workaround: if the esys is still the same
just reuse the previous multiplex controller, I'm guessing this is what the Hashtbl was meant for anyway.

A better solution would be to have something like create_ssl_multiplex_controller_for_existing_session that would take the existing Ssl.socket instead of creating a new one, and then Https_client#continue wouldn't have to drop the connection if esys or tmo changes.

Best regards,
--Edwin

[Ocamlnet-devel] How to implement an asynchronous processor without RPC

[Davide R. <dav...@im...>]

Hi,

is there exist any possible way to implement a server using only Netplex (avoiding the RPC layer implication),
which provides a asynchronous processor such that can handles several connection in parallel? 

Best regards,
--Davide.

Re: [Ocamlnet-devel] netmcore: becoming a worker after fork+exec ?

[Gerd S. <in...@ge...>]

Am Sonntag, den 08.09.2013, 17:46 -0400 schrieb Piotr Mardziel:
> Hello Gerd, thank you for the information!
> 
> Would you say Netcamlbox are efficient modes (among the ocamlnet set of modules, or plain old pipes) of communicating frequent but small blocks of data with a fork-exec'ed program? Seems like within the constraints of OSX; this process will be the only one capable of performing drawing operations (via SDL and OpenGL) and will thus be getting many, frequent but small communications from the workers.

This sounds like a good match. The efficiency of Netcamlbox and
Netmulticore should be about the same, because they both rely on the
same internal functions for value copying. If you compare Netcamlbox
with, say Netmcore_queue:

 - Netcamlbox has limited message size and limited number of slots.
   Netmcore_queue contents can be as large as data fit into the    
   underlying mempool.
 - Netcamlbox allows it that several senders fill the mail slots
   independently and in parallel. Netmcore_queue needs to get a
   common lock when a new value is added, so only one sender can
   do it at a time.
 - Netcamlbox is of course more restricted in functionality: it's
   only a message box, and you cannot add your own special data
   structures as in Netmulticore.

So, there are pros and cons. If there is an upper bound of the message
size, Netcamlbox will probably work better.

If you compare with Unix domain sockets serving datagrams, Netcamlbox
will probably win, because you don't have to convert your values to
string, but simply can put them as-is into the mailbox. This means
camlbox_send copies the value from the local memory of the sender
process to the shared mailbox, and on the receiver side you have the
choice between camlbox_get returning a pointer directly into the
mailbox, or the safer but slower camlbox_get_copy. So if camlbox_get is
an option, the value is only copied once, not twice.

Hope this helps,

Gerd

> 
> Best,
> Piotr (Peter) Mardziel <pi...@gm...>
> http://www.cs.umd.edu/~piotrm
> 
> 
> 
> On Sep 7, 2013, at 5:44 PM, Gerd Stolpmann <in...@ge...> wrote:
> 
> > Hi Piotr,
> > 
> > I don't think this is even possible. The point is that fork() ensures
> > that master and worker have the mempool mapped at exactly the same
> > address (because the mapping is just inherited). If you exec in between,
> > the mempool is automatically unmapped in the child, and you would have
> > to map it again. But the process is re-initialized because of the exec,
> > and it is not sure that the address range is still free (and this is a
> > real problem, as processes are nowadays initialized with some randomness
> > to make addresses unpredictable), and so there is no good method to
> > re-map the mempool at the same address.
> > 
> > So that's the background why there isn't the option of passing file
> > names or descriptors to Netmcore_mempool.
> > 
> > I don't know what you have in mind, but maybe Netcamlbox is already
> > sufficient? This module doesn't suffer from this problem, because every
> > process can here map the shared memory block at different addresses.
> > 
> > Gerd
> > 
> > 
> > Am Samstag, den 07.09.2013, 00:16 -0400 schrieb Piotr Mardziel:
> >> Hello ocamlnet developers,
> >> 
> >> Is it possible to establish the master/worker relationship with a process that has been fork+exec'ed (or Netsys_posix.spawn'ed) but is of the same executable as the process doing Netmcore.startup ?
> >> 
> >> Let me explain. There is a problem on OSX in that forked child processes cannot access some OS functionality (for example SDL will not work on a forked child process). One has to exec in order to get this functionality back. I would like to be able to communicate with this process (of the same executable if need be) using the convenient netmcore modules but I don't see a way for this process to get a handle on resources managed by the master process. It seems like it is easy to get as far as the getting a handle to the shared memory from which a Netmcore_mempool is created (by simply passing its name as a command argument to exec), but one cannot actually create the resource suitable for interacting with the master and other workers, as far as I can tell.
> >> 
> >> Any info about this would be greatly appreciated.
> >> 
> >> Best,
> >> Piotr (Peter) Mardziel <pi...@gm...>
> >> http://www.cs.umd.edu/~piotrm
> >> 
> >> 
> >> 
> >> 
> >> ------------------------------------------------------------------------------
> >> Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
> >> Discover the easy way to master current and previous Microsoft technologies
> >> and advance your career. Get an incredible 1,500+ hours of step-by-step
> >> tutorial videos with LearnDevNow. Subscribe today and save!
> >> http://pubads.g.doubleclick.net/gampad/clk?id=58041391&iu=/4140/ostg.clktrk
> >> _______________________________________________
> >> Ocamlnet-devel mailing list
> >> Oca...@li...
> >> https://lists.sourceforge.net/lists/listinfo/ocamlnet-devel
> >> 
> > 
> > -- 
> > ------------------------------------------------------------
> > Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
> > My OCaml site:          http://www.camlcity.org
> > Contact details:        http://www.camlcity.org/contact.html
> > Company homepage:       http://www.gerd-stolpmann.de
> > ------------------------------------------------------------
> 
> 

-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
My OCaml site:          http://www.camlcity.org
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------

Re: [Ocamlnet-devel] netmcore: becoming a worker after fork+exec ?

[Piotr M. <pi...@gm...>]

Hello Gerd, thank you for the information!

Would you say Netcamlbox are efficient modes (among the ocamlnet set of modules, or plain old pipes) of communicating frequent but small blocks of data with a fork-exec'ed program? Seems like within the constraints of OSX; this process will be the only one capable of performing drawing operations (via SDL and OpenGL) and will thus be getting many, frequent but small communications from the workers.

Best,
Piotr (Peter) Mardziel <pi...@gm...>
http://www.cs.umd.edu/~piotrm



On Sep 7, 2013, at 5:44 PM, Gerd Stolpmann <in...@ge...> wrote:

> Hi Piotr,
> 
> I don't think this is even possible. The point is that fork() ensures
> that master and worker have the mempool mapped at exactly the same
> address (because the mapping is just inherited). If you exec in between,
> the mempool is automatically unmapped in the child, and you would have
> to map it again. But the process is re-initialized because of the exec,
> and it is not sure that the address range is still free (and this is a
> real problem, as processes are nowadays initialized with some randomness
> to make addresses unpredictable), and so there is no good method to
> re-map the mempool at the same address.
> 
> So that's the background why there isn't the option of passing file
> names or descriptors to Netmcore_mempool.
> 
> I don't know what you have in mind, but maybe Netcamlbox is already
> sufficient? This module doesn't suffer from this problem, because every
> process can here map the shared memory block at different addresses.
> 
> Gerd
> 
> 
> Am Samstag, den 07.09.2013, 00:16 -0400 schrieb Piotr Mardziel:
>> Hello ocamlnet developers,
>> 
>> Is it possible to establish the master/worker relationship with a process that has been fork+exec'ed (or Netsys_posix.spawn'ed) but is of the same executable as the process doing Netmcore.startup ?
>> 
>> Let me explain. There is a problem on OSX in that forked child processes cannot access some OS functionality (for example SDL will not work on a forked child process). One has to exec in order to get this functionality back. I would like to be able to communicate with this process (of the same executable if need be) using the convenient netmcore modules but I don't see a way for this process to get a handle on resources managed by the master process. It seems like it is easy to get as far as the getting a handle to the shared memory from which a Netmcore_mempool is created (by simply passing its name as a command argument to exec), but one cannot actually create the resource suitable for interacting with the master and other workers, as far as I can tell.
>> 
>> Any info about this would be greatly appreciated.
>> 
>> Best,
>> Piotr (Peter) Mardziel <pi...@gm...>
>> http://www.cs.umd.edu/~piotrm
>> 
>> 
>> 
>> 
>> ------------------------------------------------------------------------------
>> Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
>> Discover the easy way to master current and previous Microsoft technologies
>> and advance your career. Get an incredible 1,500+ hours of step-by-step
>> tutorial videos with LearnDevNow. Subscribe today and save!
>> http://pubads.g.doubleclick.net/gampad/clk?id=58041391&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Ocamlnet-devel mailing list
>> Oca...@li...
>> https://lists.sourceforge.net/lists/listinfo/ocamlnet-devel
>> 
> 
> -- 
> ------------------------------------------------------------
> Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
> My OCaml site:          http://www.camlcity.org
> Contact details:        http://www.camlcity.org/contact.html
> Company homepage:       http://www.gerd-stolpmann.de
> ------------------------------------------------------------

Re: [Ocamlnet-devel] How to handle errors from Https_client?

[Gerd S. <in...@ge...>]

Hi,

this is a bug in the error path. The fix was quite simple. Find it in
svn for the time being (revs 1906 and 1907):

https://godirepo.camlcity.org/svn/lib-ocamlnet2/trunk/code/

I don't see a workaround outside ocamlnet.

Gerd

Am Dienstag, den 03.09.2013, 10:37 +0300 schrieb Török Edwin:
> Hi,
> 
> Using Https_client with OCamlnet 3.6.3 and 3.7.3 some exceptions seem to escape the pipeline.
> For example if I take examples/netclient/simple/http_download.ml, and initialize Https_client according to the docs and trigger an https error on purpose (try to use HTTPS on an HTTP port):
> Fatal error: exception Invalid_argument("remove_resource: the group is terminated")
> 
> Also if I take examples/netclient/simple/http_mt.ml and initialize Https_client as the docs suggest I get
> this on an https error:
> Thread 1 killed on uncaught exception Invalid_argument("remove_resource: the group is terminated")
> 
> With Http_client and pipeline#add_with_callback everything is fine: I get my callback invoked,
> and I can check http_call#status for the actual error, and the pipeline keeps working.
> 
> With Https_client and the Convenience module I get the exception but since it creates a new pipeline each time I can just wrap it and catch the exception there, and keep using it after the first error.
> With the pipeline it looks like after the first https error the pipeline becomes unusable.
> 
> What am I doing wrong?
> 
> I have attached a testcase, let me know if you need more info.
> 
> Best regards,
> --Edwin
> 
> 
> ------------------------------------------------------------------------------
> Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
> Discover the easy way to master current and previous Microsoft technologies
> and advance your career. Get an incredible 1,500+ hours of step-by-step
> tutorial videos with LearnDevNow. Subscribe today and save!
> http://pubads.g.doubleclick.net/gampad/clk?id=58040911&iu=/4140/ostg.clktrk
> _______________________________________________ Ocamlnet-devel mailing list Oca...@li... https://lists.sourceforge.net/lists/listinfo/ocamlnet-devel

-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
My OCaml site:          http://www.camlcity.org
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------

Re: [Ocamlnet-devel] netmcore: becoming a worker after fork+exec ?

[Gerd S. <in...@ge...>]

Hi Piotr,

I don't think this is even possible. The point is that fork() ensures
that master and worker have the mempool mapped at exactly the same
address (because the mapping is just inherited). If you exec in between,
the mempool is automatically unmapped in the child, and you would have
to map it again. But the process is re-initialized because of the exec,
and it is not sure that the address range is still free (and this is a
real problem, as processes are nowadays initialized with some randomness
to make addresses unpredictable), and so there is no good method to
re-map the mempool at the same address.

So that's the background why there isn't the option of passing file
names or descriptors to Netmcore_mempool.

I don't know what you have in mind, but maybe Netcamlbox is already
sufficient? This module doesn't suffer from this problem, because every
process can here map the shared memory block at different addresses.

Gerd


Am Samstag, den 07.09.2013, 00:16 -0400 schrieb Piotr Mardziel:
> Hello ocamlnet developers,
> 
> Is it possible to establish the master/worker relationship with a process that has been fork+exec'ed (or Netsys_posix.spawn'ed) but is of the same executable as the process doing Netmcore.startup ?
> 
> Let me explain. There is a problem on OSX in that forked child processes cannot access some OS functionality (for example SDL will not work on a forked child process). One has to exec in order to get this functionality back. I would like to be able to communicate with this process (of the same executable if need be) using the convenient netmcore modules but I don't see a way for this process to get a handle on resources managed by the master process. It seems like it is easy to get as far as the getting a handle to the shared memory from which a Netmcore_mempool is created (by simply passing its name as a command argument to exec), but one cannot actually create the resource suitable for interacting with the master and other workers, as far as I can tell.
> 
> Any info about this would be greatly appreciated.
> 
> Best,
> Piotr (Peter) Mardziel <pi...@gm...>
> http://www.cs.umd.edu/~piotrm
> 
> 
> 
> 
> ------------------------------------------------------------------------------
> Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
> Discover the easy way to master current and previous Microsoft technologies
> and advance your career. Get an incredible 1,500+ hours of step-by-step
> tutorial videos with LearnDevNow. Subscribe today and save!
> http://pubads.g.doubleclick.net/gampad/clk?id=58041391&iu=/4140/ostg.clktrk
> _______________________________________________
> Ocamlnet-devel mailing list
> Oca...@li...
> https://lists.sourceforge.net/lists/listinfo/ocamlnet-devel
> 

-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
My OCaml site:          http://www.camlcity.org
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------

[Ocamlnet-devel] netmcore: becoming a worker after fork+exec ?

[Piotr M. <pi...@gm...>]

Hello ocamlnet developers,

Is it possible to establish the master/worker relationship with a process that has been fork+exec'ed (or Netsys_posix.spawn'ed) but is of the same executable as the process doing Netmcore.startup ?

Let me explain. There is a problem on OSX in that forked child processes cannot access some OS functionality (for example SDL will not work on a forked child process). One has to exec in order to get this functionality back. I would like to be able to communicate with this process (of the same executable if need be) using the convenient netmcore modules but I don't see a way for this process to get a handle on resources managed by the master process. It seems like it is easy to get as far as the getting a handle to the shared memory from which a Netmcore_mempool is created (by simply passing its name as a command argument to exec), but one cannot actually create the resource suitable for interacting with the master and other workers, as far as I can tell.

Any info about this would be greatly appreciated.

Best,
Piotr (Peter) Mardziel <pi...@gm...>
http://www.cs.umd.edu/~piotrm

[Ocamlnet-devel] How to handle errors from Https_client?

[Török E. <edw...@et...>]

Hi,

Using Https_client with OCamlnet 3.6.3 and 3.7.3 some exceptions seem to escape the pipeline.
For example if I take examples/netclient/simple/http_download.ml, and initialize Https_client according to the docs and trigger an https error on purpose (try to use HTTPS on an HTTP port):
Fatal error: exception Invalid_argument("remove_resource: the group is terminated")

Also if I take examples/netclient/simple/http_mt.ml and initialize Https_client as the docs suggest I get
this on an https error:
Thread 1 killed on uncaught exception Invalid_argument("remove_resource: the group is terminated")

With Http_client and pipeline#add_with_callback everything is fine: I get my callback invoked,
and I can check http_call#status for the actual error, and the pipeline keeps working.

With Https_client and the Convenience module I get the exception but since it creates a new pipeline each time I can just wrap it and catch the exception there, and keep using it after the first error.
With the pipeline it looks like after the first https error the pipeline becomes unusable.

What am I doing wrong?

I have attached a testcase, let me know if you need more info.

Best regards,
--Edwin

[Ocamlnet-devel] [ANN] Ocamlnet-3.7.3

[Gerd S. <in...@ge...>]

Hi,

Ocamlnet-3.7.3 is now available for download. Changes in this release:

 - Support for ocaml-4.01
 - New directives for ocamlrpcgen to control the language mapping better
 - Bug fixes here and there

Details about this release can be found on the project page:

http://projects.camlcity.org/projects/ocamlnet.html

See ChageLog for a detailed list of changes:

http://projects.camlcity.org/projects/dl/ocamlnet-3.7.3/ChangeLog

Gerd


PS. The versions 3.7.0 to 3.7.2 were release attempts but turned out as
buggy. Don't use them if you fetched one (they were visible for a short
period).
-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
My OCaml site:          http://www.camlcity.org
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------