ocamlnet-devel — Developer discussion

Re: [Ocamlnet-devel] Safe to use RPC-SSL ?

[Gerd S. <in...@ge...>]

Am Montag, den 26.08.2013, 15:26 +0200 schrieb Thomas Calderon:
> Hi there,
> 
> 
> The example code qserver_auth_ssl.ml has the following header:
>   (* Configure qserver for SSL authentication (UNSAFE) *)
> 
> 
> Can someone explain why the code is considered unsafe ?

Well, I wrote that, but honestly I don't remember. The code looks good.

Gerd


> 
> 
> Thanks.
> ------------------------------------------------------------------------------
> Introducing Performance Central, a new site from SourceForge and 
> AppDynamics. Performance Central is your source for news, insights, 
> analysis and resources for efficient Application Performance Management. 
> Visit us today!
> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk
> _______________________________________________ Ocamlnet-devel mailing list Oca...@li... https://lists.sourceforge.net/lists/listinfo/ocamlnet-devel

-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
My OCaml site:          http://www.camlcity.org
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------

[Ocamlnet-devel] Safe to use RPC-SSL ?

[Thomas C. <cal...@gm...>]

Hi there,

The example code *qserver_auth_ssl.ml *has the following header:
  (* Configure qserver for SSL authentication (UNSAFE) *)

Can someone explain why the code is considered unsafe ?

Thanks.

Re: [Ocamlnet-devel] Compilation issues with ocaml 4.01(beta1)

[Gerd S. <in...@ge...>]

Am Mittwoch, den 21.08.2013, 15:57 +0200 schrieb Jacques-Pascal Deplaix:
> Hi,
> 
> ocamlnet is unable to compile with ocamlnet 4.01(beta1) due to some
> changes (especially O_CLOEXEC added in Unix.open_flag).
> 
> Some fixes are available here:
> https://github.com/OCamlPro/opam-repository/commit/7dca72113f57cbfa74f698de969ed38acdcc6f69

I don't understand the first one.

The second one (double stat) is already in 3.6.6.

> For the rest (O_CLOEXEC) more recently added, it need the same changes
> than for O_SHARE_DELETE
> (https://godirepo.camlcity.org/wwwsvn?rev=1790&root=lib-ocamlnet2&view=rev).
> 
> I hope it will be fixed and released soon, and thanks in advance,

Probably later this week.

Gerd

> 
> Cheers,
> 
> ------------------------------------------------------------------------------
> Introducing Performance Central, a new site from SourceForge and 
> AppDynamics. Performance Central is your source for news, insights, 
> analysis and resources for efficient Application Performance Management. 
> Visit us today!
> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk
> _______________________________________________
> Ocamlnet-devel mailing list
> Oca...@li...
> https://lists.sourceforge.net/lists/listinfo/ocamlnet-devel
> 

-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
My OCaml site:          http://www.camlcity.org
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------

Re: [Ocamlnet-devel] Compilation issues with ocaml 4.01(beta1)

[Jacques-Pascal D. <jp....@gm...>]

Ok, thanks.

This is not my patches, someone told me that it was supposed to solve 
the compilation issue (at least before the O_CLOEXEC addition), so I 
linked them here.

But maybe the first one is related to the change « Reject multiple 
declarations of the same method or instance variable in an object ».

On 08/21/2013 06:32 PM, Gerd Stolpmann wrote:
> Am Mittwoch, den 21.08.2013, 15:57 +0200 schrieb Jacques-Pascal Deplaix:
>> Hi,
>>
>> ocamlnet is unable to compile with ocamlnet 4.01(beta1) due to some
>> changes (especially O_CLOEXEC added in Unix.open_flag).
>>
>> Some fixes are available here:
>> https://github.com/OCamlPro/opam-repository/commit/7dca72113f57cbfa74f698de969ed38acdcc6f69
> I don't understand the first one.
>
> The second one (double stat) is already in 3.6.6.
>
>> For the rest (O_CLOEXEC) more recently added, it need the same changes
>> than for O_SHARE_DELETE
>> (https://godirepo.camlcity.org/wwwsvn?rev=1790&root=lib-ocamlnet2&view=rev).
>>
>> I hope it will be fixed and released soon, and thanks in advance,
> Probably later this week.
>
> Gerd
>
>> Cheers,
>>
>> ------------------------------------------------------------------------------
>> Introducing Performance Central, a new site from SourceForge and
>> AppDynamics. Performance Central is your source for news, insights,
>> analysis and resources for efficient Application Performance Management.
>> Visit us today!
>> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Ocamlnet-devel mailing list
>> Oca...@li...
>> https://lists.sourceforge.net/lists/listinfo/ocamlnet-devel
>>

[Ocamlnet-devel] Compilation issues with ocaml 4.01(beta1)

[Jacques-Pascal D. <jp....@gm...>]

Hi,

ocamlnet is unable to compile with ocamlnet 4.01(beta1) due to some
changes (especially O_CLOEXEC added in Unix.open_flag).

Some fixes are available here:
https://github.com/OCamlPro/opam-repository/commit/7dca72113f57cbfa74f698de969ed38acdcc6f69

For the rest (O_CLOEXEC) more recently added, it need the same changes
than for O_SHARE_DELETE
(https://godirepo.camlcity.org/wwwsvn?rev=1790&root=lib-ocamlnet2&view=rev).

I hope it will be fixed and released soon, and thanks in advance,

Cheers,

[Ocamlnet-devel] [ANN] ocamlnet-3.6.6

[Gerd S. <in...@ge...>]

Hi,

I'm happy to announce ocamlnet-3.6.6. This is a pure bug fixing release.

Details about this release can be found on the project page:

http://projects.camlcity.org/projects/ocamlnet.html

See ChageLog for a detailed list of changes:

http://projects.camlcity.org/projects/dl/ocamlnet-3.6.6/ChangeLog

Gerd
-- 
Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
Creator of GODI and camlcity.org.
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
*** Searching for new projects! Need consulting for system
*** programming in Ocaml? Gerd Stolpmann can help you.

Re: [Ocamlnet-devel] Getting configuration variables within hooks in Netplex

[Thomas C. <cal...@gm...>]

Hi,

Thanks for the reply. I had tried something like that, but somehow I missed
it.

Cheers.

Thomas.


On Thu, Jul 11, 2013 at 4:38 PM, Gerd Stolpmann <in...@ge...>wrote:

> Am Donnerstag, den 11.07.2013, 16:14 +0200 schrieb Thomas Calderon:
> > Hi there,
> >
> >
> > I would like to know if it is possible retrieve configuration parameters
> from a hook.
> > I get that the variable goes down from the configure and setup
> functions. However I would like to make the variable accessible from the
> hook (see example below).
> >
> >
> > Example below:
> >
> >
> > ...
> > let setup srv my_variable =
> >   ...
> >   (register some rpc)
> >   ...
> >   my_variable
> >
> >
> >
> >
> > let configure cf addr =
> >   let my_variable =
> >     try
> >       cf # string_param (cf # resolve_parameter addr "my_variable")
> >     with
> >       | Not_found ->
> >       failwith "Required parameter my_variable is missing!" in
> >   my_variable
> > ;;
> >
> >
> > let rpc_my_factory =
> >   Rpc_netplex.rpc_factory
> >     ~configure
> >     ~name:"my_rpc"
> >     ~setup
> >     ~hooks:(fun _ ->
> >           object(self)
> >         inherit Netplex_kit.empty_processor_hooks()
> >         method post_start_hook _ =
> >           Do_Something my_variable;                               (*
> here *)
> >
> >           ()
> >           end
> >          )
> >     ()
>
> Yes, this is possible: the output of ~configure is the input of ~hooks:
>
> let rpc_my_factory =
>   Rpc_netplex.rpc_factory
>     ~configure
>     ~name:"my_rpc"
>     ~setup
>     ~hooks:(fun my_variable ->    (* look here *)
>           object(self)
>             inherit Netplex_kit.empty_processor_hooks()
>             method post_start_hook _ =
>               Do_Something my_variable;
>               ()
>           end
>          )
>     ()
>
>
> Gerd
>
> --
> ------------------------------------------------------------
> Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
> Creator of GODI and camlcity.org.
> Contact details:        http://www.camlcity.org/contact.html
> Company homepage:       http://www.gerd-stolpmann.de
> ------------------------------------------------------------
>
>

Re: [Ocamlnet-devel] Getting configuration variables within hooks in Netplex

[Gerd S. <in...@ge...>]

Am Donnerstag, den 11.07.2013, 16:14 +0200 schrieb Thomas Calderon:
> Hi there,
> 
> 
> I would like to know if it is possible retrieve configuration parameters from a hook.
> I get that the variable goes down from the configure and setup functions. However I would like to make the variable accessible from the hook (see example below).
> 
> 
> Example below:
> 
> 
> ...
> let setup srv my_variable = 
>   ...
>   (register some rpc)
>   ...
>   my_variable
> 
> 
> 
> 
> let configure cf addr =
>   let my_variable =
>     try
>       cf # string_param (cf # resolve_parameter addr "my_variable")
>     with
>       | Not_found ->
>       failwith "Required parameter my_variable is missing!" in
>   my_variable
> ;;
> 
> 
> let rpc_my_factory =
>   Rpc_netplex.rpc_factory
>     ~configure
>     ~name:"my_rpc"
>     ~setup
>     ~hooks:(fun _ ->
>           object(self)
>         inherit Netplex_kit.empty_processor_hooks()
>         method post_start_hook _ =
>           Do_Something my_variable;                               (* here *)
> 
>           ()
>           end
>          )
>     ()

Yes, this is possible: the output of ~configure is the input of ~hooks:

let rpc_my_factory =
  Rpc_netplex.rpc_factory
    ~configure
    ~name:"my_rpc"
    ~setup
    ~hooks:(fun my_variable ->    (* look here *)
          object(self)
            inherit Netplex_kit.empty_processor_hooks()
            method post_start_hook _ =
              Do_Something my_variable;
              ()
          end
         )
    ()


Gerd

-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
Creator of GODI and camlcity.org.
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------

[Ocamlnet-devel] Getting configuration variables within hooks in Netplex

[Thomas C. <cal...@gm...>]

Hi there,

I would like to know if it is possible retrieve configuration parameters
from a hook.
I get that the variable goes down from the configure and setup functions.
However I would like to make the variable accessible from the hook (see
example below).

Example below:

...
let setup srv my_variable =
  ...
  (register some rpc)
  ...
  my_variable


let configure cf addr =
  let my_variable =
    try
      cf # string_param (cf # resolve_parameter addr "my_variable")
    with
      | Not_found ->
      failwith "Required parameter my_variable is missing!" in
  my_variable
;;

let rpc_my_factory =
  Rpc_netplex.rpc_factory
    ~configure
    ~name:"my_rpc"
    ~setup
    ~hooks:(fun _ ->
          object(self)
        inherit Netplex_kit.empty_processor_hooks()
        method post_start_hook _ =
          Do_Something my_variable;                               (* here *)
          ()
          end
         )
    ()

Re: [Ocamlnet-devel] 3.6.x contains duplicate method in netpop triggering a new OCaml 4.01.0 error

[Gerd S. <in...@ge...>]

Thanks, David, for the report. I fixed that in svn.

Gerd

Am Donnerstag, den 13.06.2013, 14:24 +0100 schrieb David Sheets:
> 3.6.x contains duplicate method, "stat", in netpop.ml which triggers a
> new OCaml 4.01.0 error.
> 
> This causes 3.6.x and forward dependencies to fail to build under
> 4.01.0+dev in OPAM.
> 
> See <http://caml.inria.fr/mantis/view.php?id=6035> and
> <https://github.com/OCamlPro/opam-repository/pull/796> for more info
> and the OPAM patch.
> 
> Thanks,
> 
> David
> 
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Windows:
> 
> Build for Windows Store.
> 
> http://p.sf.net/sfu/windows-dev2dev
> _______________________________________________
> Ocamlnet-devel mailing list
> Oca...@li...
> https://lists.sourceforge.net/lists/listinfo/ocamlnet-devel
> 

[Ocamlnet-devel] 3.6.x contains duplicate method in netpop triggering a new OCaml 4.01.0 error

[David S. <sh...@al...>]

3.6.x contains duplicate method, "stat", in netpop.ml which triggers a
new OCaml 4.01.0 error.

This causes 3.6.x and forward dependencies to fail to build under
4.01.0+dev in OPAM.

See <http://caml.inria.fr/mantis/view.php?id=6035> and
<https://github.com/OCamlPro/opam-repository/pull/796> for more info
and the OPAM patch.

Thanks,

David

[Ocamlnet-devel] [ANN] Ocamlnet-3.6.5

[Gerd S. <in...@ge...>]

Well, things go wrong...

There was a build problem in 3.6.4 so that netstring-pcre did not work
properly. This is fixed in the new versiopn 3.6.5 I just released.

Gerd


Am Montag, den 03.06.2013, 13:58 +0200 schrieb Gerd Stolpmann:
> Hi,
> 
> I've just released Ocamlnet-3.6.4. This is a maintenance release
> including:
> 
>  - New configure options for PCRE (-enable-full-pcre, -enable-pcre).
>    There is also documentation about the PCRE issue in Regexp.html
>    (remember that PCRE is no longer the default regexp engine).
>  - More documentation for Netmulticore: Netmcore_basics
>  - New Netplex module for mailboxes: Netplex_mbox.
>  - netcgi2-apache builds against apache-2.4
> 
> plus various smaller fixes and additions.
> 
> For a full description, see the ChangeLog.
> 
> Get Ocamlnet, read the manual etc. from  
> http://projects.camlcity.org/projects/ocamlnet.html
> 
> Gerd
> -- 
> ------------------------------------------------------------
> Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany 
> ge...@ge...          http://www.gerd-stolpmann.de
> Phone: +49-6151-153855                  Fax: +49-6151-997714
> ------------------------------------------------------------
> 
> 
> 

[Ocamlnet-devel] [ANN] Ocamlnet-3.6.4

[Gerd S. <in...@ge...>]

Hi,

I've just released Ocamlnet-3.6.4. This is a maintenance release
including:

 - New configure options for PCRE (-enable-full-pcre, -enable-pcre).
   There is also documentation about the PCRE issue in Regexp.html
   (remember that PCRE is no longer the default regexp engine).
 - More documentation for Netmulticore: Netmcore_basics
 - New Netplex module for mailboxes: Netplex_mbox.
 - netcgi2-apache builds against apache-2.4

plus various smaller fixes and additions.

For a full description, see the ChangeLog.

Get Ocamlnet, read the manual etc. from  
http://projects.camlcity.org/projects/ocamlnet.html

Gerd
-- 
------------------------------------------------------------
Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany 
ge...@ge...          http://www.gerd-stolpmann.de
Phone: +49-6151-153855                  Fax: +49-6151-997714
------------------------------------------------------------

Re: [Ocamlnet-devel] Seeking advice for a couple of issues

[Gerd S. <in...@ge...>]

Am 02.04.2013 20:47:36 schrieb(en) jde...@vf...:
> The fix for chunked request processing seems to work well.  Quite a
> drastic improvement in processing time. My latest tests are showing
> processing time of slightly greater than 2 seconds to process 10,000
> rows/documents from a CouchDb get operation. Just a thought, is it
> necessary for each chunk to be processed by the event system?  If all
> the chunks were treated as one event (decoupled), I'd bet that the
> processing time would drop significantly.
> 
> I'm having some trouble using the skip_challenge option on the Basic
> Auth handler.
> 
> The following snippet is my latest attempt:

Hi,

Sorry for the late response. You can fix it with:

> let () =
>    let keys = new Http_client.key_ring() in
>    let key = Http_client.key ~user:"admin" ~password:"admin" ~realm:""
> 			    ~domain:["http://localhost:5984"] in

   ~realm:"anywhere"
   ~domain:["http://localhost:5984/"]

I did not mention the "anywhere" change in my latest email, but it is  
documented in the mli. The domain string must include a final slash.

Gerd


>    keys#add_key key;
>    let auth = new Http_client.basic_auth_handler  
> ~enable_auth_in_advance:true
> 		 ~skip_challenge:(Some "*") keys in
>    let pipeline = new Http_client.pipeline in
>    pipeline # add_auth_handler auth;
>    let call = new Http_client.get "http://localhost:5984/camldb" in
>    let hdr = call#request_header `Base in
>    hdr#update_field "Content-type" "application/json";
>    pipeline#add call;
>    pipeline#run();
>    printf "rc=%d\n" (call#response_status_code);
>    let hdr = call#request_header `Effective in
>    List.iter (fun (k, v) -> printf "%s: %s\n" k v) hdr#fields;
> 
> This yields a return code 401 with no Authorization header added to
> the sent GET request.  No luck with PUT or DELETE requests either.
> 
> Thanks again for all your help.  I've enjoyed working with your  
> library.
> 
> Regards,
> John
> 
> Quoting "Gerd Stolpmann" <in...@ge...>:
> 
> > Am 29.03.2013 01:11:39 schrieb(en) John Derrick:
> >> Thanks for the reply, Gerd.
> >>
> >> First, I should mention I'm using Ocaml 4.001, the Mingw build on
> >> Win7.  I'm
> >> using Ocamlnet version 3.6.1 (or newer).
> >>
> >> I tried using ~enable_auth_in_advance as you suggested but was  
> unable
> >> to get
> >> it working.  I must be doing something wrong.
> >> Here is my latest attempt:
> >
> > Confirmed, this does not work. I had this wrong in my mind -
> > auth_in_advance does not work when the server never generates a
> > challenge.
> >
> > I've added a new mode skip_challenge, e.g.
> >
> > let auth = new Http_client.basic_auth_handler ~skip_challenge:(Some
> > "*") keys
> >
> > This is set to "*" or to the domain URL (e.g. "http://localhost/")  
> to
> > which it applies.
> >
> >> method private do_run_pipe call =
> >>   let keys = new Http_client.key_ring() in
> >>   let auth = new Http_client.basic_auth_handler
> >> ~enable_auth_in_advance:true
> >> keys in
> >>   let key = Http_client.key ~user:"admin" ~password:"admin"  
> ~realm:""
> >>     ~domain:["localhost"] in
> >
> > This is also a problem in the current code: you would have to set
> > domain to "http://localhost:80/". Not really well documented. The  
> new
> > version also allows now "*", and you can omit the port number.
> >
> > That the domain is a full URL has to do with the definitions in the
> > HTTP protocol.
> >
> >>   keys#add_key key;
> >>   let pipeline = new Http_client.pipeline in
> >>   pipeline # add_auth_handler auth;
> >>   let hdr = call#request_header `Base in
> >>   hdr#update_field "Content-type" "application/json";
> >>   pipeline#add call;
> >>   pipeline#run();
> >>   call#response_status_code
> >>
> >> Packet capture shows that no Authorization header was added to a  
> PUT
> >> request
> >> with the above block of code.  Return code is 401.
> >>
> >> On the second issue, I have TCP_NODELAY set to true in the CouchDb
> >> config.
> >> The process is using heavy CPU while it runs.  Do you think the  
> issue
> >> could
> >> be related to OS and/or OCaml build?
> >
> > Turns out this is in deed a problem with event handling, namely that
> > there was some progress counting, and this resulted in a performance
> > bug when the chain of event handling gets long.
> >
> > The new version is not yet released, but you can grab it from the  
> svn
> > server: https://godirepo.camlcity.org/svn/lib-ocamlnet2/trunk/
> >
> > Gerd
> >
> >
> >> Thanks again for your help,
> >> John
> >>
> >> -----Original Message-----
> >> From: Gerd Stolpmann
> >> Sent: Thursday, March 28, 2013 9:06 AM
> >> To: John Derrick
> >> Cc: oca...@li...
> >> Subject: AW: [Ocamlnet-devel] Seeking advice for a couple of issues
> >>
> >> Am 27.03.2013 23:41:36 schrieb(en) John Derrick:
> >> > Hi All,
> >> >
> >> > I've been working with Http_client to develop a no-frills CouchDb
> >> > driver. I'm relatively new to OCaml, but have many years of
> >> experience
> >> > with various imperative langs.
> >> >
> >> > I'm looking for help with a couple of issues. My first issue is
> >> Basic
> >> > Authentication. I've noticed that Basic Auth and Basic Auth  
> Session
> >> > do not include the "Authorization:" header when making GET or PUT
> >> > requests UNLESS a challenge is received. Is this the expected
> >> > behavior? Not a serious issue as it's easy to add my own
> >> > Authorization entry to the header. Apparently CouchDb does not
> >> send  an
> >> > auth challenge.
> >>
> >> It is in deed possible to enable this behavior (it's off by  
> default):
> >>
> >>   - Create the authentication handler like this:
> >>
> >>     let keys = new Http_client.key_ring()
> >>     let auth = new Http_client.basic_auth_handler
> >>                   ~enable_auth_in_advance:true keys
> >>
> >>     The enable_auth_in_advance option makes it working.
> >>
> >>   - Add credentials:
> >>
> >>     let key = Http_client.key ...
> >>     keys # add_key key
> >>
> >>   - Add this to the pipeline:
> >>
> >>     pipeline # add_auth_handler auth
> >>
> >> Well, this behavior has always been a matter of discussion.  
> Normally,
> >> you don't want to send credentials before you know that the server  
> is
> >> really the server (remember that you can run http over ssl, and the
> >> ssl
> >> connection authenticates the server), and this way it is specified  
> in
> >> RFC 2617. Historically, though, browsers have always implemented  
> some
> >> form of sending the credentials without prior challenge, namely  
> when
> >> the web page was visited the second time.
> >>
> >> > The second issue is chunked request performance. When peforming a
> >> > query against CouchDb (say _all_docs for example) via a GET  
> request,
> >> > Ocamlnet seems to have issues processing the returned data.  
> CouchDb
> >> > returns many small blocks of data in chunked format. Processing a
> >> > collection of 1000 docs can take several minutes. I'm not sure if
> >> > this relates to the event system performance or socket  
> performance.
> >> > Does any one know of a solution to this issue?
> >>
> >> I don't think it has to do with the event processing, as the OS
> >> buffers
> >> data up if the client is not fast enough, so for the next read()  
> the
> >> client gets many chunks at once. Also, 1000 events (if they are  
> really
> >> generated) are not really much, and should not take minutes, but  
> only
> >> milliseconds.
> >>
> >> Does the client consume a lot of CPU? Or is it just waiting?
> >>
> >> In theory there could be a bad interaction with the TCP buffering,
> >> especially when CouchDB disables the Nagle algorithm (TCP_NODELAY),
> >> which it probably does. But you report a really bad behavior, so  
> I've
> >> actually no clue here.
> >>
> >> Gerd
> >>
> >>
> >> >
> >> > Thanks to all who contributed to the development of Ocamlnet.  
> It's
> >> an
> >> > impressive collection of work.
> >> >
> >> > Regards,
> >> > John
> >> >
> >> >
> >> >
> >> > -------------------------------------------------
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > VFEmail.net - http://www.vfemail.net
> >> >
> >> >
> >> > $14.95 ONETIME Lifetime accounts with Privacy Features!
> >> >
> >> >
> >> > 15GB disk! No bandwidth quotas!
> >> >
> >> >
> >> > Commercial and Bulk Mail Options!
> >> >
> >> >
> >> >
> >>
> >> ------Zitierte Anlage------
> >> >
> >>  
> ------------------------------------------------------------------------------
> >> > Own the Future-Intel&reg; Level Up Game Demo Contest 2013
> >> > Rise to greatness in Intel's independent game demo contest.
> >> > Compete for recognition, cash, and the chance to get your game
> >> > on Steam. $5K grand prize plus 10 genre and skill prizes.
> >> > Submit your demo by 6/6/13. http://p.sf.net/sfu/intel_levelupd2d
> >>
> >> ------Zitierte Anlage------
> >> > _______________________________________________
> >> > Ocamlnet-devel mailing list
> >> > Oca...@li...
> >> > https://lists.sourceforge.net/lists/listinfo/ocamlnet-devel
> >> >
> >>
> >>
> >>
> >> --
> >> ------------------------------------------------------------
> >> Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
> >> Creator of GODI and camlcity.org.
> >> Contact details:        http://www.camlcity.org/contact.html
> >> Company homepage:       http://www.gerd-stolpmann.de
> >> ------------------------------------------------------------
> >>
> >>
> >> -------------------------------------------------
> >>
> >> VFEmail.net - http://www.vfemail.net
> >> $14.95 ONETIME Lifetime accounts with Privacy Features!
> >> 15GB disk! No bandwidth quotas!
> >> Commercial and Bulk Mail Options!
> >>
> >>  
> ------------------------------------------------------------------------------
> >> Own the Future-Intel(R) Level Up Game Demo Contest 2013
> >> Rise to greatness in Intel's independent game demo contest. Compete
> >> for recognition, cash, and the chance to get your game on Steam.
> >> $5K grand prize plus 10 genre and skill prizes. Submit your demo
> >> by 6/6/13. http://altfarm.mediaplex.com/ad/ck/12124-176961-30367-2
> >> _______________________________________________
> >> Ocamlnet-devel mailing list
> >> Oca...@li...
> >> https://lists.sourceforge.net/lists/listinfo/ocamlnet-devel
> >>
> >>
> >
> >
> >
> > --
> > ------------------------------------------------------------
> > Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
> > Creator of GODI and camlcity.org.
> > Contact details:        http://www.camlcity.org/contact.html
> > Company homepage:       http://www.gerd-stolpmann.de
> > ------------------------------------------------------------
> >
> 
> 
> 
> -------------------------------------------------
> 
> VFEmail.net - http://www.vfemail.net
> $14.95 ONETIME Lifetime accounts with Privacy Features!
> 15GB disk! No bandwidth quotas!
> Commercial and Bulk Mail Options!
> 
> ------------------------------------------------------------------------------
> Minimize network downtime and maximize team effectiveness.
> Reduce network management and security costs.Learn how to hire
> the most talented Cisco Certified professionals. Visit the
> Employer Resources Portal
> http://www.cisco.com/web/learning/employer_resources/index.html
> _______________________________________________
> Ocamlnet-devel mailing list
> Oca...@li...
> https://lists.sourceforge.net/lists/listinfo/ocamlnet-devel
> 
> 



-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
Creator of GODI and camlcity.org.
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------

Re: [Ocamlnet-devel] Seeking advice for a couple of issues

[<jde...@vf...>]

The fix for chunked request processing seems to work well.  Quite a  
drastic improvement in processing time. My latest tests are showing  
processing time of slightly greater than 2 seconds to process 10,000  
rows/documents from a CouchDb get operation. Just a thought, is it  
necessary for each chunk to be processed by the event system?  If all  
the chunks were treated as one event (decoupled), I'd bet that the  
processing time would drop significantly.

I'm having some trouble using the skip_challenge option on the Basic  
Auth handler.

The following snippet is my latest attempt:

let () =
   let keys = new Http_client.key_ring() in
   let key = Http_client.key ~user:"admin" ~password:"admin" ~realm:""
			    ~domain:["http://localhost:5984"] in
   keys#add_key key;
   let auth = new Http_client.basic_auth_handler ~enable_auth_in_advance:true
		 ~skip_challenge:(Some "*") keys in
   let pipeline = new Http_client.pipeline in
   pipeline # add_auth_handler auth;
   let call = new Http_client.get "http://localhost:5984/camldb" in
   let hdr = call#request_header `Base in
   hdr#update_field "Content-type" "application/json";
   pipeline#add call;
   pipeline#run();
   printf "rc=%d\n" (call#response_status_code);
   let hdr = call#request_header `Effective in
   List.iter (fun (k, v) -> printf "%s: %s\n" k v) hdr#fields;

This yields a return code 401 with no Authorization header added to  
the sent GET request.  No luck with PUT or DELETE requests either.

Thanks again for all your help.  I've enjoyed working with your library.

Regards,
John

Quoting "Gerd Stolpmann" <in...@ge...>:

> Am 29.03.2013 01:11:39 schrieb(en) John Derrick:
>> Thanks for the reply, Gerd.
>>
>> First, I should mention I'm using Ocaml 4.001, the Mingw build on
>> Win7.  I'm
>> using Ocamlnet version 3.6.1 (or newer).
>>
>> I tried using ~enable_auth_in_advance as you suggested but was unable
>> to get
>> it working.  I must be doing something wrong.
>> Here is my latest attempt:
>
> Confirmed, this does not work. I had this wrong in my mind -
> auth_in_advance does not work when the server never generates a
> challenge.
>
> I've added a new mode skip_challenge, e.g.
>
> let auth = new Http_client.basic_auth_handler ~skip_challenge:(Some
> "*") keys
>
> This is set to "*" or to the domain URL (e.g. "http://localhost/") to
> which it applies.
>
>> method private do_run_pipe call =
>>   let keys = new Http_client.key_ring() in
>>   let auth = new Http_client.basic_auth_handler
>> ~enable_auth_in_advance:true
>> keys in
>>   let key = Http_client.key ~user:"admin" ~password:"admin" ~realm:""
>>     ~domain:["localhost"] in
>
> This is also a problem in the current code: you would have to set
> domain to "http://localhost:80/". Not really well documented. The new
> version also allows now "*", and you can omit the port number.
>
> That the domain is a full URL has to do with the definitions in the
> HTTP protocol.
>
>>   keys#add_key key;
>>   let pipeline = new Http_client.pipeline in
>>   pipeline # add_auth_handler auth;
>>   let hdr = call#request_header `Base in
>>   hdr#update_field "Content-type" "application/json";
>>   pipeline#add call;
>>   pipeline#run();
>>   call#response_status_code
>>
>> Packet capture shows that no Authorization header was added to a PUT
>> request
>> with the above block of code.  Return code is 401.
>>
>> On the second issue, I have TCP_NODELAY set to true in the CouchDb
>> config.
>> The process is using heavy CPU while it runs.  Do you think the issue
>> could
>> be related to OS and/or OCaml build?
>
> Turns out this is in deed a problem with event handling, namely that
> there was some progress counting, and this resulted in a performance
> bug when the chain of event handling gets long.
>
> The new version is not yet released, but you can grab it from the svn
> server: https://godirepo.camlcity.org/svn/lib-ocamlnet2/trunk/
>
> Gerd
>
>
>> Thanks again for your help,
>> John
>>
>> -----Original Message-----
>> From: Gerd Stolpmann
>> Sent: Thursday, March 28, 2013 9:06 AM
>> To: John Derrick
>> Cc: oca...@li...
>> Subject: AW: [Ocamlnet-devel] Seeking advice for a couple of issues
>>
>> Am 27.03.2013 23:41:36 schrieb(en) John Derrick:
>> > Hi All,
>> >
>> > I've been working with Http_client to develop a no-frills CouchDb
>> > driver. I'm relatively new to OCaml, but have many years of
>> experience
>> > with various imperative langs.
>> >
>> > I'm looking for help with a couple of issues. My first issue is
>> Basic
>> > Authentication. I've noticed that Basic Auth and Basic Auth Session
>> > do not include the "Authorization:" header when making GET or PUT
>> > requests UNLESS a challenge is received. Is this the expected
>> > behavior? Not a serious issue as it's easy to add my own
>> > Authorization entry to the header. Apparently CouchDb does not
>> send  an
>> > auth challenge.
>>
>> It is in deed possible to enable this behavior (it's off by default):
>>
>>   - Create the authentication handler like this:
>>
>>     let keys = new Http_client.key_ring()
>>     let auth = new Http_client.basic_auth_handler
>>                   ~enable_auth_in_advance:true keys
>>
>>     The enable_auth_in_advance option makes it working.
>>
>>   - Add credentials:
>>
>>     let key = Http_client.key ...
>>     keys # add_key key
>>
>>   - Add this to the pipeline:
>>
>>     pipeline # add_auth_handler auth
>>
>> Well, this behavior has always been a matter of discussion. Normally,
>> you don't want to send credentials before you know that the server is
>> really the server (remember that you can run http over ssl, and the
>> ssl
>> connection authenticates the server), and this way it is specified in
>> RFC 2617. Historically, though, browsers have always implemented some
>> form of sending the credentials without prior challenge, namely when
>> the web page was visited the second time.
>>
>> > The second issue is chunked request performance. When peforming a
>> > query against CouchDb (say _all_docs for example) via a GET request,
>> > Ocamlnet seems to have issues processing the returned data. CouchDb
>> > returns many small blocks of data in chunked format. Processing a
>> > collection of 1000 docs can take several minutes. I'm not sure if
>> > this relates to the event system performance or socket performance.
>> > Does any one know of a solution to this issue?
>>
>> I don't think it has to do with the event processing, as the OS
>> buffers
>> data up if the client is not fast enough, so for the next read() the
>> client gets many chunks at once. Also, 1000 events (if they are really
>> generated) are not really much, and should not take minutes, but only
>> milliseconds.
>>
>> Does the client consume a lot of CPU? Or is it just waiting?
>>
>> In theory there could be a bad interaction with the TCP buffering,
>> especially when CouchDB disables the Nagle algorithm (TCP_NODELAY),
>> which it probably does. But you report a really bad behavior, so I've
>> actually no clue here.
>>
>> Gerd
>>
>>
>> >
>> > Thanks to all who contributed to the development of Ocamlnet. It's
>> an
>> > impressive collection of work.
>> >
>> > Regards,
>> > John
>> >
>> >
>> >
>> > -------------------------------------------------
>> >
>> >
>> >
>> >
>> >
>> > VFEmail.net - http://www.vfemail.net
>> >
>> >
>> > $14.95 ONETIME Lifetime accounts with Privacy Features!
>> >
>> >
>> > 15GB disk! No bandwidth quotas!
>> >
>> >
>> > Commercial and Bulk Mail Options!
>> >
>> >
>> >
>>
>> ------Zitierte Anlage------
>> >
>> ------------------------------------------------------------------------------
>> > Own the Future-Intel&reg; Level Up Game Demo Contest 2013
>> > Rise to greatness in Intel's independent game demo contest.
>> > Compete for recognition, cash, and the chance to get your game
>> > on Steam. $5K grand prize plus 10 genre and skill prizes.
>> > Submit your demo by 6/6/13. http://p.sf.net/sfu/intel_levelupd2d
>>
>> ------Zitierte Anlage------
>> > _______________________________________________
>> > Ocamlnet-devel mailing list
>> > Oca...@li...
>> > https://lists.sourceforge.net/lists/listinfo/ocamlnet-devel
>> >
>>
>>
>>
>> --
>> ------------------------------------------------------------
>> Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
>> Creator of GODI and camlcity.org.
>> Contact details:        http://www.camlcity.org/contact.html
>> Company homepage:       http://www.gerd-stolpmann.de
>> ------------------------------------------------------------
>>
>>
>> -------------------------------------------------
>>
>> VFEmail.net - http://www.vfemail.net
>> $14.95 ONETIME Lifetime accounts with Privacy Features!
>> 15GB disk! No bandwidth quotas!
>> Commercial and Bulk Mail Options!
>>
>> ------------------------------------------------------------------------------
>> Own the Future-Intel(R) Level Up Game Demo Contest 2013
>> Rise to greatness in Intel's independent game demo contest. Compete
>> for recognition, cash, and the chance to get your game on Steam.
>> $5K grand prize plus 10 genre and skill prizes. Submit your demo
>> by 6/6/13. http://altfarm.mediaplex.com/ad/ck/12124-176961-30367-2
>> _______________________________________________
>> Ocamlnet-devel mailing list
>> Oca...@li...
>> https://lists.sourceforge.net/lists/listinfo/ocamlnet-devel
>>
>>
>
>
>
> --
> ------------------------------------------------------------
> Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
> Creator of GODI and camlcity.org.
> Contact details:        http://www.camlcity.org/contact.html
> Company homepage:       http://www.gerd-stolpmann.de
> ------------------------------------------------------------
>



-------------------------------------------------

VFEmail.net - http://www.vfemail.net
$14.95 ONETIME Lifetime accounts with Privacy Features!  
15GB disk! No bandwidth quotas!
Commercial and Bulk Mail Options!  

Re: [Ocamlnet-devel] Seeking advice for a couple of issues

[Gerd S. <in...@ge...>]

Am 29.03.2013 01:11:39 schrieb(en) John Derrick:
> Thanks for the reply, Gerd.
> 
> First, I should mention I'm using Ocaml 4.001, the Mingw build on  
> Win7.  I'm
> using Ocamlnet version 3.6.1 (or newer).
> 
> I tried using ~enable_auth_in_advance as you suggested but was unable  
> to get
> it working.  I must be doing something wrong.
> Here is my latest attempt:

Confirmed, this does not work. I had this wrong in my mind -  
auth_in_advance does not work when the server never generates a  
challenge.

I've added a new mode skip_challenge, e.g.

let auth = new Http_client.basic_auth_handler ~skip_challenge:(Some  
"*") keys

This is set to "*" or to the domain URL (e.g. "http://localhost/") to  
which it applies.

> method private do_run_pipe call =
>   let keys = new Http_client.key_ring() in
>   let auth = new Http_client.basic_auth_handler  
> ~enable_auth_in_advance:true
> keys in
>   let key = Http_client.key ~user:"admin" ~password:"admin" ~realm:""
>     ~domain:["localhost"] in

This is also a problem in the current code: you would have to set  
domain to "http://localhost:80/". Not really well documented. The new  
version also allows now "*", and you can omit the port number.

That the domain is a full URL has to do with the definitions in the  
HTTP protocol.

>   keys#add_key key;
>   let pipeline = new Http_client.pipeline in
>   pipeline # add_auth_handler auth;
>   let hdr = call#request_header `Base in
>   hdr#update_field "Content-type" "application/json";
>   pipeline#add call;
>   pipeline#run();
>   call#response_status_code
> 
> Packet capture shows that no Authorization header was added to a PUT  
> request
> with the above block of code.  Return code is 401.
> 
> On the second issue, I have TCP_NODELAY set to true in the CouchDb  
> config.
> The process is using heavy CPU while it runs.  Do you think the issue  
> could
> be related to OS and/or OCaml build?

Turns out this is in deed a problem with event handling, namely that  
there was some progress counting, and this resulted in a performance  
bug when the chain of event handling gets long.

The new version is not yet released, but you can grab it from the svn  
server: https://godirepo.camlcity.org/svn/lib-ocamlnet2/trunk/

Gerd


> Thanks again for your help,
> John
> 
> -----Original Message-----
> From: Gerd Stolpmann
> Sent: Thursday, March 28, 2013 9:06 AM
> To: John Derrick
> Cc: oca...@li...
> Subject: AW: [Ocamlnet-devel] Seeking advice for a couple of issues
> 
> Am 27.03.2013 23:41:36 schrieb(en) John Derrick:
> > Hi All,
> >
> > I've been working with Http_client to develop a no-frills CouchDb
> > driver. I'm relatively new to OCaml, but have many years of  
> experience
> > with various imperative langs.
> >
> > I'm looking for help with a couple of issues. My first issue is  
> Basic
> > Authentication. I've noticed that Basic Auth and Basic Auth Session
> > do not include the "Authorization:" header when making GET or PUT
> > requests UNLESS a challenge is received. Is this the expected
> > behavior? Not a serious issue as it's easy to add my own
> > Authorization entry to the header. Apparently CouchDb does not  
> send  an
> > auth challenge.
> 
> It is in deed possible to enable this behavior (it's off by default):
> 
>   - Create the authentication handler like this:
> 
>     let keys = new Http_client.key_ring()
>     let auth = new Http_client.basic_auth_handler
>                   ~enable_auth_in_advance:true keys
> 
>     The enable_auth_in_advance option makes it working.
> 
>   - Add credentials:
> 
>     let key = Http_client.key ...
>     keys # add_key key
> 
>   - Add this to the pipeline:
> 
>     pipeline # add_auth_handler auth
> 
> Well, this behavior has always been a matter of discussion. Normally,
> you don't want to send credentials before you know that the server is
> really the server (remember that you can run http over ssl, and the  
> ssl
> connection authenticates the server), and this way it is specified in
> RFC 2617. Historically, though, browsers have always implemented some
> form of sending the credentials without prior challenge, namely when
> the web page was visited the second time.
> 
> > The second issue is chunked request performance. When peforming a
> > query against CouchDb (say _all_docs for example) via a GET request,
> > Ocamlnet seems to have issues processing the returned data. CouchDb
> > returns many small blocks of data in chunked format. Processing a
> > collection of 1000 docs can take several minutes. I'm not sure if
> > this relates to the event system performance or socket performance.
> > Does any one know of a solution to this issue?
> 
> I don't think it has to do with the event processing, as the OS  
> buffers
> data up if the client is not fast enough, so for the next read() the
> client gets many chunks at once. Also, 1000 events (if they are really
> generated) are not really much, and should not take minutes, but only
> milliseconds.
> 
> Does the client consume a lot of CPU? Or is it just waiting?
> 
> In theory there could be a bad interaction with the TCP buffering,
> especially when CouchDB disables the Nagle algorithm (TCP_NODELAY),
> which it probably does. But you report a really bad behavior, so I've
> actually no clue here.
> 
> Gerd
> 
> 
> >
> > Thanks to all who contributed to the development of Ocamlnet. It's  
> an
> > impressive collection of work.
> >
> > Regards,
> > John
> >
> >
> >
> > -------------------------------------------------
> >
> >
> >
> >
> >
> > VFEmail.net - http://www.vfemail.net
> >
> >
> > $14.95 ONETIME Lifetime accounts with Privacy Features!
> >
> >
> > 15GB disk! No bandwidth quotas!
> >
> >
> > Commercial and Bulk Mail Options!
> >
> >
> >
> 
> ------Zitierte Anlage------
> >  
> ------------------------------------------------------------------------------
> > Own the Future-Intel® Level Up Game Demo Contest 2013
> > Rise to greatness in Intel's independent game demo contest.
> > Compete for recognition, cash, and the chance to get your game
> > on Steam. $5K grand prize plus 10 genre and skill prizes.
> > Submit your demo by 6/6/13. http://p.sf.net/sfu/intel_levelupd2d
> 
> ------Zitierte Anlage------
> > _______________________________________________
> > Ocamlnet-devel mailing list
> > Oca...@li...
> > https://lists.sourceforge.net/lists/listinfo/ocamlnet-devel
> >
> 
> 
> 
> --
> ------------------------------------------------------------
> Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
> Creator of GODI and camlcity.org.
> Contact details:        http://www.camlcity.org/contact.html
> Company homepage:       http://www.gerd-stolpmann.de
> ------------------------------------------------------------
> 
> 
> -------------------------------------------------
> 
> VFEmail.net - http://www.vfemail.net
> $14.95 ONETIME Lifetime accounts with Privacy Features!
> 15GB disk! No bandwidth quotas!
> Commercial and Bulk Mail Options!
> 
> ------------------------------------------------------------------------------
> Own the Future-Intel(R) Level Up Game Demo Contest 2013
> Rise to greatness in Intel's independent game demo contest. Compete
> for recognition, cash, and the chance to get your game on Steam.
> $5K grand prize plus 10 genre and skill prizes. Submit your demo
> by 6/6/13. http://altfarm.mediaplex.com/ad/ck/12124-176961-30367-2
> _______________________________________________
> Ocamlnet-devel mailing list
> Oca...@li...
> https://lists.sourceforge.net/lists/listinfo/ocamlnet-devel
> 
> 



-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
Creator of GODI and camlcity.org.
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------

Re: [Ocamlnet-devel] Seeking advice for a couple of issues

[John D. <jde...@vf...>]

Thanks for the reply, Gerd.

First, I should mention I'm using Ocaml 4.001, the Mingw build on Win7.  I'm 
using Ocamlnet version 3.6.1 (or newer).

I tried using ~enable_auth_in_advance as you suggested but was unable to get 
it working.  I must be doing something wrong.
Here is my latest attempt:

method private do_run_pipe call =
  let keys = new Http_client.key_ring() in
  let auth = new Http_client.basic_auth_handler ~enable_auth_in_advance:true 
keys in
  let key = Http_client.key ~user:"admin" ~password:"admin" ~realm:""
    ~domain:["localhost"] in
  keys#add_key key;
  let pipeline = new Http_client.pipeline in
  pipeline # add_auth_handler auth;
  let hdr = call#request_header `Base in
  hdr#update_field "Content-type" "application/json";
  pipeline#add call;
  pipeline#run();
  call#response_status_code

Packet capture shows that no Authorization header was added to a PUT request 
with the above block of code.  Return code is 401.

On the second issue, I have TCP_NODELAY set to true in the CouchDb config. 
The process is using heavy CPU while it runs.  Do you think the issue could 
be related to OS and/or OCaml build?

Thanks again for your help,
John

-----Original Message----- 
From: Gerd Stolpmann
Sent: Thursday, March 28, 2013 9:06 AM
To: John Derrick
Cc: oca...@li...
Subject: AW: [Ocamlnet-devel] Seeking advice for a couple of issues

Am 27.03.2013 23:41:36 schrieb(en) John Derrick:
> Hi All,
>
> I've been working with Http_client to develop a no-frills CouchDb
> driver. I'm relatively new to OCaml, but have many years of experience
> with various imperative langs.
>
> I'm looking for help with a couple of issues. My first issue is Basic
> Authentication. I've noticed that Basic Auth and Basic Auth Session
> do not include the "Authorization:" header when making GET or PUT
> requests UNLESS a challenge is received. Is this the expected
> behavior? Not a serious issue as it's easy to add my own
> Authorization entry to the header. Apparently CouchDb does not send  an 
> auth challenge.

It is in deed possible to enable this behavior (it's off by default):

  - Create the authentication handler like this:

    let keys = new Http_client.key_ring()
    let auth = new Http_client.basic_auth_handler
                  ~enable_auth_in_advance:true keys

    The enable_auth_in_advance option makes it working.

  - Add credentials:

    let key = Http_client.key ...
    keys # add_key key

  - Add this to the pipeline:

    pipeline # add_auth_handler auth

Well, this behavior has always been a matter of discussion. Normally,
you don't want to send credentials before you know that the server is
really the server (remember that you can run http over ssl, and the ssl
connection authenticates the server), and this way it is specified in
RFC 2617. Historically, though, browsers have always implemented some
form of sending the credentials without prior challenge, namely when
the web page was visited the second time.

> The second issue is chunked request performance. When peforming a
> query against CouchDb (say _all_docs for example) via a GET request,
> Ocamlnet seems to have issues processing the returned data. CouchDb
> returns many small blocks of data in chunked format. Processing a
> collection of 1000 docs can take several minutes. I'm not sure if
> this relates to the event system performance or socket performance.
> Does any one know of a solution to this issue?

I don't think it has to do with the event processing, as the OS buffers
data up if the client is not fast enough, so for the next read() the
client gets many chunks at once. Also, 1000 events (if they are really
generated) are not really much, and should not take minutes, but only
milliseconds.

Does the client consume a lot of CPU? Or is it just waiting?

In theory there could be a bad interaction with the TCP buffering,
especially when CouchDB disables the Nagle algorithm (TCP_NODELAY),
which it probably does. But you report a really bad behavior, so I've
actually no clue here.

Gerd


>
> Thanks to all who contributed to the development of Ocamlnet. It's an
> impressive collection of work.
>
> Regards,
> John
>
>
>
> -------------------------------------------------
>
>
>
>
>
> VFEmail.net - http://www.vfemail.net
>
>
> $14.95 ONETIME Lifetime accounts with Privacy Features!
>
>
> 15GB disk! No bandwidth quotas!
>
>
> Commercial and Bulk Mail Options!
>
>
>

------Zitierte Anlage------
> ------------------------------------------------------------------------------
> Own the Future-Intel® Level Up Game Demo Contest 2013
> Rise to greatness in Intel's independent game demo contest.
> Compete for recognition, cash, and the chance to get your game
> on Steam. $5K grand prize plus 10 genre and skill prizes.
> Submit your demo by 6/6/13. http://p.sf.net/sfu/intel_levelupd2d

------Zitierte Anlage------
> _______________________________________________
> Ocamlnet-devel mailing list
> Oca...@li...
> https://lists.sourceforge.net/lists/listinfo/ocamlnet-devel
>



-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
Creator of GODI and camlcity.org.
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------ 


-------------------------------------------------

VFEmail.net - http://www.vfemail.net
$14.95 ONETIME Lifetime accounts with Privacy Features!  
15GB disk! No bandwidth quotas!
Commercial and Bulk Mail Options!  

Re: [Ocamlnet-devel] Seeking advice for a couple of issues

[Gerd S. <in...@ge...>]

Am 27.03.2013 23:41:36 schrieb(en) John Derrick:
> Hi All,
> 
> I've been working with Http_client to develop a no-frills CouchDb
> driver. I'm relatively new to OCaml, but have many years of experience
> with various imperative langs.
> 
> I'm looking for help with a couple of issues. My first issue is Basic
> Authentication. I've noticed that Basic Auth and Basic Auth Session
> do not include the "Authorization:" header when making GET or PUT
> requests UNLESS a challenge is received. Is this the expected
> behavior? Not a serious issue as it's easy to add my own
> Authorization entry to the header. Apparently CouchDb does not send  
> an auth challenge.

It is in deed possible to enable this behavior (it's off by default):

  - Create the authentication handler like this:

    let keys = new Http_client.key_ring()
    let auth = new Http_client.basic_auth_handler
                  ~enable_auth_in_advance:true keys

    The enable_auth_in_advance option makes it working.

  - Add credentials:

    let key = Http_client.key ...
    keys # add_key key

  - Add this to the pipeline:

    pipeline # add_auth_handler auth

Well, this behavior has always been a matter of discussion. Normally,  
you don't want to send credentials before you know that the server is  
really the server (remember that you can run http over ssl, and the ssl  
connection authenticates the server), and this way it is specified in  
RFC 2617. Historically, though, browsers have always implemented some  
form of sending the credentials without prior challenge, namely when  
the web page was visited the second time.

> The second issue is chunked request performance. When peforming a
> query against CouchDb (say _all_docs for example) via a GET request,
> Ocamlnet seems to have issues processing the returned data. CouchDb
> returns many small blocks of data in chunked format. Processing a
> collection of 1000 docs can take several minutes. I'm not sure if
> this relates to the event system performance or socket performance.
> Does any one know of a solution to this issue?

I don't think it has to do with the event processing, as the OS buffers  
data up if the client is not fast enough, so for the next read() the  
client gets many chunks at once. Also, 1000 events (if they are really  
generated) are not really much, and should not take minutes, but only  
milliseconds.

Does the client consume a lot of CPU? Or is it just waiting?

In theory there could be a bad interaction with the TCP buffering,  
especially when CouchDB disables the Nagle algorithm (TCP_NODELAY),  
which it probably does. But you report a really bad behavior, so I've  
actually no clue here.

Gerd


> 
> Thanks to all who contributed to the development of Ocamlnet. It's an
> impressive collection of work.
> 
> Regards,
> John
> 
> 
> 
> -------------------------------------------------
> 
> 
> 
> 
> 
> VFEmail.net - http://www.vfemail.net
> 
> 
> $14.95 ONETIME Lifetime accounts with Privacy Features!
> 
> 
> 15GB disk! No bandwidth quotas!
> 
> 
> Commercial and Bulk Mail Options!
> 
> 
> 

------Zitierte Anlage------
> ------------------------------------------------------------------------------
> Own the Future-Intel® Level Up Game Demo Contest 2013
> Rise to greatness in Intel's independent game demo contest.
> Compete for recognition, cash, and the chance to get your game
> on Steam. $5K grand prize plus 10 genre and skill prizes.
> Submit your demo by 6/6/13. http://p.sf.net/sfu/intel_levelupd2d

------Zitierte Anlage------
> _______________________________________________
> Ocamlnet-devel mailing list
> Oca...@li...
> https://lists.sourceforge.net/lists/listinfo/ocamlnet-devel
> 



-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
Creator of GODI and camlcity.org.
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------

[Ocamlnet-devel] Seeking advice for a couple of issues

[John D. <jde...@vf...>]

Hi All,

I've been working with Http_client to develop a no-frills CouchDb 
driver. I'm relatively new to OCaml, but have many years of experience 
with various imperative langs.

I'm looking for help with a couple of issues. My first issue is Basic 
Authentication. I've noticed that Basic Auth and Basic Auth Session 
do not include the "Authorization:" header when making GET or PUT 
requests UNLESS a challenge is received. Is this the expected 
behavior? Not a serious issue as it's easy to add my own 
Authorization entry to the header. Apparently CouchDb does not send an auth challenge.

The second issue is chunked request performance. When peforming a 
query against CouchDb (say _all_docs for example) via a GET request, 
Ocamlnet seems to have issues processing the returned data. CouchDb 
returns many small blocks of data in chunked format. Processing a 
collection of 1000 docs can take several minutes. I'm not sure if 
this relates to the event system performance or socket performance. 
Does any one know of a solution to this issue?

Thanks to all who contributed to the development of Ocamlnet. It's an 
impressive collection of work.

Regards,
John



-------------------------------------------------





VFEmail.net - http://www.vfemail.net


$14.95 ONETIME Lifetime accounts with Privacy Features!  


15GB disk! No bandwidth quotas!


Commercial and Bulk Mail Options!  

Re: [Ocamlnet-devel] NetHTTPd/Netplex subtasking

[Gerd S. <in...@ge...>]

Am Donnerstag, den 10.01.2013, 15:17 +0100 schrieb Pierre-Alexandre
Voye:
> 
> 
> I followed one of the simpler example (don't remember which one), in
> fact, my app starts with this code :
> 
> let start handlers =
>   let (option_list, cmdline_infos) = Netplex_main.args() in
> 
> (*  .... cut ... *)
>  let nethttpd_factory =  Nethttpd_plex.nethttpd_factory
> ~handlers:handlers () in
>   Netplex_main.startup  ( Netplex_mp.mp ~keep_fd_open:true () )
>     Netplex_log.logger_factories  
>     Netplex_workload.workload_manager_factories 
>     [ nethttpd_factory ]          
>    cmdline_infos
> 
> 
> Normally, only the threads will use the object.

That's fine then.

> Because of my inability to user shared memory, I don't have any byte
> of global memory in this software (I use Memcache for that)
> 
> 
> So, if i start a thread and return the string result, is there a risk
> that the processes is killed by the master process before the thread
> finish his job (In theory not) ?

In theory and practice, this risk exists in deed. But you can do
something about it. The process is not killed with a signal, but there
is an orderly shutdown procedure going through several steps until
terminating the process, and it is possible to run some user code for
process cleanup. Nethttpd_plex.nethttpd_factory takes a ~hooks argument:

let hooks =
  object
    inherit Netplex_kit.empty_processor_hooks()

    method shutdown() = wait_for_the_termination_of_the_threads()
  end

I think a simple counter for the number of threads would be sufficient
(remember that "incr counter" and "decr counter" are atomic), and you
could just busy-wait (while !counter > 0 do sleep 1 done).

Note that there is also a second hook, pre_finish_hook, which is even
called later, but this hook is time-bound. If the maximum time is
exceeded, the process is finally killed (the timeout is set by
~terminate_tmo of Netplex_mp.mp).

Gerd

> 
> 
> 2013/1/10 Gerd Stolpmann <in...@ge...>
>         Am Donnerstag, den 10.01.2013, 13:19 +0100 schrieb
>         Pierre-Alexandre
>         Voye:
>         
>         >
>         > Can I use a thread without risk ?
>         
>         
>         Normally, yes, if you ensure that library functions do not
>         "see" that
>         there are several threads, i.e. any object is only used by one
>         thread at
>         a time. There are normally no mutexes controlling parallel
>         access
>         built-in (except where described). There are also no global
>         variables
>         making multi-threading impossible (and if so, there are
>         mutexes around
>         them).
>         
>         I must admit that I do not exactly understand your
>         architectural
>         problem. What I've got is that
>         
>          - You get a request via Nethttpd
>          - You call your service routine via the CGI-style interface
>          - that routine computes the result to return via CGI
>          - but that routine also needs to do some side work that is
>            not needed for the CGI return value, and could be run
>            outside the strict HTTP request/response cycle
>         
>         The answer to the question how you can run such side work
>         depends very
>         much on how you deploy Nethttpd. Do you use the Netplex
>         containers, or
>         did you follow one of the simpler examples?
>         
>         Gerd
>         
>         
>         
>         
>         --
>         ------------------------------------------------------------
>         Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
>         Creator of GODI and camlcity.org.
>         Contact details:        http://www.camlcity.org/contact.html
>         Company homepage:       http://www.gerd-stolpmann.de
>         *** Searching for new projects! Need consulting for system
>         *** programming in Ocaml? Gerd Stolpmann can help you.
>         ------------------------------------------------------------
>         
> 
> 
> 
> -- 
> ---------------------
> https://twitter.com/#!/ontologiae/
> http://linuxfr.org/users/montaigne
> 
> ------------------------------------------------------------------------------
> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
> MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
> with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
> MVPs and experts. ON SALE this month only -- learn more at:
> http://p.sf.net/sfu/learnmore_122712
> _______________________________________________ Ocamlnet-devel mailing list Oca...@li... https://lists.sourceforge.net/lists/listinfo/ocamlnet-devel

-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
Creator of GODI and camlcity.org.
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
*** Searching for new projects! Need consulting for system
*** programming in Ocaml? Gerd Stolpmann can help you.
------------------------------------------------------------

Re: [Ocamlnet-devel] NetHTTPd/Netplex subtasking

[Pierre-Alexandre V. <ont...@gm...>]

I followed one of the simpler example (don't remember which one), in fact,
my app starts with this code :

let start handlers =
  let (option_list, cmdline_infos) = Netplex_main.args() in
(*  .... cut ... *)
 let nethttpd_factory =  Nethttpd_plex.nethttpd_factory  ~handlers:handlers
() in
  Netplex_main.startup  ( Netplex_mp.mp ~keep_fd_open:true () )
    Netplex_log.logger_factories
    Netplex_workload.workload_manager_factories
    [ nethttpd_factory ]
   cmdline_infos

Normally, only the threads will use the object.
Because of my inability to user shared memory, I don't have any byte of
global memory in this software (I use Memcache for that)

So, if i start a thread and return the string result, is there a risk that
the processes is killed by the master process before the thread finish his
job (In theory not) ?

2013/1/10 Gerd Stolpmann <in...@ge...>

> Am Donnerstag, den 10.01.2013, 13:19 +0100 schrieb Pierre-Alexandre
> Voye:
>
> >
> > Can I use a thread without risk ?
>
> Normally, yes, if you ensure that library functions do not "see" that
> there are several threads, i.e. any object is only used by one thread at
> a time. There are normally no mutexes controlling parallel access
> built-in (except where described). There are also no global variables
> making multi-threading impossible (and if so, there are mutexes around
> them).
>
> I must admit that I do not exactly understand your architectural
> problem. What I've got is that
>
>  - You get a request via Nethttpd
>  - You call your service routine via the CGI-style interface
>  - that routine computes the result to return via CGI
>  - but that routine also needs to do some side work that is
>    not needed for the CGI return value, and could be run
>    outside the strict HTTP request/response cycle
>
> The answer to the question how you can run such side work depends very
> much on how you deploy Nethttpd. Do you use the Netplex containers, or
> did you follow one of the simpler examples?
>
> Gerd
>
>
>
> --
> ------------------------------------------------------------
> Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
> Creator of GODI and camlcity.org.
> Contact details:        http://www.camlcity.org/contact.html
> Company homepage:       http://www.gerd-stolpmann.de
> *** Searching for new projects! Need consulting for system
> *** programming in Ocaml? Gerd Stolpmann can help you.
> ------------------------------------------------------------
>
>


-- 
---------------------
https://twitter.com/#!/ontologiae/
http://linuxfr.org/users/montaigne

Re: [Ocamlnet-devel] NetHTTPd/Netplex subtasking

[Gerd S. <in...@ge...>]

Am Donnerstag, den 10.01.2013, 13:19 +0100 schrieb Pierre-Alexandre
Voye:
> Hello,
> 
> I'm facing more and more a problem with my application which is based
> on NetHTTPd/NetCGI.
> 
> 
> My software is based on CGI entry point (url and arguments values).
> Each CGI opens a memcache + postgreSQL connexion, makes his task,
> closes memcache + postgreSQL connexion and return the result (a string
> of course, most of the time JSON data).
> 
> 
> I use Memcache for several reasons, partly because of my inability to
> understand how to use shared memory despite the tutorial. 
> 
> 
> 
> Some CGI are becoming to long to execute (more than 30s).
> Usually, the result I need to return is calculated quickly (less than
> 15s), but I need to make lot of treatments before returning it.
> 
> Thus, I'm looking for a way to return the result my client need, and
> continuing all tasks I need. So, I need to launch an asynchronous
> task.
> 
> 
> The ugly way I only see is to make my app calling itself by Http, give
> to the CGI, all the information it need to continue the task.
> 
> But it is so ugly, I can't make it this way.
> 
> 
> Is there a way to lauch a task asynchronously ?
> 
> Can I use a thread without risk ?

Normally, yes, if you ensure that library functions do not "see" that
there are several threads, i.e. any object is only used by one thread at
a time. There are normally no mutexes controlling parallel access
built-in (except where described). There are also no global variables
making multi-threading impossible (and if so, there are mutexes around
them).

I must admit that I do not exactly understand your architectural
problem. What I've got is that

 - You get a request via Nethttpd
 - You call your service routine via the CGI-style interface
 - that routine computes the result to return via CGI
 - but that routine also needs to do some side work that is
   not needed for the CGI return value, and could be run
   outside the strict HTTP request/response cycle

The answer to the question how you can run such side work depends very
much on how you deploy Nethttpd. Do you use the Netplex containers, or
did you follow one of the simpler examples?

Gerd


> 
> Regards,
> 
> 
> P-A
> 
> 
> -- 
> ---------------------
> https://twitter.com/#!/ontologiae/
> http://linuxfr.org/users/montaigne
> 
> ------------------------------------------------------------------------------
> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
> MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
> with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
> MVPs and experts. ON SALE this month only -- learn more at:
> http://p.sf.net/sfu/learnmore_122712
> _______________________________________________ Ocamlnet-devel mailing list Oca...@li... https://lists.sourceforge.net/lists/listinfo/ocamlnet-devel

-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
Creator of GODI and camlcity.org.
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
*** Searching for new projects! Need consulting for system
*** programming in Ocaml? Gerd Stolpmann can help you.
------------------------------------------------------------

[Ocamlnet-devel] NetHTTPd/Netplex subtasking

[Pierre-Alexandre V. <ont...@gm...>]

Hello,
I'm facing more and more a problem with my application which is based on
NetHTTPd/NetCGI.

My software is based on CGI entry point (url and arguments values). Each
CGI opens a memcache + postgreSQL connexion, makes his task, closes
memcache + postgreSQL connexion and return the result (a string of course,
most of the time JSON data).

I use Memcache for several reasons, partly because of my inability to
understand how to use shared memory despite the tutorial.

Some CGI are becoming to long to execute (more than 30s).
Usually, the result I need to return is calculated quickly (less than 15s),
but I need to make lot of treatments before returning it.

Thus, I'm looking for a way to return the result my client need, and
continuing all tasks I need. So, I need to launch an asynchronous task.

The ugly way I only see is to make my app calling itself by Http, give to
the CGI, all the information it need to continue the task.
But it is so ugly, I can't make it this way.

Is there a way to lauch a task asynchronously ?
Can I use a thread without risk ?

Regards,

P-A

-- 
---------------------
https://twitter.com/#!/ontologiae/
http://linuxfr.org/users/montaigne

Re: [Ocamlnet-devel] IMAP client

[Gerd S. <in...@ge...>]

Am 07.12.2012 00:13:59 schrieb(en) Julien Tesson:
> Hi !
> 
> I would be interested in an IMAP client implementation and I was  
> wondering if it was planned to be included in Ocamlnet or if someone  
> had already worked on this ?

No, at least I do not work on this.

> I may implement this in a near future if it has not been done, any  
> advice so that it could be easily integrated in Ocamlnet ?

Depends on what you are planning to do, e.g. whether it is sync or  
async. For a sync client a starting point could be the simple pop  
client in src/pop. For an async client look at the FTP client (but be  
warned, asnyc code is really an order of magnitude more difficult to  
develop).

Generally, you should try to reuse the existing concepts when possible.

Ocamlnet allows it to override DNS resolution globally. Please use the  
functions in Uq_resolver, and this is done right.

I'm planning to develop some SASL functionality, which is probably very  
useful for IMAP. The existing functions are not very accessible  
(actually, there is code for DIGEST-MD5 hidden in the http client, and  
there is SCRAM-SHA1 code, but with an interface suited for GSSAPI and  
not SASL). Also, some bindings to Cyrus would be good. I don't know  
when I've time to do this

> 
> Regards,
> Julien.
> 
> ------------------------------------------------------------------------------
> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> Remotely access PCs and mobile devices and provide instant support
> Improve your efficiency, and focus on delivering more value-add  
> services
> Discover what IT Professionals Know. Rescue delivers
> http://p.sf.net/sfu/logmein_12329d2d
> _______________________________________________
> Ocamlnet-devel mailing list
> Oca...@li...
> https://lists.sourceforge.net/lists/listinfo/ocamlnet-devel
> 
> 

-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    ge...@ge...
Creator of GODI and camlcity.org.
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------

[Ocamlnet-devel] IMAP client

[Julien T. <jul...@la...>]

Hi !

I would be interested in an IMAP client implementation and I was wondering if it was planned to be included in Ocamlnet or if someone had already worked on this ?
I may implement this in a near future if it has not been done, any advice so that it could be easily integrated in Ocamlnet ?

Regards,
Julien.