cppcms-users

[Cppcms-users] dispatcher and async connections

[Frank E. <fra...@an...>]

hi,

something else came up. when i use the dispatcher in an async class and
a url is requested which is not handled the default main() method in the
application class does not handle async connections properly. there's no
finalize/async_complete_response and thus giving a 502 bad gateway
response instead of 404.

i guess the default main method should handle async connections in
addition to sync connections.

frank.
-- 
Dipl.-Ing. (FH) Frank Enderle

anamica UG (haftungsbeschränkt)
Beinsteinerstr. 6
71334 Waiblingen

Telefon:  +49 151 14981091
Telefax:  +49 7151 1335770
E-Mail:   fra...@an...
Internet: www.anamica.de

Handelsregister: AG Stuttgart HRB 732357
Geschäftsführer: Yvonne Holzwarth, Frank Enderle

[Cppcms-users] async connections and response headers

[Frank E. <fra...@an...>]

hi,

i have an async application:

class Test : public cppcms::application {
public:
    Test(cppcms::service &srv)
        : cppcms::application(srv) {
        dispatcher().assign("^/test", &Test::test, this);
    }

    void main(std::string url) {
        if(!dispatcher().dispatch(url)) {
            response().status(404);
            response().finalize();
            release_context()->async_complete_response();
        }
    }

    void test() {
        response().set_content_header("text/plain");
        response().out() << "test";
        response().finalize();
        release_context()->async_complete_response();
    }
};

int main(int argc, char *argv[]) {
    try {
        cppcms::service srv(argc, argv);
        booster::intrusive_ptr<Test> app = new Test(srv);
        srv.applications_pool().mount(app, cppcms::mount_point("test"));
        srv.run();
    } catch(std::exception const &e) {
        std::cerr << e.what() << std::endl;
    }
}

if i start the application using scgi behind nginx and i call
http://.../test/test i get the expected 'test' string in the browser,
but the application sends nothing else (i.e. no response headers) and
the nginx log emits the following:

upstream sent neither valid HTTP/1.0 header nor "Status" header line
while reading response header from upstream

whats wrong here?

thanks,

frank

Re: [Cppcms-users] async connections and response headers

[Artyom <art...@ya...>]

Hello,

There had been some restructuring in the internal handling of requests and 
responses
and looks like something is messed up. Looks like there are more then few bugs 
need to be fixed
connected to all this.

Hopefully tomorrow I'll be able to commit fixes and cleanup the mess,

Meanwhile I'll recommend to stick with released beta version, which comes before
numerous updates that messed many things up or wait a little.

The code below and the previous code you send should work with released beta 
version
and they fail in svn. I'll update you as soon as I fix this issues in the svn.

Thanks for points you had bring up.

  Artyom




----- Original Message ----
> From: Frank Enderle <fra...@an...>
> To: cpp...@li...
> Sent: Tue, July 13, 2010 9:20:10 PM
> Subject: [Cppcms-users] async connections and response headers
> 
> hi,
> 
> i have an async application:
> 
> class Test : public  cppcms::application {
> public:
>     Test(cppcms::service  &srv)
>         : cppcms::application(srv) {
>          dispatcher().assign("^/test", &Test::test,  this);
>     }
> 
>     void main(std::string url)  {
>         if(!dispatcher().dispatch(url)) {
>              response().status(404);
>              response().finalize();
>              release_context()->async_complete_response();
>          }
>     }
> 
>     void test()  {
>          response().set_content_header("text/plain");
>          response().out() << "test";
>          response().finalize();
>          release_context()->async_complete_response();
>      }
> };
> 
> int main(int argc, char *argv[]) {
>     try  {
>         cppcms::service srv(argc, argv);
>          booster::intrusive_ptr<Test> app = new  Test(srv);
>         srv.applications_pool().mount(app,  cppcms::mount_point("test"));
>          srv.run();
>     } catch(std::exception const &e) {
>          std::cerr << e.what() << std::endl;
>      }
> }
> 
> if i start the application using scgi behind nginx and i  call
> http://.../test/test i get the expected 'test' string in the  browser,
> but the application sends nothing else (i.e. no response headers)  and
> the nginx log emits the following:
> 
> upstream sent neither valid  HTTP/1.0 header nor "Status" header line
> while reading response header from  upstream
> 
> whats wrong  here?
> 
> thanks,
> 
> frank
> 
> 
> 
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by  Sprint
> What will you do first with EVO, the first 4G phone?
> Visit sprint.com/first --  http://p.sf.net/sfu/sprint-com-first
> _______________________________________________
> Cppcms-users  mailing list
> Cpp...@li...
> https://lists.sourceforge.net/lists/listinfo/cppcms-users
> 


      

Re: [Cppcms-users] async connections and response headers

[Frank E. <fra...@an...>]

hey,

i switched to the beta. however the problem with the missing response
headers persist. so please notify me when you commit changes that should
address this. meanwhile i try to live without response headers.

cheers

frank

Am 13.07.2010 21:02, schrieb Artyom:
> Hello,
> 
> There had been some restructuring in the internal handling of requests and 
> responses
> and looks like something is messed up. Looks like there are more then few bugs 
> need to be fixed
> connected to all this.
> 
> Hopefully tomorrow I'll be able to commit fixes and cleanup the mess,
> 
> Meanwhile I'll recommend to stick with released beta version, which comes before
> numerous updates that messed many things up or wait a little.
> 
> The code below and the previous code you send should work with released beta 
> version
> and they fail in svn. I'll update you as soon as I fix this issues in the svn.
> 
> Thanks for points you had bring up.
> 
>   Artyom
> 
> 
> 
> 
> ----- Original Message ----
>> From: Frank Enderle <fra...@an...>
>> To: cpp...@li...
>> Sent: Tue, July 13, 2010 9:20:10 PM
>> Subject: [Cppcms-users] async connections and response headers
>>
>> hi,
>>
>> i have an async application:
>>
>> class Test : public  cppcms::application {
>> public:
>>     Test(cppcms::service  &srv)
>>         : cppcms::application(srv) {
>>          dispatcher().assign("^/test", &Test::test,  this);
>>     }
>>
>>     void main(std::string url)  {
>>         if(!dispatcher().dispatch(url)) {
>>              response().status(404);
>>              response().finalize();
>>              release_context()->async_complete_response();
>>          }
>>     }
>>
>>     void test()  {
>>          response().set_content_header("text/plain");
>>          response().out() << "test";
>>          response().finalize();
>>          release_context()->async_complete_response();
>>      }
>> };
>>
>> int main(int argc, char *argv[]) {
>>     try  {
>>         cppcms::service srv(argc, argv);
>>          booster::intrusive_ptr<Test> app = new  Test(srv);
>>         srv.applications_pool().mount(app,  cppcms::mount_point("test"));
>>          srv.run();
>>     } catch(std::exception const &e) {
>>          std::cerr << e.what() << std::endl;
>>      }
>> }
>>
>> if i start the application using scgi behind nginx and i  call
>> http://.../test/test i get the expected 'test' string in the  browser,
>> but the application sends nothing else (i.e. no response headers)  and
>> the nginx log emits the following:
>>
>> upstream sent neither valid  HTTP/1.0 header nor "Status" header line
>> while reading response header from  upstream
>>
>> whats wrong  here?
>>
>> thanks,
>>
>> frank
>>
>>
>>
>> ------------------------------------------------------------------------------
>> This SF.net email is sponsored by  Sprint
>> What will you do first with EVO, the first 4G phone?
>> Visit sprint.com/first --  http://p.sf.net/sfu/sprint-com-first
>> _______________________________________________
>> Cppcms-users  mailing list
>> Cpp...@li...
>> https://lists.sourceforge.net/lists/listinfo/cppcms-users
>>
> 
> 
>       
> 
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Sprint
> What will you do first with EVO, the first 4G phone?
> Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
> _______________________________________________
> Cppcms-users mailing list
> Cpp...@li...
> https://lists.sourceforge.net/lists/listinfo/cppcms-users


-- 
Dipl.-Ing. (FH) Frank Enderle

anamica UG (haftungsbeschränkt)
Beinsteinerstr. 6
71334 Waiblingen

Telefon:  +49 151 14981091
Telefax:  +49 7151 1335770
E-Mail:   fra...@an...
Internet: www.anamica.de

Handelsregister: AG Stuttgart HRB 732357
Geschäftsführer: Yvonne Holzwarth, Frank Enderle

Re: [Cppcms-users] async connections and response headers

[Frank E. <fra...@an...>]

hi artyom,

the 404 issue is now fixed. making response().finalize() going away is
also much more intuitive; the cleanup after main if the application
didn't do so also.

what remains is the header issue. my test application still sends only
the content without headers. i attached capture from wireshark of the
http conversation between server and browser and the scgi conversation
between nginx and the application. while in the http conversation no
response headers are shown the scgi dump shows that at least
content-type and x-powered had been sent. nginx obviously filtered them
out due to the missing standard headers. the test application has been
compiled against svn-1317.

i also attached the application again since i did the changes you
suggested. i also tried to set the status code explicitly without success.

maybe you can have a look at this again - maybe i just made some mistake
in the application..

frank..

Am 13.07.2010 23:29, schrieb Artyom:
> Hello,
> 
> 
> 
>>
>> hey,
>>
>> i switched to the beta. however the problem with the missing  response
>> headers persist. so please notify me when you commit changes that  should
>> address this. meanwhile i try to live without response  headers.
> 
> 
> Ok. I cleaup the synchronous and asynchronous request handling added test cases 
> for both,
> now it looks fine, including sending 404 in case of unmapped url.
> 
> Hopefully no more surprises are expected.
> 
> One important change in behavior had added:
> 
> - if the application do not releases asynchronous request before exiting main 
> function, it is automatically
>   finalized and closed, so if you want to postpone response, don't forget to 
> call release_context().
> - it is no more required to call response().finalized() function as 
> async_complete_response() does all job.
> 
> 
> Thanks for the issue report,
> 
>   Artyom
> 
> 
>       
> 
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Sprint
> What will you do first with EVO, the first 4G phone?
> Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
> _______________________________________________
> Cppcms-users mailing list
> Cpp...@li...
> https://lists.sourceforge.net/lists/listinfo/cppcms-users


-- 
Dipl.-Ing. (FH) Frank Enderle

anamica UG (haftungsbeschränkt)
Beinsteinerstr. 6
71334 Waiblingen

Telefon:  +49 151 14981091
Telefax:  +49 7151 1335770
E-Mail:   fra...@an...
Internet: www.anamica.de

Handelsregister: AG Stuttgart HRB 732357
Geschäftsführer: Yvonne Holzwarth, Frank Enderle

Re: [Cppcms-users] async connections and response headers

[Frank E. <fra...@an...>]

thanks for pointing this out. it now works (yippie) but if i omit
setting the status explicit to 200 i still get no response headers. from
your mail i think it should default to 200 - shouldn't it? or is this a
flaw in the nginx's scgi implementation..

thanks again,

frank.

Am 14.07.2010 10:41, schrieb Artyom:
> Ok, after quick look on our code the problem is obvious:
> 
> You can't change response headers **after** writing data to output stream
> as in *CGI protocol all headers must be written before the response body.
> 
> See: 
> http://art-blog.no-ip.info/cppcms_ref_v0_99_1/classcppcms_1_1http_1_1response.html#39e48c676a3f2c79b26d60b8d455658c
> 
> 
> 
> So you must do all changed in headers before writing to output stream.
> 
> So instead of:
> 
>         release_context()->async_complete_response();
> 
> You need:
> 
>         
>         response().set_content_header("text/plain");
>         response().status(200);
>         response().out() << "test";
>         release_context()->async_complete_response();
> 
> Notes:
> 
> - Status is assumed be default in CGI protocols as 200, so you don't
>   need to set it explicitly, one status like 404, 400 or 301
>   require explicit CGI header Status:
> - You may use response().set_plain_text_header() instead of writing:
>   response().set_content_header("text/plain"); (which does the same).
> 
> So in short you may write this as:
> 
>         response().set_plain_text_header();
>         response().out() << "test";
>         release_context()->async_complete_response();
> 
> Note, set_content_header(std::string) still adds charset by default,
>   
> See: 
> http://art-blog.no-ip.info/cppcms_ref_v0_99_1/classcppcms_1_1http_1_1response.html#50ae55ff605bc655f5b19021f0f4f71d
> 
> 
> So if you want to set raw header use set_header(name,value) function.
> 
> 
> 
>> what remains is the header issue. my  test application still sends only
>> the content without headers. i attached  capture from wireshark of the
>> http conversation between server and browser  and the scgi conversation
>> between nginx and the application. while in the  http conversation no
>> response headers are shown the scgi dump shows that at  least
>> content-type and x-powered had been sent. nginx obviously filtered  them
>> out due to the missing standard headers. the test application has  been
>> compiled against svn-1317.
>>
>> i also attached the application again  since i did the changes you
>> suggested. i also tried to set the status code  explicitly without success.
>>
> 
> Artyom
> 
> 
>       
> 
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Sprint
> What will you do first with EVO, the first 4G phone?
> Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
> _______________________________________________
> Cppcms-users mailing list
> Cpp...@li...
> https://lists.sourceforge.net/lists/listinfo/cppcms-users


-- 
Dipl.-Ing. (FH) Frank Enderle

anamica UG (haftungsbeschränkt)
Beinsteinerstr. 6
71334 Waiblingen

Telefon:  +49 151 14981091
Telefax:  +49 7151 1335770
E-Mail:   fra...@an...
Internet: www.anamica.de

Handelsregister: AG Stuttgart HRB 732357
Geschäftsführer: Yvonne Holzwarth, Frank Enderle

Re: [Cppcms-users] async connections and response headers

[Artyom <art...@ya...>]

Hello,



> 
> hey,
> 
> i switched to the beta. however the problem with the missing  response
> headers persist. so please notify me when you commit changes that  should
> address this. meanwhile i try to live without response  headers.


Ok. I cleaup the synchronous and asynchronous request handling added test cases 
for both,
now it looks fine, including sending 404 in case of unmapped url.

Hopefully no more surprises are expected.

One important change in behavior had added:

- if the application do not releases asynchronous request before exiting main 
function, it is automatically
  finalized and closed, so if you want to postpone response, don't forget to 
call release_context().
- it is no more required to call response().finalized() function as 
async_complete_response() does all job.


Thanks for the issue report,

  Artyom


      

Re: [Cppcms-users] async connections and response headers

[Artyom <art...@ya...>]

Ok, after quick look on our code the problem is obvious:

You can't change response headers **after** writing data to output stream
as in *CGI protocol all headers must be written before the response body.

See: 
http://art-blog.no-ip.info/cppcms_ref_v0_99_1/classcppcms_1_1http_1_1response.html#39e48c676a3f2c79b26d60b8d455658c



So you must do all changed in headers before writing to output stream.

So instead of:

        release_context()->async_complete_response();

You need:

        
        response().set_content_header("text/plain");
        response().status(200);
        response().out() << "test";
        release_context()->async_complete_response();

Notes:

- Status is assumed be default in CGI protocols as 200, so you don't
  need to set it explicitly, one status like 404, 400 or 301
  require explicit CGI header Status:
- You may use response().set_plain_text_header() instead of writing:
  response().set_content_header("text/plain"); (which does the same).

So in short you may write this as:

        response().set_plain_text_header();
        response().out() << "test";
        release_context()->async_complete_response();

Note, set_content_header(std::string) still adds charset by default,
  
See: 
http://art-blog.no-ip.info/cppcms_ref_v0_99_1/classcppcms_1_1http_1_1response.html#50ae55ff605bc655f5b19021f0f4f71d


So if you want to set raw header use set_header(name,value) function.



> what remains is the header issue. my  test application still sends only
> the content without headers. i attached  capture from wireshark of the
> http conversation between server and browser  and the scgi conversation
> between nginx and the application. while in the  http conversation no
> response headers are shown the scgi dump shows that at  least
> content-type and x-powered had been sent. nginx obviously filtered  them
> out due to the missing standard headers. the test application has  been
> compiled against svn-1317.
> 
> i also attached the application again  since i did the changes you
> suggested. i also tried to set the status code  explicitly without success.
> 

Artyom


      

Re: [Cppcms-users] async connections and response headers

[Artyom <art...@ya...>]

Hello,

Looks like Nginx's SCGI backend does not confirm to CGI/SCGI standard
See: http://wiki.nginx.org/NginxNgxSCGIModule bugs section.

So I would suggest using FastCGI with nginx or use other web server like
lighttpd or apache that has confirming implementations.

Regards,
  Artyom


P.S.: Also I see that according to documentation of Nginx's SCGI:
      "In the current version SCRIPT_NAME and PATH_INFO are not defined.
      they are very important for CppCMS applications correct dispatching.



> 
> thanks for pointing this out. it now works (yippie) but if i omit
> setting the  status explicit to 200 i still get no response headers. from
> your mail i  think it should default to 200 - shouldn't it? or is this a
> flaw in the  nginx's scgi implementation..
> 
> thanks again,
> 
> frank.
> 


      

Re: [Cppcms-users] async connections and response headers

[Frank E. <fra...@an...>]

ok - then i'll switch over to fastcgi. thanks for investigating this.
the SCRIPT_NAME and PATH_INFO issue gave me also a lot of headache, but
i emulate them by using rewrite rules within nginx:

        location ~ ^/.* {
            set $script $uri;
            set $path_info "";

            if ($uri ~ "^/([^/]+)(/?.*)$") {
                set $script $1;
                set $path_info $2;
            }

            scgi_pass 127.0.0.1:8080;

            include scgi_params;
            scgi_param SCRIPT_NAME $script;
            scgi_param PATH_INFO $path_info;
        }

if you don't mind i'd like to ask one further question. the target of
the project i'm working on is to work with many, many longpoll
connections so i'm wondering why i need to nginx in front of cppcms.
wouldn't it be more efficient to have cppcms handle the http
communication itself? what are the advantages using a webserver in front
of cppcms (besides that it could deliver static content, etc..)

thanks,

frank

Am 14.07.2010 12:07, schrieb Artyom:
> Hello,
> 
> Looks like Nginx's SCGI backend does not confirm to CGI/SCGI standard
> See: http://wiki.nginx.org/NginxNgxSCGIModule bugs section.
> 
> So I would suggest using FastCGI with nginx or use other web server like
> lighttpd or apache that has confirming implementations.
> 
> Regards,
>   Artyom
> 
> 
> P.S.: Also I see that according to documentation of Nginx's SCGI:
>       "In the current version SCRIPT_NAME and PATH_INFO are not defined.
>       they are very important for CppCMS applications correct dispatching.
> 
> 
> 
>>
>> thanks for pointing this out. it now works (yippie) but if i omit
>> setting the  status explicit to 200 i still get no response headers. from
>> your mail i  think it should default to 200 - shouldn't it? or is this a
>> flaw in the  nginx's scgi implementation..
>>
>> thanks again,
>>
>> frank.
>>
> 
> 
>       
> 
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Sprint
> What will you do first with EVO, the first 4G phone?
> Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
> _______________________________________________
> Cppcms-users mailing list
> Cpp...@li...
> https://lists.sourceforge.net/lists/listinfo/cppcms-users


-- 
Dipl.-Ing. (FH) Frank Enderle

anamica UG (haftungsbeschränkt)
Beinsteinerstr. 6
71334 Waiblingen

Telefon:  +49 151 14981091
Telefax:  +49 7151 1335770
E-Mail:   fra...@an...
Internet: www.anamica.de

Handelsregister: AG Stuttgart HRB 732357
Geschäftsführer: Yvonne Holzwarth, Frank Enderle

Re: [Cppcms-users] async connections and response headers

[Artyom <art...@ya...>]

> 
> if you don't mind i'd like to ask one further question. the
> target of
> the project i'm working on is to work with many, many
> longpoll
> connections so i'm wondering why i need to nginx in front
> of cppcms.
> wouldn't it be more efficient to have cppcms handle the
> http
> communication itself?

Probably

> what are the advantages using a
> webserver in front
> of cppcms (besides that it could deliver static content,
> etc..)
> 

It would require implementation of highly secure
HTTP connector for CppCMS which is rather not simple as it would
require access from outside world.

HTTP servers very common targets for attacks so, it is better to
have a good server in front of you rather then developing your own.

So HTTP server gives you solid well debugged front end that
hides you from intruders.

Also CppCMS's HTTP server is very simple and developed for debugging
purposes only, also you may deploy it behind proxy if it sanitises
the input well.

Regards,
  Artyom