naviserver-devel — Developer discussion

Re: [naviserver-devel] PHP streaming hangs

[Vlad S. <vl...@cr...>]

> 
> Does this actually need to flush?
> 
> It depends when php calls it. In the case where there's no body, eg. a
> 302 redirect, then maybe this signals 'all done' and you're to write
> the headers. In the case where there is a body it's not needed because
> naviserver will construct and dump the headers when you try to write
> the first data, and it will send both with a single syscall (and
> packet, if it fits), which is more efficient.
> 
> So you should try not to flush. You might try to do nothing in this
> call, and detect when you've finished sending in the case of a
> zero-length body some other way. When php returns control to your
> Ns_Op function you could check to see if any bytes were sent and if
> not, flush the headers.

I tried it, with code before senfile implementation where we had flush 
function it worked perfectly, so PHP is calling this only when it needs 
to actually send headers. Not sure i can find out about body coming, 
even in 302 case there can be body but may be not. Will try again

> But anyway, you're checking for ctx->conn != NULL before setting the
> response code, then using it unconditionally when you write the data.
> 
> Ns_ConnWriteData can fail but you're ignoring the return code and
> returning success

It is mostly possible that the problem is in nsphp :-((

> 
> 
> Does php try to set a length header? You may need to check for it in
> here and call Ns_ConnSetLengthHeader.
> 

This i checked, no length


> 
> Whether the write succeeds or fails, you're returning the original
> buffer length, which I guess signals success to php. It might be hard
> to figure out exactly how much did get sent, because
> conn->nContentSent includes the headers bytes. But maybe php doesn't
> care? You'll have to check if you can return -1 or 0 or something.

when abort is called, i do not return str_length, it might be logjmp 
inside php


> You might also want to check if php exposes it's buffering. If you can
> figure out that your ub_write call is being called with all the data
> that's going to be sent (the script has finished executing) or the
> last piece of data, you shouldn't pass the flag NS_CONN_STREAM.
> Otherwise, you should.

No buffering, PHP calls it with all pieces separately

> 
> 
>> I tried that but could not get it to work, it generates configure but
>> when i run it i never could make it finish, there are always some errors.
> 
> 
> 
> Do you need to buy into the full build process? Maybe you can just
> pull in the header and link against the library..?

The problem with PHP that it can support only one SAPI implementation 
and nsphp is not just extesion but SAPI handler, so just compiling it as 
extension will not work

Re: [naviserver-devel] PHP streaming hangs

[Stephen D. <sd...@gm...>]

On Sat, Nov 8, 2008 at 5:00 AM, Vlad Seryakov <vl...@cr...> wrote:
> Yes, keep-alive is the key, nshttp_test closes connection so it cannot
> test this

/*
 * php_ns_sapi_send_headers() flushes the headers to the client.
 * Called before real content is sent by PHP.
 */

static int php_ns_sapi_send_headers(sapi_headers_struct *sapi_headers TSRMLS_DC)
{
    ns_context *ctx = SG(server_context);

    if (ctx->conn != NULL) {
        Ns_ConnSetResponseStatus(ctx->conn,
SG(sapi_headers).http_response_code);
    }

    Ns_ConnWriteData(ctx->conn, NULL, 0, NS_CONN_STREAM);

    return SAPI_HEADER_SENT_SUCCESSFULLY;
}


Does this actually need to flush?

It depends when php calls it. In the case where there's no body, eg. a
302 redirect, then maybe this signals 'all done' and you're to write
the headers. In the case where there is a body it's not needed because
naviserver will construct and dump the headers when you try to write
the first data, and it will send both with a single syscall (and
packet, if it fits), which is more efficient.

So you should try not to flush. You might try to do nothing in this
call, and detect when you've finished sending in the case of a
zero-length body some other way. When php returns control to your
Ns_Op function you could check to see if any bytes were sent and if
not, flush the headers.

But anyway, you're checking for ctx->conn != NULL before setting the
response code, then using it unconditionally when you write the data.

Ns_ConnWriteData can fail but you're ignoring the return code and
returning success.




/*
 * php_ns_sapi_header_handler() sets a HTTP reply header to be sent to
the client.
 */


Does php try to set a length header? You may need to check for it in
here and call Ns_ConnSetLengthHeader.



/*
 * php_ns_sapi_ub_write() writes data to the client connection.
 */

static int php_ns_sapi_ub_write(const char *str, uint str_length TSRMLS_DC)
{
    ns_context *ctx = SG(server_context);

    if (!ctx->conn) {
        int size = ctx->buffer ? strlen(ctx->buffer) : 0;
        ctx->buffer = ns_realloc(ctx->buffer, size + str_length + 1);
        strncpy(&(ctx->buffer[size]), str, str_length);
        ctx->buffer[size + str_length] = 0;
        return str_length;
    }

    if (Ns_ConnWriteData(ctx->conn, (void *) str, str_length,
NS_CONN_STREAM) != NS_OK) {
        php_handle_aborted_connection();
    }

    return str_length;
}


Whether the write succeeds or fails, you're returning the original
buffer length, which I guess signals success to php. It might be hard
to figure out exactly how much did get sent, because
conn->nContentSent includes the headers bytes. But maybe php doesn't
care? You'll have to check if you can return -1 or 0 or something.

You might also want to check if php exposes it's buffering. If you can
figure out that your ub_write call is being called with all the data
that's going to be sent (the script has finished executing) or the
last piece of data, you shouldn't pass the flag NS_CONN_STREAM.
Otherwise, you should.



> I tried that but could not get it to work, it generates configure but
> when i run it i never could make it finish, there are always some errors.



Do you need to buy into the full build process? Maybe you can just
pull in the header and link against the library..?

Re: [naviserver-devel] PHP streaming hangs

[Vlad S. <vl...@cr...>]

In telnet session, i do it endlessly, no hangs, mistery

GET /test.php HTTP/1.1
Host: vlad.mpower.net
Cookie: test=1

HTTP/1.1 304 Not Modified
MIME-Version: 1.0
Server: NaviServer/4.99.3
Date: Sat, 08 Nov 2008 05:06:44 GMT
X-Powered-By: PHP/5.2.6
Set-Cookie: test=1; expires=Sat, 08-Nov-2008 05:07:44 GMT; path=/
Content-type: text/html
Connection: keep-alive
Transfer-Encoding: chunked

0

Stephen Deasey wrote:
> On Sat, Nov 8, 2008 at 3:38 AM, Vlad Seryakov <vl...@cr...> wrote:
>> Added, runs fine but that the thing, it hangs in firefox, not in the test
>>
> 
> 
> OK, so there's a difference.
> 
> Maybe keepalive -- you mentioned what looked like responses garbled
> together? An HTTP 1.1 request will force keepalive:
> 
> test php-1.1 {sequential requests w/keepalive} -body {
>     foreach n {0 1 2} {
>         lappend results [nstest_http -http 1.1 \
>                              -getbody 1 \
>                              -getheaders content-length \
>                              GET /test.php]
>     }
>     set results
> } -cleanup {
>     unset -nocomplain results
> } -result "{200 10 {test page\n}} {200 10 {test page\n}} {200 10 {test page\n}}"
> 
> 
> 
> Also, can you adjust the build to use an already installed php? See:
> README.SELF-CONTAINED-EXTENSIONS in the php source for directions.
> 
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

Re: [naviserver-devel] PHP streaming hangs

[Vlad S. <vl...@cr...>]

Yes, keep-alive is the key, nshttp_test closes connection so it cannot 
test this

I tried that but could not get it to work, it generates configure but 
when i run it i never could make it finish, there are always some errors.


Stephen Deasey wrote:
> On Sat, Nov 8, 2008 at 3:38 AM, Vlad Seryakov <vl...@cr...> wrote:
>> Added, runs fine but that the thing, it hangs in firefox, not in the test
>>
> 
> 
> OK, so there's a difference.
> 
> Maybe keepalive -- you mentioned what looked like responses garbled
> together? An HTTP 1.1 request will force keepalive:
> 
> test php-1.1 {sequential requests w/keepalive} -body {
>     foreach n {0 1 2} {
>         lappend results [nstest_http -http 1.1 \
>                              -getbody 1 \
>                              -getheaders content-length \
>                              GET /test.php]
>     }
>     set results
> } -cleanup {
>     unset -nocomplain results
> } -result "{200 10 {test page\n}} {200 10 {test page\n}} {200 10 {test page\n}}"
> 
> 
> 
> Also, can you adjust the build to use an already installed php? See:
> README.SELF-CONTAINED-EXTENSIONS in the php source for directions.
> 
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

Re: [naviserver-devel] PHP streaming hangs

[Stephen D. <sd...@gm...>]

On Sat, Nov 8, 2008 at 3:38 AM, Vlad Seryakov <vl...@cr...> wrote:
>
> Added, runs fine but that the thing, it hangs in firefox, not in the test
>


OK, so there's a difference.

Maybe keepalive -- you mentioned what looked like responses garbled
together? An HTTP 1.1 request will force keepalive:

test php-1.1 {sequential requests w/keepalive} -body {
    foreach n {0 1 2} {
        lappend results [nstest_http -http 1.1 \
                             -getbody 1 \
                             -getheaders content-length \
                             GET /test.php]
    }
    set results
} -cleanup {
    unset -nocomplain results
} -result "{200 10 {test page\n}} {200 10 {test page\n}} {200 10 {test page\n}}"



Also, can you adjust the build to use an already installed php? See:
README.SELF-CONTAINED-EXTENSIONS in the php source for directions.

Re: [naviserver-devel] PHP streaming hangs

[Vlad S. <vl...@cr...>]

Added, runs fine but that the thing, it hangs in firefox, not in the test

Stephen Deasey wrote:
> On Fri, Nov 7, 2008 at 10:24 PM, Vlad Seryakov <vl...@cr...> wrote:
> 
>> I want to find the problem and then will put test for future checks
> 
> 
> No, you write the test to help find the bug.
> 
> 
>> It is funny, using telnet and submitting same headers never triggers
>> this, but when using firefox, after sevrral requests it hangs
> 
> 
> Rather than typing the same thing into telnet over and over again, and
> then repeating it with firefox, and then hoping other people can guess
> right and reproduce what you're doing and have them repeat it all too,
> why not check into CVS a 10 line test file?  It's just GOT to be
> easier, right?
> 
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

Re: [naviserver-devel] PHP streaming hangs

[Stephen D. <sd...@gm...>]

On Fri, Nov 7, 2008 at 10:24 PM, Vlad Seryakov <vl...@cr...> wrote:

> I want to find the problem and then will put test for future checks


No, you write the test to help find the bug.


> It is funny, using telnet and submitting same headers never triggers
> this, but when using firefox, after sevrral requests it hangs


Rather than typing the same thing into telnet over and over again, and
then repeating it with firefox, and then hoping other people can guess
right and reproduce what you're doing and have them repeat it all too,
why not check into CVS a 10 line test file?  It's just GOT to be
easier, right?

Re: [naviserver-devel] PHP streaming hangs

[Vlad S. <vl...@cr...>]

It is funny, using telnet and submitting same headers never triggers 
this, but when using firefox, after sevrral requests it hangs, and after 
timeout outputs:

0

c

test page

0

as it lost sequence and puts the whole buffer with trailer as the first 
one. I want to find the problem and then will put test for future checks

Stephen Deasey wrote:
> On Fri, Nov 7, 2008 at 9:23 PM, Vlad Seryakov <vl...@cr...> wrote:
>> This is the simple PHP page that triggers hanging, when attached to the
>> nsd, backtrace on all thread show they all are in the condwait or poll
>> wait, seems like sent/expected bytes miscalculation.
>>
>> <?php
>>
>> $test = @$_COOKIE["test"];
>> setcookie("test", "1", time() + 60, '/');
>>
>> if ($test == "1") {
>>    header("HTTP/1.0 304 Not Modified");
>>    exit();
>> }
>> ?>
>> test page
>>
> 
> 
> Make a 'tests' directory and add the above snippet. Make it run from
> 'make test'.
> 
> The 'nsvfs' module in CVS is an example of an external module which
> has tests using the nstest_http command. You could just 'cp -a
> ../nsvfs/tests tests' and then cut 'n paste from the Makefile.
> 

Re: [naviserver-devel] PHP streaming hangs

[Stephen D. <sd...@gm...>]

On Fri, Nov 7, 2008 at 9:23 PM, Vlad Seryakov <vl...@cr...> wrote:
>
> This is the simple PHP page that triggers hanging, when attached to the
> nsd, backtrace on all thread show they all are in the condwait or poll
> wait, seems like sent/expected bytes miscalculation.
>
> <?php
>
> $test = @$_COOKIE["test"];
> setcookie("test", "1", time() + 60, '/');
>
> if ($test == "1") {
>    header("HTTP/1.0 304 Not Modified");
>    exit();
> }
> ?>
> test page
>


Make a 'tests' directory and add the above snippet. Make it run from
'make test'.

The 'nsvfs' module in CVS is an example of an external module which
has tests using the nstest_http command. You could just 'cp -a
../nsvfs/tests tests' and then cut 'n paste from the Makefile.

[naviserver-devel] PHP streaming hangs

[Vlad S. <vl...@cr...>]

This is the simple PHP page that triggers hanging, when attached to the 
nsd, backtrace on all thread show they all are in the condwait or poll 
wait, seems like sent/expected bytes miscalculation.

<?php

$test = @$_COOKIE["test"];
setcookie("test", "1", time() + 60, '/');

if ($test == "1") {
    header("HTTP/1.0 304 Not Modified");
    exit();
}
?>
test page

Re: [naviserver-devel] naviserver/nsd driver.c,1.119,1.120

[Vlad S. <vl...@cr...>]

>>> For example, In the snippet above, you can see the newly added
>>> Ns_ConnSockContent which returns a pointer to the read buffer (which
>>> is otherwise private). Ordinarily the buffer would contain the HTTP
>>> request line, any headers, then any body. You can get at the body with
>>> the existing Ns_ConnContent. So if you just return "SYSLOG /
>>> HTTP/1.0\r\n\r\n" and then your bytes in Recv(), the request would be
>>> constructed and then you could parse the protocol beginning in the
>>> body content.
>>>
>>
>> I will try to take a look at this. ...
>>
> 
> 
> Did you take a look at this?  What do you think?

I did and still did not come to any conclusions. The biggest concern is 
in order to make driver simple all drivers will be complicated and look 
like a hacks except nssock :-))
I want other dirvers to be first class citizens the same way as nssock 
which means driver needs to do a little be more than to be pure simple 
HTTP server.

Re: [naviserver-devel] modules/nsssl nsssl.c,1.6,1.7

[Vlad S. <vl...@cr...>]

Great, thanks

> 
> 
> Fixed. For SSL (and for all drivers) just leave sendfFileProc as NULL
> and your sendProc will be called as needed to send file/buffer chunks.
> 
> If you were writing something like an FTP server, then you would want
> to copy nssock and call Ns_SockSendFileBufs to optimise the
> reading-from-disk/writing-to-network. Leaving it NULL will always be
> correct though.
> 
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

Re: [naviserver-devel] naviserver/nsd driver.c,1.119,1.120

[Stephen D. <sd...@gm...>]

On Wed, Oct 22, 2008 at 10:35 PM, Vlad Seryakov <vl...@cr...> wrote:
>>>> static ssize_t
>>>> Recv(Ns_Sock *sock, struct iovec *bufs, int nbufs, Ns_Time
>>>> *timeoutPtr, int flags)
>>>> {
>>>>     static const char request[] = "GET /whateva HTTP/1.0\r\n\r\n";
>>>>     size_t requestLen = sizeof(request);
>>>>     socklen_t sockLen = sizeof(struct sockaddr_in);
>>>>
>>>>     memcpy(bufs->iov_base, request, requestLen);
>>>>
>>>>     return recvfrom(sock->sock,
>>>>                     bufs->iov_base + requestLen,
>>>>                     bufs->iov_len - requestLen,
>>>>                     0, (struct sockaddr*) &sock->sa, &size);
>>>> }
>>>>
>>>> (checking for buffer lengths skipped here)
>>>>
>>>>
>>>> The advantage of doing it this way is that everything is much simpler
>>>> from the driver threads point of view. It doesn't need to know
>>>> anything special about non-standard protocols, and there isn't
>>>> anything in the driver callback api that isn't also useful to the HTTP
>>>> server.
>>>
>>> Arguable this is not very clear and simple, driver will have to hack
>>> buffers and every request should pretend to be HTTP just so driver
>>> thread would think that it serves only HTTP requests.
>>
>>
>> It's clear and simple. It's maybe not very pretty :-)
>>
>>
>>> If we have special codes and already have 3, does another one breaks
>>> anythings if only one code makes overall process simpler not only in
>>> driver thread but for drivers itself?
>>
>>
>> It's a trade off.
>>
>> Either non-standard drivers prepend their first read with 20 bytes in Recv().
>> (~ two lines of code)
>>
>> *OR*
>>
>> You need to change all parts of the sever that expect an HTTP request
>> structure to be available. AND you need to account for the fact that
>> sometimes it gets created in the normal fashion, but other times it
>> gets created later, or not at all.
>>
>>
>> You've tried to ways to do this:
>>
>>
>> 1) Exposing a new Ns_DriverSetRequest routine.
>>
>> Here's what it looked like in nssyslogd:
>>
>> http://naviserver.cvs.sourceforge.net/viewvc/naviserver/modules/nssyslogd/nssyslogd.c?revision=1.18&view=markup
>>
>> NS_EXPORT int
>> Ns_ModuleInit(char *server, char *module)
>> {
>>     ...
>>     Ns_RegisterRequest(server, "SYSLOG",  "/", SyslogRequestProc,
>> NULL, srvPtr, 0);
>>     ...
>> }
>>
>> static NS_DRIVER_ACCEPT_STATUS
>> Accept(Ns_Sock *sock, SOCKET listensock, struct sockaddr *sockaddrPtr,
>> int *socklenPtr)
>> {
>>     sock->sock = listensock;
>>     Ns_DriverSetRequest(sock, "SYSLOG / HTTP/1.0");
>>     return NS_DRIVER_ACCEPT_DATA;
>> }
>>
>>
>> How is this functionally different to what I proposed (example in Recv above)?
>>
>>
>> 2) Doing away with the need for a Ns_Request all together.
>>
>> Which now looks like this:
>>
>> http://naviserver.cvs.sourceforge.net/viewvc/naviserver/modules/nssyslogd/nssyslogd.c?revision=1.19&view=markup
>>
>> NS_EXPORT int
>> Ns_ModuleInit(char *server, char *module)
>> {
>>     ...
>>     init.requestProc = Request;
>>     ...
>>     init.opts = NS_DRIVER_ASYNC | NS_DRIVER_NOPARSE;
>>     ...
>> }
>>
>> static int
>> Request(void *arg, Ns_Conn *conn)
>> {
>>     Ns_DString *dsPtr = Ns_ConnSockContent(conn);
>>     SyslogRequest *req = SyslogRequestCreate(server, sockPtr->sock,
>> dsPtr->string, dsPtr->length, &sa);
>>     SyslogRequestProcess(req);
>>     ...
>> }
>>
>>
>> The strategy here seems to be to completely ignore the standard
>> request structure and do your own thing. There's nothing wrong with
>> this in the abstract -- you're not actually using any of the HTTP
>> stuff. However, the rest of the server fully expects the HTTP request
>> to be present and filled in, and this is why you've had to touch
>> dozens of files in dozens of places fixing up all code that might trip
>> over a null request field. (And the public API has changed...)
>>
>>
>> As far as I can see, neither method 1 or 2 achieves anything than
>> passing "METHOD / HTTP/1.0\r\n\r\n" does, using considerably less
>> code.
>>
>> For example, In the snippet above, you can see the newly added
>> Ns_ConnSockContent which returns a pointer to the read buffer (which
>> is otherwise private). Ordinarily the buffer would contain the HTTP
>> request line, any headers, then any body. You can get at the body with
>> the existing Ns_ConnContent. So if you just return "SYSLOG /
>> HTTP/1.0\r\n\r\n" and then your bytes in Recv(), the request would be
>> constructed and then you could parse the protocol beginning in the
>> body content.
>>
>
>
> I will try to take a look at this. ...
>


Did you take a look at this?  What do you think?

Re: [naviserver-devel] modules/nsssl nsssl.c,1.6,1.7

[Stephen D. <sd...@gm...>]

On Wed, Oct 22, 2008 at 8:58 PM, Vlad Seryakov <vl...@cr...> wrote:
> Stephen Deasey wrote:
>> On Wed, Oct 22, 2008 at 9:03 PM, Vlad Seryakov <vl...@cr...> wrote:
>>> I tried to use Ns_SockSendBufs() but it sends data directly to socket
>>> where i need to send via driver's send proc.
>>
>>
>> Just pass your Send() proc directly, and delete SendBufs():
>>
>>  static ssize_t
>>  SendFile(Ns_Sock *sock, Ns_FileVec *bufs, int nbufs, Ns_Time
>> *timeoutPtr, int flags)
>>  {
>> -    return Ns_SockSendFileBufsIndirect(sock->sock, bufs, nbufs,
>> timeoutPtr, flags, SendBufs);
>> +    return Ns_SockSendFileBufsIndirect(sock->sock, bufs, nbufs,
>> timeoutPtr, flags, Send);
>>  }
>>
>>
>>> Why connPtr->nContentSent is not updated, all send called with conn
>>> parameters should do this
>>
>>
>> Yes, but Ns_ConnSend() is higher up the stack, so when sending data it
>> looks something like this:
>>
>> return.c: Ns_ConnReturnData
>> connio.c: Ns_ConnWriteData
>> connio.c: Ns_ConnWriteVData
>> connio.c: Ns_ConnSend
>> driver.c: NsDriverSend
>> nsssl/nsssl.c: Send
>>
>>
>> By the time your Send() is called we've already passed through
>> Ns_ConnSend(). nContentSent will be updated as the stack unwinds.
>>
>> If you call Ns_ConnSend() from within Send(), that's a loop...
>>
>
> The first argument is different, Ns_Sock vs SOCKET
>


Fixed. For SSL (and for all drivers) just leave sendfFileProc as NULL
and your sendProc will be called as needed to send file/buffer chunks.

If you were writing something like an FTP server, then you would want
to copy nssock and call Ns_SockSendFileBufs to optimise the
reading-from-disk/writing-to-network. Leaving it NULL will always be
correct though.

Re: [naviserver-devel] Relocating ns_atclose calls

[Vlad S. <vl...@cr...>]

Actually, i do not need it anymore


Stephen Deasey wrote:
> On Fri, Nov 7, 2008 at 4:32 PM, Vlad Seryakov <vl...@cr...> wrote:
> 
>> In most cases ... it will work as before because
>> atclose blocks and socket is not yet returned to driver
> 
> 
> driver.c:NsSockClose() does return the Sock to the driver thread, and
> it does not block.


>> ... if i need to use socket garanteed at connection end, this is the
>> only place i can do it.
> 
> 
> Why? What are you trying to do?
> 
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

Re: [naviserver-devel] naviserver/tcl fastpath.tcl,1.5,1.6

[Vlad S. <vl...@cr...>]

Yes, bad change

Stephen Deasey wrote:
> On Fri, Nov 7, 2008 at 3:12 AM, Vlad Seryakov
> <ser...@us...> wrote:
> 
>> +     # If address set to 0.0.0.0 in nssock to listen on
>> +     # all interfaces we may try to use Host: header to
>> +     # return proper urls
>> +     #
>> +
>> +     if { $loc == "http://0.0.0.0" } {
>> +       set host [ns_set iget [ns_conn headers] Host]
>> +       if { $host != "" } {
>> +         set loc http://$host
>> +       }
>> +     }
> 
> 
> You can't take unfiltered input from the host header and then feed it
> back to ns_returnredirect as this vulnerable to a response splitting
> attack.
> 
>   http://www.google.co.uk/search?q=response+splitting
> 
> It seems like this may be a more general kind of bug. After all, it's
> not just in directory listings that 0.0.0.0 is an invalid host
> address. Take a look at conn.c:Ns_ConnLocationAppend() -- looks like
> this is where the fix needs to go.
> 
> This is a tricky are so add some extra tests in tests/ns_conn_host.test
> 
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

Re: [naviserver-devel] naviserver/tcl fastpath.tcl,1.5,1.6

[Stephen D. <sd...@gm...>]

On Fri, Nov 7, 2008 at 3:12 AM, Vlad Seryakov
<ser...@us...> wrote:

> +     # If address set to 0.0.0.0 in nssock to listen on
> +     # all interfaces we may try to use Host: header to
> +     # return proper urls
> +     #
> +
> +     if { $loc == "http://0.0.0.0" } {
> +       set host [ns_set iget [ns_conn headers] Host]
> +       if { $host != "" } {
> +         set loc http://$host
> +       }
> +     }


You can't take unfiltered input from the host header and then feed it
back to ns_returnredirect as this vulnerable to a response splitting
attack.

  http://www.google.co.uk/search?q=response+splitting

It seems like this may be a more general kind of bug. After all, it's
not just in directory listings that 0.0.0.0 is an invalid host
address. Take a look at conn.c:Ns_ConnLocationAppend() -- looks like
this is where the fix needs to go.

This is a tricky are so add some extra tests in tests/ns_conn_host.test

Re: [naviserver-devel] Relocating ns_atclose calls

[Stephen D. <sd...@gm...>]

On Fri, Nov 7, 2008 at 4:32 PM, Vlad Seryakov <vl...@cr...> wrote:

> In most cases ... it will work as before because
> atclose blocks and socket is not yet returned to driver


driver.c:NsSockClose() does return the Sock to the driver thread, and
it does not block.


> ... if i need to use socket garanteed at connection end, this is the
> only place i can do it.


Why? What are you trying to do?

Re: [naviserver-devel] Relocating ns_atclose calls

[Vasiljevic Z. <zv...@ar...>]

On 07.11.2008, at 17:32, Vlad Seryakov wrote:

>
> Is everybody OK with this?

I can live with this.
Zoran

[naviserver-devel] Relocating ns_atclose calls

[Vlad S. <vl...@cr...>]

i have a suggestion to move NsTclRunAtClose(connPtr->itPtr); in connio.c 
Ns_connClose before NsSockClose(connPtr->sockPtr, keep); to give atclose 
handlers a chance to write something into still opened socket.
In most cases it will not be used and it will work as before because 
atclose blocks and socket is not yet returned to driver but at the same 
time if i need to use socket garanteed at connection end, this is the 
only place i can do it.
Filters are not garanteed to be called, they depend on return status.

Is everybody OK with this?

Re: [naviserver-devel] nsphp and other broken things

[Vlad S. <vl...@cr...>]

Ok, while preparing test case i was able to pinpoint the location of the 
problem.
I use PmWiki and it uses If-modified-Since header to return 304 
responses. In new code send_headers was no-op but actually i have to 
call Ns_ConnWriteData with 0 length to flush headers.

Now it works, it was module problem.

Stephen Deasey wrote:
> On Tue, Oct 28, 2008 at 5:02 PM, Vlad Seryakov <vl...@cr...> wrote:
>> After 2008-09-11, nsphp stopped working, even with the latest update to
>> non-obsolete functions, it looks like something in the core flow,
>> especially streaming part doe snot work properly.
>>
>> I tried all versions and the latest stable is marked in ChangeLog at
>> 2008-09-11.
>>
>> This is only HTTP related and it could be problem in nsphp, but it was
>> very stable until functions it used became obsolete. I started using
>> Ns_ConnWriteData with or without NS_CONN_STREAM and browsers stopped
>> showing web pages.
>>
>> These functions would tell that streaming is required and no
>> Content-Length will be supplied:
>>
>> Ns_ConnSetRequiredHeaders(ctx->conn, ctype, -1);
>> Ns_ConnFlushHeaders(ctx->conn, SG(sapi_headers).http_response_code);
>>
>> Then
>> Ns_ConnWriteData without any flags would just send data to the browser
>>
>> In new code, the server does all things automatically but NS_CONN_STREAM
>> flags is required to tell that we are streaming. Flush is not used
>> anymore. When i telnet i get page back without content-length but still,
>> every second request hangs or comes empty.
>>
> 
> 
> I'm not sure how you're triggering this. Can you write a test and
> point me at it?
> 
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

Re: [naviserver-devel] nsphp and other broken things

[Vlad S. <vl...@cr...>]

Nope, did not help in all cases.
Now, it handles 304 correctly but next request to other page results in 
timeout and then result like this:

0

HTTP/1.1 304 Not Modified
MIME-Version: 1.0
Server: NaviServer/4.99.2
Date: Thu, 30 Oct 2008 01:35:29 GMT
X-Powered-By: PHP/5.2.6
Content-type: text/html
Connection: keep-alive
Transfer-Encoding: chunked

0

I am using PmWiki, i attached very simple and basic installation. Try to 
click on links and after several requests it show chunked headers. This 
happens after first headers flush

Vlad Seryakov wrote:
> Ok, while preparing test case i was able to pinpoint the location of the 
> problem.
> I use PmWiki and it uses If-modified-Since header to return 304 
> responses. In new code send_headers was no-op but actually i have to 
> call Ns_ConnWriteData with 0 length to flush headers.
> 
> Now it works, it was module problem.
> 
> Stephen Deasey wrote:
>> On Tue, Oct 28, 2008 at 5:02 PM, Vlad Seryakov 
>> <vl...@cr...> wrote:
>>> After 2008-09-11, nsphp stopped working, even with the latest update to
>>> non-obsolete functions, it looks like something in the core flow,
>>> especially streaming part doe snot work properly.
>>>
>>> I tried all versions and the latest stable is marked in ChangeLog at
>>> 2008-09-11.
>>>
>>> This is only HTTP related and it could be problem in nsphp, but it was
>>> very stable until functions it used became obsolete. I started using
>>> Ns_ConnWriteData with or without NS_CONN_STREAM and browsers stopped
>>> showing web pages.
>>>
>>> These functions would tell that streaming is required and no
>>> Content-Length will be supplied:
>>>
>>> Ns_ConnSetRequiredHeaders(ctx->conn, ctype, -1);
>>> Ns_ConnFlushHeaders(ctx->conn, SG(sapi_headers).http_response_code);
>>>
>>> Then
>>> Ns_ConnWriteData without any flags would just send data to the browser
>>>
>>> In new code, the server does all things automatically but NS_CONN_STREAM
>>> flags is required to tell that we are streaming. Flush is not used
>>> anymore. When i telnet i get page back without content-length but still,
>>> every second request hangs or comes empty.
>>>
>>
>>
>> I'm not sure how you're triggering this. Can you write a test and
>> point me at it?
>>
>> -------------------------------------------------------------------------
>> This SF.Net email is sponsored by the Moblin Your Move Developer's 
>> challenge
>> Build the coolest Linux based applications with Moblin SDK & win great 
>> prizes
>> Grand prize is a trip for two to an Open Source event anywhere in the 
>> world
>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>> _______________________________________________
>> naviserver-devel mailing list
>> nav...@li...
>> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>>
> 

Re: [naviserver-devel] nsphp and other broken things

[Stephen D. <sd...@gm...>]

On Tue, Oct 28, 2008 at 5:02 PM, Vlad Seryakov <vl...@cr...> wrote:
> After 2008-09-11, nsphp stopped working, even with the latest update to
> non-obsolete functions, it looks like something in the core flow,
> especially streaming part doe snot work properly.
>
> I tried all versions and the latest stable is marked in ChangeLog at
> 2008-09-11.
>
> This is only HTTP related and it could be problem in nsphp, but it was
> very stable until functions it used became obsolete. I started using
> Ns_ConnWriteData with or without NS_CONN_STREAM and browsers stopped
> showing web pages.
>
> These functions would tell that streaming is required and no
> Content-Length will be supplied:
>
> Ns_ConnSetRequiredHeaders(ctx->conn, ctype, -1);
> Ns_ConnFlushHeaders(ctx->conn, SG(sapi_headers).http_response_code);
>
> Then
> Ns_ConnWriteData without any flags would just send data to the browser
>
> In new code, the server does all things automatically but NS_CONN_STREAM
> flags is required to tell that we are streaming. Flush is not used
> anymore. When i telnet i get page back without content-length but still,
> every second request hangs or comes empty.
>


I'm not sure how you're triggering this. Can you write a test and
point me at it?

[naviserver-devel] nsphp and other broken things

[Vlad S. <vl...@cr...>]

After 2008-09-11, nsphp stopped working, even with the latest update to 
non-obsolete functions, it looks like something in the core flow, 
especially streaming part doe snot work properly.

I tried all versions and the latest stable is marked in ChangeLog at 
2008-09-11.

This is only HTTP related and it could be problem in nsphp, but it was 
very stable until functions it used became obsolete. I started using 
Ns_ConnWriteData with or without NS_CONN_STREAM and browsers stopped 
showing web pages.

These functions would tell that streaming is required and no 
Content-Length will be supplied:

Ns_ConnSetRequiredHeaders(ctx->conn, ctype, -1);
Ns_ConnFlushHeaders(ctx->conn, SG(sapi_headers).http_response_code);

Then
Ns_ConnWriteData without any flags would just send data to the browser

In new code, the server does all things automatically but NS_CONN_STREAM 
flags is required to tell that we are streaming. Flush is not used 
anymore. When i telnet i get page back without content-length but still, 
every second request hangs or comes empty.

Re: [naviserver-devel] How to listen on 2 ports?

[Vasiljevic Z. <zv...@ar...>]

On 24.10.2008, at 21:04, Vlad Seryakov wrote:

> Just load nssock second time with different name
>
> ns_section ns/server/modules
> ns_param nssock nssock.so
> ns_param nssock2 nssock.so
>
> ns_section ns/server/module/nssock
> ns_param	port 90
>
> ns_section ns/server/module/nssock2
> ns_param	port 81

Yo! Very nice. Thanks a lot!
Zoran