whisker-devel

[whisker] Parsing requests.

[Misto O. <mis...@gm...>]

Hi to all!

I'm usign lib-whisker as http lib.
I have my own multi thread server with it own socket.
I want to parse an http request with whisker using the server's socket;
I didn't found any usable function for this behaviour. I'm arranged just
cutting and past code from the function _http_do_request_ex , (code:1
*see below)
calling this new function read_req, I wrote this code

my %req = LW2::http_new_request();	
my %res;
my $client = $socket->accept();
my $stream  = LW2::stream_setsock($client);
my $r = LW2::read_req($stream,\%req,\%res);	

The problem is that the headers are not correctly parsed.
but the body works fine.

print $res{whisker}->{data};   

Any suggestion?  There is another mode to do this work, or is the lib
that doesn't support
requests headers?

Thanks for your time.

----------------------------------- [code 1] ------------------------------
sub read_req{
    my ($stream, $hin, $hout, $raw)=@_; 

    return 2 if(! defined $stream);
    return 2 if(!(defined $hin  && ref($hin) ));
    return 2 if(!(defined $hout && ref($hout)));
    my $W = $hin->{whisker};

##### read and parse response
 my @H;
 if($$W{'version'} ne '0.9'){
 do { # catch '100 Continue' responses
  my $resp=_http_getline($stream);
  
  if(!defined $resp){
	$$hout{whisker}->{error}='error reading HTTP response';
	$$hout{whisker}->{data}=$stream->{bufin};
	$stream->{'close'}->();
	return 1;}

  $$hout{whisker}->{'raw_header_data'}.=$resp
	if(defined $$W{'save_raw_headers'});

  if($resp!~/^([^\/]+)\/(\d\.\d)([ \t]+)(\d+)([ \t]*)(.*?)([\r\n]+)/){
	$$hout{whisker}->{'error'}='invalid HTTP response';
	$$hout{whisker}->{'data'}=$resp;
	while(defined ($_=_http_getline($stream))){ 
		$$hout{whisker}->{'data'}.=$_;}
	$stream->{'close'}->();
	return $$W{'invalid_protocol_return_value'}||1;}

  $$hout{whisker}->{protocol}		= $1;
  $$hout{whisker}->{version}		= $2;
  $$hout{whisker}->{http_space1}	= $3;
  $$hout{whisker}->{code}		= $4;
  $$hout{whisker}->{http_space2}	= $5;
  $$hout{whisker}->{message}		= $6;
  $$hout{whisker}->{http_eol}		= $7;
  $$hout{whisker}->{'100_continue'}++ if($4 == 100);

  @H=http_read_headers($stream,$hin,$hout);
  if(!$H[0]){
	$$hout{whisker}->{'error'}='Error in reading headers: '
		.$H[1];
	$stream->{'close'}->(); return 1; }

  if( !defined $H[3] ){ # connection
	$H[3] = (defined $$hin{'Connection'}) ? lc($$hin{'Connection'}) : 
		'close'; }

 } while($$hout{whisker}->{'code'}==100);

 } else { # http ver 0.9, we need to fake it since headers are not sent
  $$hout{whisker}->{version}	='0.9';
  $$hout{whisker}->{code}	=200;
  $$hout{whisker}->{http_message}='';
  $H[3]='close';
 }

 if(defined $$W{data_sock}){
   $$hout{whisker}->{data_sock} = $stream->{sock};
 } else {
   if($$W{'force_bodysnatch'} || ( $$W{'method'} ne 'HEAD' && 
		$$hout{whisker}->{'code'}!=206 &&
		$$hout{whisker}->{'code'}!=102)){
	return 1 if(!http_read_body($stream,$hin,$hout,$H[1],$H[2]));
   }

   if(($H[3] ne 'keep-alive' || ( defined $$hin{'Connection'} &&
		$$hin{'Connection'}=~m/close/i)) && $$W{'force_open'}!=1){
	$stream->{'close'}->();
   }	 

   $stream->{'close'}->() if($$W{'force_close'}>0 || $$W{'ssl'}>0);
 }

 if(defined $$W{'header_delete_on_success'} &&
ref($$W{'header_delete_on_success'})){
 	foreach (@{ $$W{'header_delete_on_success'} }){
 		delete $hin->{$_} if(exists $hin->{$_}); }
	delete $$W{header_delete_on_success};
 }

 $stream->{reqs}++;
 $$hout{whisker}->{'stats_reqs'}=$stream->{reqs};
 $$hout{whisker}->{'stats_syns'}=$stream->{syns};
 $$hout{whisker}->{'socket_state'}=$stream->{state};
 delete $$hout{whisker}->{'error'}; # no error
 return 0;

}
----------------------------------- [eof]------------------------------

Re: [whisker] Parsing requests.

[<rf...@wi...>]

On Tue, 11 Jan 2005, Misto Orp wrote:

> I want to parse an http request with whisker using the server's socket;

That's quite advanced libwhisker-fu!  But yes, it's possible.  In fact, 
when I redid 1.x into 2.0, I purposefully rearranged some internal 
functions specifically so that you can use libwhisker in server 
applications.  I've been using libwhisker 2.x to power my own proxy and 
web server applications for quite a while now.

> I didn't found any usable function for this behaviour.

You're right, I didn't include any actual function to parse requests in 
the library.  I'll consider doing so for a future version.

Pasted below is pseudo-code for what I use in my applications to parse 
HTTP requests.  Note that it might need slight tweaks in the handling of 
Connection/Proxy-Connection headers and the target host+port, depending on 
if you're using it in a proxy situation vs. a webserver situation.

I did not actually test the code below, so it might need some tiny 
changes; but for the most part it should illustrate how to go about doing 
it.

Cheers,
- rfp

-----------------------------------------------------------------------


#######################################
# setup socket
#

$stream = LW2::stream_setsock( $socket );
$stream->{timeout}='10';

#######################################
# attempt to read the request line
#

# find the position of the [CR]LF
$lf_pos = index($stream->{bufin}, "\n");

while( $lf_pos == -1 ){

 	# LF not found, try to read more into buffer
 	if(!$stream->{read}->()){

 		# reading more data failed or timed out, so abort

 		# TODO: abort handling client
 		die/return;
 	}

 	# We got more data, so let's see if we have the LF yet
 	$lf_pos = index($stream->{bufin}, "\n");
}


# if we get here, we have a request line; let's parse it
$reqline = substr($stream->{bufin}, 0, $pos+1, '');

if($reqline !~ m/^([A-Z]+)[ \t]+([^ \t]+)[ \t]+HTTP\/([.0-9]+)/){

 	# parsing the request failed, so abort

 	# TODO: abort handling client
 	die/return;
}

# setup some request values
$method = $1;
$uri = $2;
$version = $3;

#######################################
# create a new request and set some values
#

# initialize a new request
$request = LW2::http_new_request();

# delete all headers
foreach (keys %$request){
 	next if($_ eq 'whisker');
 	delete $request->{$_};
}

# setup some default values necessary to parse the request
$request->{whisker}->{lowercase_incoming_headers} = 0;
$request->{whisker}->{normalize_incoming_headers} = 1;
$request->{whisker}->{ignore_duplicate_headers} = 0;

# put our values into the request
$request->{whisker}->{method} = $method;
$request->{whisker}->{version} = $version;

# parse in the URI
LW2::uri_split($uri, $request);

#
# NOTE: {whisker}->{host} and {whisker}->{port} are already known,
# since the client is contacting us...that is, unless it's a proxy
# application, in which case, the URI will contain the host and port
#

#######################################
# handle the headers
#

@p = LW2::http_read_headers($stream, $request, $request);

# check for error
if($p[0]==0){
 	# there was an error reading the headers

 	# TODO: abort handling the client
 	die/return;
}

#######################################
# handle the body content
#

if(defined $p[2] && $p[2] > 0){

 	# read the body of the request
 	$res = LW2::http_read_body($stream, $request, $request,
 		lc($p[1]), $p[2]);

 	if(!$res){
 		# error reading body

 		# TODO: abort handling the client
 		die/return;
 	}
}

# at this point, our request should be all set to go!