From: Thinus P. <th...@er...> - 2005-09-26 09:36:32
|
Hi there We're sending an http request to yaws. See (1). The request contains a string called ussdString. This string is a uuencoded string ussdString="*116*111*AAEGAGQAABEzAABgNIUgAABRdNAABYcAAAAAAAAAgQ6+8zh0D9iBDr7zOHQP2A==1#" Yaws captures this data via script (2). In our logs we get the string as: ussdString=\"*116*111*AAEGAGQAABEzAABgNIUgAABWEtAABYcAAAAAAAAAgQ6<space>8zh0D9iBDr7zOHQP2A==1#\" What's happing is that yaws is dropping the '+' character from our string and inserting a space (verified as ascii code 32 (decimal)) in its place. The tcpdump confirms that the message is sent correctly to yaws. After getting the data from the http request in yaws we get this character replacement. Is this a bug in yaws, or is the + character something special? Or is the problem that we just can't write a proper yaws script ;) The only strings that gets mangled are those containing a + character somewhere. Any ideas? Cheers Thinus ***************************************************** (1) tcpdump of the http request. Hypertext Transfer Protocol Connection: keep-alive\r\n Host: 172.16.100.105:8080\r\n User-Agent: com.truteq.net.http.Connection\r\n Content-type: text/xml\r\n Accept: text/xml\r\n Content-length: 146\r\n \r\n Line-based text data: text/xml <ussdreq tid="999999" reqType="1" msisdn="27842202223" ussdString="*116*111*AAEGAGQAABEzAABgNIUgAABRdNAABYcAAAAAAAAAgQ6+8zh0D9iBDr7zOHQP2A==1#" /> Hypertext Transfer Protocol \r\n ****************************************************** ****************************************************** (2) Script to capture data in yaws <erl> out(Args) -> GetValue = fun(Data, What) -> Pos = string:str(Data, What), Ret1 = string:substr(Data, Pos+length(What)+2, length(Data)-Pos), Pos2 = string:chr(Ret1, 34), string:substr(Ret1,1,Pos2-1) end, {http_request, HttpRequest, Doc, Version} = Args#arg.req, case HttpRequest of 'POST' -> [{UssdReq, PostData}] = yaws_api:parse_post(Args); 'GET' -> PostData = yaws_api:parse_query(Args) end, Tid = string:substr(PostData, 2, 7), error_logger:info_msg("~nPostData = ~p~n",[PostData]), %% Extract required data Msisdn = GetValue(PostData, "msisdn"), ReqType = GetValue(PostData, "reqType"), SubType = GetValue(PostData, "subType"), UssdString = GetValue(PostData, "ussdString"), UssdStringBin = list_to_binary(UssdString), %% Required variables by prod system. error_logger:info_msg("~nUssdString = ~p~nUssdString Binary: ~p~n", [UssdString, UssdStringBin]), *snip* ***************************************************** -- Thinus Pollard Mobile: +27 72 075 2751 |
From: Rob S. <ro...@sc...> - 2005-09-26 10:35:57
|
Thinus Pollard wrote: > Hi there > > We're sending an http request to yaws. See (1). The request contains a string > called ussdString. This string is a uuencoded string > > ussdString="*116*111*AAEGAGQAABEzAABgNIUgAABRdNAABYcAAAAAAAAAgQ6+8zh0D9iBDr7zOHQP2A==1#" > > Yaws captures this data via script (2). > > In our logs we get the string as: > ussdString=\"*116*111*AAEGAGQAABEzAABgNIUgAABWEtAABYcAAAAAAAAAgQ6<space>8zh0D9iBDr7zOHQP2A==1#\" > > What's happing is that yaws is dropping the '+' character from our string and > inserting a space (verified as ascii code 32 (decimal)) in its place. > <snip some additonal info> This is correct behaviour and is due to url encoding, that is making a string safe to be send in a URL, One of the things that happens is that a space is translated into a '+' sign. Yaws 'decodes any string and thus transfers any '+' into a space. In order to have this working correctly you'll have to URL encode the string before you send it (the '+' sign will then be transferred into a %xx, which yaws will correctly decode) Regards /Rob |
From: Daniel S. <da...@er...> - 2005-09-26 11:13:39
|
Would this change with RFC1738 ----- snip Thus, only alphanumerics, the special characters "$-_.+!*'(),", and reserved characters used for their reserved purposes may be used unencoded within a URL. ----- end snip Regards On Monday 26 September 2005 12:35, Rob Schmersel wrote: > Thinus Pollard wrote: > > Hi there > > > > We're sending an http request to yaws. See (1). The request contains a > > string called ussdString. This string is a uuencoded string > > > > ussdString="*116*111*AAEGAGQAABEzAABgNIUgAABRdNAABYcAAAAAAAAAgQ6+8zh0D9iB > >Dr7zOHQP2A==1#" > > > > Yaws captures this data via script (2). > > > > In our logs we get the string as: > > ussdString=\"*116*111*AAEGAGQAABEzAABgNIUgAABWEtAABYcAAAAAAAAAgQ6<space>8 > >zh0D9iBDr7zOHQP2A==1#\" > > > > What's happing is that yaws is dropping the '+' character from our string > > and inserting a space (verified as ascii code 32 (decimal)) in its place. > > <snip some additonal info> > > This is correct behaviour and is due to url encoding, that is making a > string safe to be send in a URL, One of the things that happens is that > a space is translated into a '+' sign. Yaws 'decodes any string and thus > transfers any '+' into a space. > In order to have this working correctly you'll have to URL encode the > string before you send it (the '+' sign will then be transferred into a > %xx, which yaws will correctly decode) > > Regards > /Rob > > > ------------------------------------------------------- > SF.Net email is sponsored by: > Tame your development challenges with Apache's Geronimo App Server. > Download it for free - -and be entered to win a 42" plasma tv or your very > own Sony(tm)PSP. Click here to play: http://sourceforge.net/geronimo.php > _______________________________________________ > Erlyaws-list mailing list > Erl...@li... > https://lists.sourceforge.net/lists/listinfo/erlyaws-list -- Daniel H Schutte Mobile: +27 84 468 3138 Office: +27 11 675 2643 The more laws and order are made prominent, the more thieves and robbers there will be. -- Lao Tsu |
From: Daniel S. <da...@er...> - 2005-09-26 12:00:20
|
One other question on this, This is XML inside the page and technically not part of the URL. Why would URL encoding still apply? Kind regards On Monday 26 September 2005 12:35, Rob Schmersel wrote: > Thinus Pollard wrote: > > Hi there > > > > We're sending an http request to yaws. See (1). The request contains a > > string called ussdString. This string is a uuencoded string > > > > ussdString="*116*111*AAEGAGQAABEzAABgNIUgAABRdNAABYcAAAAAAAAAgQ6+8zh0D9iB > >Dr7zOHQP2A==1#" > > > > Yaws captures this data via script (2). > > > > In our logs we get the string as: > > ussdString=\"*116*111*AAEGAGQAABEzAABgNIUgAABWEtAABYcAAAAAAAAAgQ6<space>8 > >zh0D9iBDr7zOHQP2A==1#\" > > > > What's happing is that yaws is dropping the '+' character from our string > > and inserting a space (verified as ascii code 32 (decimal)) in its place. > > <snip some additonal info> > > This is correct behaviour and is due to url encoding, that is making a > string safe to be send in a URL, One of the things that happens is that > a space is translated into a '+' sign. Yaws 'decodes any string and thus > transfers any '+' into a space. > In order to have this working correctly you'll have to URL encode the > string before you send it (the '+' sign will then be transferred into a > %xx, which yaws will correctly decode) > > Regards > /Rob > > > ------------------------------------------------------- > SF.Net email is sponsored by: > Tame your development challenges with Apache's Geronimo App Server. > Download it for free - -and be entered to win a 42" plasma tv or your very > own Sony(tm)PSP. Click here to play: http://sourceforge.net/geronimo.php > _______________________________________________ > Erlyaws-list mailing list > Erl...@li... > https://lists.sourceforge.net/lists/listinfo/erlyaws-list -- Daniel H Schutte Mobile: +27 84 468 3138 Office: +27 11 675 2643 The more laws and order are made prominent, the more thieves and robbers there will be. -- Lao Tsu |
From: Rob S. <ro...@sc...> - 2005-09-26 14:44:57
|
Sorry missed that, Yaws should not change the body of the request (and as far as I know does not change it, never had any problem with it). /Rob Daniel Schutte wrote: > One other question on this, > > This is XML inside the page and technically not part of the URL. Why would > URL encoding still apply? > > Kind regards > > On Monday 26 September 2005 12:35, Rob Schmersel wrote: > >>Thinus Pollard wrote: >> >>>Hi there >>> >>>We're sending an http request to yaws. See (1). The request contains a >>>string called ussdString. This string is a uuencoded string >>> >>>ussdString="*116*111*AAEGAGQAABEzAABgNIUgAABRdNAABYcAAAAAAAAAgQ6+8zh0D9iB >>>Dr7zOHQP2A==1#" >>> >>>Yaws captures this data via script (2). >>> >>>In our logs we get the string as: >>>ussdString=\"*116*111*AAEGAGQAABEzAABgNIUgAABWEtAABYcAAAAAAAAAgQ6<space>8 >>>zh0D9iBDr7zOHQP2A==1#\" >>> >>>What's happing is that yaws is dropping the '+' character from our string >>>and inserting a space (verified as ascii code 32 (decimal)) in its place. >> >><snip some additonal info> >> >>This is correct behaviour and is due to url encoding, that is making a >>string safe to be send in a URL, One of the things that happens is that >>a space is translated into a '+' sign. Yaws 'decodes any string and thus >>transfers any '+' into a space. >>In order to have this working correctly you'll have to URL encode the >>string before you send it (the '+' sign will then be transferred into a >>%xx, which yaws will correctly decode) >> >>Regards >>/Rob >> >> >>------------------------------------------------------- >>SF.Net email is sponsored by: >>Tame your development challenges with Apache's Geronimo App Server. >>Download it for free - -and be entered to win a 42" plasma tv or your very >>own Sony(tm)PSP. Click here to play: http://sourceforge.net/geronimo.php >>_______________________________________________ >>Erlyaws-list mailing list >>Erl...@li... >>https://lists.sourceforge.net/lists/listinfo/erlyaws-list > > |
From: David H. <dav...@bl...> - 2005-09-26 15:10:56
|
Rob Schmersel wrote: > Thinus Pollard wrote: > >>What's happing is that yaws is dropping the '+' character from our string and >>inserting a space (verified as ascii code 32 (decimal)) in its place. > > <snip some additonal info> > > This is correct behaviour and is due to url encoding, that is making a > string safe to be send in a URL, One of the things that happens is that > a space is translated into a '+' sign. Yaws 'decodes any string and thus > transfers any '+' into a space. > In order to have this working correctly you'll have to URL encode the > string before you send it (the '+' sign will then be transferred into a > %xx, which yaws will correctly decode) Alternatively, you could use the "modified Base64 for URL" encoding mentioned at <http://en.wikipedia.org/wiki/Base64> instead of uuencode. -- David Hopwood <dav...@bl...> |
From: Daniel S. <da...@er...> - 2005-09-26 15:21:42
|
The problem is actually that we send a UUencoded string onwards to a service provider - they convert it to XML content and forward it to us. The devices are out in the field and can not be changed rapidly. Thanks to Klacke, there is a way to get the "raw" content - so that we do not have this issue. On Monday 26 September 2005 17:10, David Hopwood wrote: > Rob Schmersel wrote: > > Thinus Pollard wrote: > >>What's happing is that yaws is dropping the '+' character from our string > >> and inserting a space (verified as ascii code 32 (decimal)) in its > >> place. > > > > <snip some additonal info> > > > > This is correct behaviour and is due to url encoding, that is making a > > string safe to be send in a URL, One of the things that happens is that > > a space is translated into a '+' sign. Yaws 'decodes any string and thus > > transfers any '+' into a space. > > In order to have this working correctly you'll have to URL encode the > > string before you send it (the '+' sign will then be transferred into a > > %xx, which yaws will correctly decode) > > Alternatively, you could use the "modified Base64 for URL" encoding > mentioned at <http://en.wikipedia.org/wiki/Base64> instead of uuencode. -- Daniel H Schutte Mobile: +27 84 468 3138 Office: +27 11 675 2643 The more laws and order are made prominent, the more thieves and robbers there will be. -- Lao Tsu |
From: Claes W. <kl...@hy...> - 2005-09-26 13:20:11
Attachments:
klacke.vcf
|
Thinus Pollard wrote: > Hi there > > We're sending an http request to yaws. See (1). The request contains a string > called ussdString. This string is a uuencoded string > > ussdString="*116*111*AAEGAGQAABEzAABgNIUgAABRdNAABYcAAAAAAAAAgQ6+8zh0D9iBDr7zOHQP2A==1#" > > Yaws captures this data via script (2). > > In our logs we get the string as: > ussdString=\"*116*111*AAEGAGQAABEzAABgNIUgAABWEtAABYcAAAAAAAAAgQ6<space>8zh0D9iBDr7zOHQP2A==1#\" > > What's happing is that yaws is dropping the '+' character from our string and > inserting a space (verified as ascii code 32 (decimal)) in its place. > > The tcpdump confirms that the message is sent correctly to yaws. After getting > the data from the http request in yaws we get this character replacement. > > Is this a bug in yaws, or is the + character something special? It's the call to parse_post which does this. It's intended and highly desired behaviour. You're best workaround if you want to handle data which is not decode by yaws is to parse Arg#arg.clidata yourselves. All data, sent from the client is kept there, untouched. /klacke |
From: Thinus P. <th...@er...> - 2005-09-26 13:31:41
|
Thank you very much. This is what we were looking for. regards, Thinus On Monday 26 September 2005 15:19, Claes Wikstom wrote: > Thinus Pollard wrote: > > Hi there > > > > We're sending an http request to yaws. See (1). The request contains a > > string called ussdString. This string is a uuencoded string > > > > ussdString="*116*111*AAEGAGQAABEzAABgNIUgAABRdNAABYcAAAAAAAAAgQ6+8zh0D9iB > >Dr7zOHQP2A==1#" > > > > Yaws captures this data via script (2). > > > > In our logs we get the string as: > > ussdString=\"*116*111*AAEGAGQAABEzAABgNIUgAABWEtAABYcAAAAAAAAAgQ6<space>8 > >zh0D9iBDr7zOHQP2A==1#\" > > > > What's happing is that yaws is dropping the '+' character from our string > > and inserting a space (verified as ascii code 32 (decimal)) in its place. > > > > The tcpdump confirms that the message is sent correctly to yaws. After > > getting the data from the http request in yaws we get this character > > replacement. > > > > Is this a bug in yaws, or is the + character something special? > > It's the call to parse_post which does this. It's intended and highly > desired behaviour. You're best workaround if you want to handle data which > is not decode by yaws is to parse Arg#arg.clidata yourselves. All data, > sent from the client is kept there, untouched. > > /klacke -- Thinus Pollard Mobile: +27 72 075 2751 |