From: Charles B. <ch...@uc...> - 2010-04-15 19:55:40
|
A query string like this who=pacheco&what=&when= doesn't seem to get past the YAWS web server. Here is the trace output from curl: => Send header, 180 bytes (0xb4) 0000: 47 45 54 20 2f 64 65 73 74 72 65 7a 61 2f 63 6f GET /destreza/co 0010: 6e 6e 65 63 74 6f 72 2e 63 67 69 3f 77 68 6f 3d nnector.cgi?who= 0020: 5e 41 26 77 68 61 74 3d 26 77 68 65 6e 3d 20 48 ^A&what=&when= H 0030: 54 54 50 2f 31 2e 31 0d 0a 55 73 65 72 2d 41 67 TTP/1.1..User-Ag 0040: 65 6e 74 3a 20 63 75 72 6c 2f 37 2e 31 39 2e 35 ent: curl/7.19.5 0050: 20 28 69 33 38 36 2d 61 70 70 6c 65 2d 64 61 72 (i386-apple-dar 0060: 77 69 6e 39 2e 38 2e 30 29 20 6c 69 62 63 75 72 win9.8.0) libcur 0070: 6c 2f 37 2e 31 39 2e 35 20 7a 6c 69 62 2f 31 2e l/7.19.5 zlib/1. 0080: 32 2e 33 0d 0a 48 6f 73 74 3a 20 61 6e 2e 6c 69 2.3..Host: an.li 0090: 62 2e 75 63 68 69 63 61 67 6f 2e 65 64 75 3a 32 b.uchicago.edu:2 00a0: 30 31 30 0d 0a 41 63 63 65 70 74 3a 20 2a 2f 2a 010..Accept: */* 00b0: 0d 0a 0d 0a .... <= Recv header, 17 bytes (0x11) 0000: 48 54 54 50 2f 31 2e 31 20 32 30 30 20 4f 4b 0d HTTP/1.1 200 OK. 0010: 0a . <= Recv header, 42 bytes (0x2a) 0000: 53 65 72 76 65 72 3a 20 59 61 77 73 2f 31 2e 38 Server: Yaws/1.8 0010: 32 20 59 65 74 20 41 6e 6f 74 68 65 72 20 57 65 2 Yet Another We 0020: 62 20 53 65 72 76 65 72 0d 0a b Server.. <= Recv header, 37 bytes (0x25) 0000: 44 61 74 65 3a 20 54 68 75 2c 20 31 35 20 41 70 Date: Thu, 15 Ap 0010: 72 20 32 30 31 30 20 31 37 3a 35 32 3a 32 39 20 r 2010 17:52:29 0020: 47 4d 54 0d 0a GMT.. <= Recv header, 19 bytes (0x13) 0000: 43 6f 6e 74 65 6e 74 2d 4c 65 6e 67 74 68 3a 20 Content-Length: 0010: 30 0d 0a 0.. <= Recv header, 25 bytes (0x19) 0000: 43 6f 6e 74 65 6e 74 2d 54 79 70 65 3a 20 74 65 Content-Type: te 0010: 78 74 2f 68 74 6d 6c 0d 0a xt/html.. <= Recv header, 2 bytes (0x2) 0000: 0d 0a .. == Info: Connection #0 to host an.lib.uchicago.edu left intact == Info: Closing connection #0 If I fill in all the values, I get something like this: => Send header, 182 bytes (0xb6) 0000: 47 45 54 20 2f 64 65 73 74 72 65 7a 61 2f 63 6f GET /destreza/co 0010: 6e 6e 65 63 74 6f 72 2e 63 67 69 3f 77 68 6f 3d nnector.cgi?who= 0020: 65 74 74 65 6e 68 61 72 64 26 77 68 61 74 3d 64 ettenhard&what=d 0030: 65 73 74 72 65 7a 61 26 77 68 65 6e 3d 31 36 2e estreza&when=16. 0040: 2e 20 48 54 54 50 2f 31 2e 31 0d 0a 55 73 65 72 . HTTP/1.1..User 0050: 2d 41 67 65 6e 74 3a 20 63 75 72 6c 2f 37 2e 31 -Agent: curl/7.1 0060: 39 2e 35 20 28 69 33 38 36 2d 61 70 70 6c 65 2d 9.5 (i386-apple- 0070: 64 61 72 77 69 6e 39 2e 38 2e 30 29 20 6c 69 62 darwin9.8.0) lib 0080: 63 75 72 6c 2f 37 2e 31 39 2e 35 20 7a 6c 69 62 curl/7.19.5 zlib 0090: 2f 31 2e 32 2e 33 0d 0a 48 6f 73 74 3a 20 61 6e /1.2.3..Host: an 00a0: 3a 32 30 31 30 0d 0a 41 63 63 65 70 74 3a 20 2a :2010..Accept: * 00b0: 2f 2a 0d 0a 0d 0a /*.... 0000: 48 54 54 50 2f 31 2e 31 20 32 30 30 20 4f 4b 0d HTTP/1.1 200 OK. 0010: 0a . <= Recv header, 42 bytes (0x2a) 0000: 53 65 72 76 65 72 3a 20 59 61 77 73 2f 31 2e 38 Server: Yaws/1.8 0010: 32 20 59 65 74 20 41 6e 6f 74 68 65 72 20 57 65 2 Yet Another We 0020: 62 20 53 65 72 76 65 72 0d 0a b Server.. <= Recv header, 37 bytes (0x25) 0000: 44 61 74 65 3a 20 54 68 75 2c 20 31 35 20 41 70 Date: Thu, 15 Ap 0010: 72 20 32 30 31 30 20 31 37 3a 35 32 3a 30 33 20 r 2010 17:52:03 0020: 47 4d 54 0d 0a GMT.. <= Recv header, 40 bytes (0x28) 0000: 43 6f 6e 74 65 6e 74 2d 54 79 70 65 3a 20 74 65 Content-Type: te 0010: 78 74 2f 68 74 6d 6c 3b 20 63 68 61 72 73 65 74 xt/html; charset 0020: 3d 55 54 46 2d 38 0d 0a =UTF-8.. <= Recv header, 28 bytes (0x1c) 0000: 54 72 61 6e 73 66 65 72 2d 45 6e 63 6f 64 69 6e Transfer-Encodin 0010: 67 3a 20 63 68 75 6e 6b 65 64 0d 0a g: chunked.. <= Recv header, 2 bytes (0x2) 0000: 0d 0a .. <= Recv data, 5923 bytes (0x1723) 0000: 31 37 31 42 0d 0a 3c 68 74 6d 6c 20 78 6d 6c 6e 171B..<html xmln 0010: 73 3d 22 68 74 74 70 3a 2f 2f 77 77 77 2e 77 33 s="http://www.w3 0020: 2e 6f 72 67 2f 31 39 39 39 2f 78 68 74 6d 6c 22 .org/1999/xhtml" 0030: 20 78 6d 6c 3a 6c 61 6e 67 3d 22 65 6e 22 20 6c xml:lang="en" l 0040: 61 6e 67 3d 22 65 6e 22 3e 0a 3c 68 65 61 64 3e ang="en">.<head> 0050: 0a 3c 6d 65 74 61 20 68 74 74 70 2d 65 71 75 69 .<meta http-equi 0060: 76 3d 22 43 6f 6e 74 65 6e 74 2d 54 79 70 65 22 v="Content-Type" 0070: 20 63 6f 6e 74 65 6e 74 3d 22 74 65 78 74 2f 68 content="text/h 0080: 74 6d 6c 3b 20 63 68 61 72 73 65 74 3d 55 54 46 tml; charset=UTF 0090: 2d 38 22 20 2f 3e 0a 3c 74 69 74 6c 65 3e 48 69 -8" />.<title>Hi 00a0: 73 74 6f 72 69 63 61 6c 20 53 70 61 6e 69 73 68 storical Spanish 00b0: 20 46 65 6e 63 69 6e 67 20 54 72 65 61 74 69 73 Fencing Treatis 00c0: 65 73 3c 2f 74 69 74 6c 65 3e 0a 3c 6c 69 6e 6b es</title>.<link [etc.] In other words, the results come back as expected. If I run either query past an Apache web server, the results come back as expected in both cases. I have encountered this behavior with both 1.82 and 1.88. |
From: Steve V. <vi...@ie...> - 2010-04-15 22:15:52
|
On Thu, Apr 15, 2010 at 3:33 PM, Charles Blair <ch...@uc...> wrote: > A query string like this > > who=pacheco&what=&when= > > doesn't seem to get past the YAWS web server. Here is the trace output > from curl: I can't seem to reproduce this behavior. I sent a query string to a .yaws page that accessed it via the #url record, no problem, and also accessed the individual query string names and values via yaws_api:queryvar() successfully. The proper URI also shows up in HTTP trace headers from yaws if I turn on tracing. Maybe you can explain a little more what you're trying to do and how you're trying to do it? --steve |
From: Charles B. <ch...@uc...> - 2010-04-16 00:11:41
|
On Thu, Apr 15, 2010 at 06:15:43PM -0400, Steve Vinoski wrote: > Maybe you can explain a little more what you're trying to do and how > you're trying to do it? Here's a trace. The query strings are being passed to a cgi executable, "connector.cgi" in what follows. % curl "http://an:2010/destreza/connector.cgi?who=pacheco&what=&when=" CGI failure: {exit_status,1}% 1> *** CLI -> SRV *** New (nossl) connection from 128.135.53.253:61707 *** CLI -> SRV *** GET /destreza/connector.cgi?who=pacheco&what=&when= HTTP/1.1 Accept: */* Host: an:2010 User-Agent: curl/7.19.5 (i386-apple-darwin9.8.0) libcurl/7.19.5 zlib/1.2.3 connector.cgi: QUERY_STRING: getEnv: does not exist (no environment variable) *** SRV -> CLI *** HTTP/1.1 500 Internal Server Error Server: Yaws/1.88 Yet Another Web Server Date: Fri, 16 Apr 2010 00:02:08 GMT Content-Length: 28 Content-Type: text/html *** CLI -> SRV *** closed % curl "http://an:2010/destreza/connector.cgi?who=^A&what=&when=" CGI failure: {exit_status,1}% *** CLI -> SRV *** New (nossl) connection from 128.135.53.253:61721 *** CLI -> SRV *** GET /destreza/connector.cgi?who=^A&what=&when= *** HTTP/1.1 Accept: */* Host: an:2010 User-Agent: curl/7.19.5 *** (i386-apple-darwin9.8.0) libcurl/7.19.5 zlib/1.2.3 connector.cgi: QUERY_STRING: getEnv: does not exist (no environment variable) *** SRV -> CLI *** HTTP/1.1 500 Internal Server Error Server: Yaws/1.88 Yet Another Web Server Date: Fri, 16 Apr 2010 00:03:39 GMT Content-Length: 28 Content-Type: text/html *** CLI -> SRV *** closed % curl "http://an:2010/destreza/connector.cgi?who=ettenhard&what=destreza&when=16.." [This works.] *** CLI -> SRV *** New (nossl) connection from 128.135.53.253:61734 *** CLI -> SRV *** GET /destreza/connector.cgi?who=ettenhard&what=destreza&when=16.. HTTP/1.1 Accept: */* Host: an:2010 User-Agent: curl/7.19.5 (i386-apple-darwin9.8.0) libcurl/7.19.5 zlib/1.2.3 *** SRV -> CLI *** HTTP/1.1 200 OK Server: Yaws/1.88 Yet Another Web Server Date: Fri, 16 Apr 2010 00:04:39 GMT Content-Type: text/html; charset=UTF-8 Transfer-Encoding: chunked *** CLI -> SRV *** closed The commonality among the failures is "key=" or just "key" in the query string. If I trim them off, then these work fine: % curl "http://an:2010/destreza/connector.cgi?who=^A" % curl "http://an:2010/destreza/connector.cgi?who=pacheco" Thanks. |
From: Steve V. <vi...@ie...> - 2010-04-16 02:18:55
|
On Thu, Apr 15, 2010 at 8:13 PM, Charles Blair <ch...@uc...> wrote: > On Thu, Apr 15, 2010 at 06:15:43PM -0400, Steve Vinoski wrote: >> Maybe you can explain a little more what you're trying to do and how >> you're trying to do it? > > Here's a trace. The query strings are being passed to a cgi > executable, "connector.cgi" in what follows. Cool, I've now duplicated what you're seeing. Unfortunately it appears to have nothing to do with yaws -- it looks like a bug in the Erlang open_port bif. I'll see if I can track it down in the C code and submit a patch. --steve |
From: Steve V. <vi...@ie...> - 2010-04-16 07:05:23
|
On Thu, Apr 15, 2010 at 10:18 PM, Steve Vinoski <vi...@ie...> wrote: > On Thu, Apr 15, 2010 at 8:13 PM, Charles Blair <ch...@uc...> wrote: >> On Thu, Apr 15, 2010 at 06:15:43PM -0400, Steve Vinoski wrote: >>> Maybe you can explain a little more what you're trying to do and how >>> you're trying to do it? >> >> Here's a trace. The query strings are being passed to a cgi >> executable, "connector.cgi" in what follows. > > Cool, I've now duplicated what you're seeing. Unfortunately it appears > to have nothing to do with yaws -- it looks like a bug in the Erlang > open_port bif. I'll see if I can track it down in the C code and > submit a patch. The bug is definitely in Erlang. I have a patch that fixes it but I have to do some unit test work before I can submit it. But I also patched Yaws to work around the problem, since obviously not everyone can just update their Erlang installations. The problem occurs when the query string ends in an equal sign. The erlang:open_port call Yaws makes for CGI sets QUERY_STRING as an environment variable, so in this case it passes a string like this: QUERY_STRING=who=charles&when=&what= Some C code deep inside Erlang checks this string, sees the equal sign at the end, and assumes it's a string of the form EnvName= which it interprets to mean that "EnvName" should be deleted from the environment since it apparently has no value. This results in QUERY_STRING never getting added to the environment. Yaws now works around this situation by checking the CGI query string to see if it ends in an equal sign, and tacking an extra ampersand on the end if it does. The extra ampersand is harmless but it avoids the Erlang bug. You can get the patched version from github.com, commit ade3631. --steve |
From: Charles B. <ch...@uc...> - 2010-04-16 14:15:23
|
On Fri, Apr 16, 2010 at 03:05:16AM -0400, Steve Vinoski wrote: > The problem occurs when the query string ends in an equal sign. The > erlang:open_port call Yaws makes for CGI sets QUERY_STRING as an > environment variable, so in this case it passes a string like this: > > QUERY_STRING=who=charles&when=&what= Just to be clear, I can also replicate the problem when the query string does not end in an equals sign, for example: QUERY_STRING=who=charles&when=&what Perhaps that also triggers the same underlying code in the same way, but any fix to the Erlang routine should be tested against that as well. Thanks for looking into this. |
From: Steve V. <vi...@ie...> - 2010-04-16 14:34:37
|
On Fri, Apr 16, 2010 at 10:17 AM, Charles Blair <ch...@uc...> wrote: > On Fri, Apr 16, 2010 at 03:05:16AM -0400, Steve Vinoski wrote: >> The problem occurs when the query string ends in an equal sign. The >> erlang:open_port call Yaws makes for CGI sets QUERY_STRING as an >> environment variable, so in this case it passes a string like this: >> >> QUERY_STRING=who=charles&when=&what= > > Just to be clear, I can also replicate the problem when the query > string does not end in an equals sign, for example: > > QUERY_STRING=who=charles&when=&what > > Perhaps that also triggers the same underlying code in the same way, > but any fix to the Erlang routine should be tested against that as > well. I can't reproduce this one. I just tried it against a stock Yaws 1.88 running on R13B04 and it works correctly. I tried a CGI script, a .yaws page, and ran my test against Erlang by itself that shows the bug I found last night, and all three work as expected. The problematic code in Erlang is dealing specifically with the '=' character used to separate env var name from value, and is specifically checking for it at the end of the string, which is why having it at the end caused problems. That's not the case here, so I would expect it to work, and it does indeed work for me. --steve |
From: Charles B. <ch...@uc...> - 2010-04-16 14:37:58
|
On Fri, Apr 16, 2010 at 10:34:31AM -0400, Steve Vinoski wrote: > I can't reproduce this one. I just tried it against a stock Yaws 1.88 > running on R13B04 I'm running R13B01; I'll upgrade and see what happens. |