[Httplib2-discuss] =?WINDOWS-1252?Q?Re:_[Httplib2-discuss]_Re:_Bug_#1455955:_Follo?= =?WINDOWS-1252?Q?wup_=96_matching_"tokens"_in_regular_expressions?=

[Thomas B. <t.b...@gm...>]

2006/3/31, Joe Gregorio <joe...@gm...>:
> Thomas,
>   That's really great work on the regex, can you also add some
> unit tests that exercise the regex?

When running the existing unit tests with my modified regex, the test
"HttpPrivateTest.testParseWWWAuthenticateMultiple4" fails: the value
of the "qop" auth-param contains a tab (\t).

Since then, I assumed the headers had been normalized, while actually
_normalize_headers only normalizes field names. Precisely, HTTP/1.1
says that "A recipient MAY replace any linear white space with a
single SP before interpreting the field value or forwarding the
message downstream." I assumed it was done while actually it's not.

There are two options here:
 - unfold and normalize white space in _normalize_headers
 - unfold and normalize white space only in _parse_www_authenticate,
only for www-authenticate or authentication-info headers before
processing
I reject the option of modifying the regex to accomodate folded field
values: it's much easier normalizing the values before processing and
it's totally HTTP/1.1-compliant.

And as I was investigating in _parse_www_authentication, I also
noticed quoted pairs (in quoted strings) are never "unquoted", so the
following (new) unit test fails:
        res =3D httplib2._parse_www_authenticate({ 'www-authenticate':
'Test realm=3D"a \\"test\\" realm"'})
        self.assertEqual(res['test']['realm'], 'a "test" realm')
as res['test']['realm'] contains 'a \\"test\\" realm'.
This (unquoting) can be done using either a regex and the "sub"
method, or splitting and joining the string. I personnaly have no
preference.

Also (and finally), as strict WWW-Authenticate "parsing" might cause
unrecoverable errors (I mean, a parameter treated as an auth-scheme,
or consuming the following challenge, instead of exceptions), I tend
to go for the simpler regex ("strict send/lax receive") I provided in
my previous mail (it still needs some testing though). Or how about
putting both regexes in the code and providing a switch for the one to
use (e.g. "httplib2.USE_STRICT_WWW_AUTHENTICATE_PARSING =3D true",
defaulting to false)?

--
Thomas Broyer