Menu
▾
▴
naviserver-devel
|
From: Zoran V. <zv...@ar...> - 2006-08-13 15:06:18
|
Hi!
Please(!) update ChangeLog when commiting new changes.
There is a whole lotta new errors in the test suite
in the form of:
---- Result was:
200 {} {1
x
0
}
---- Result should have been (exact matching):
200 1 x
==== tclresp-4.2 FAILED
Now, what IS this? I cannot remember seeing those
errors couple of days ago when I last commited.
Since then I saw no entries in the ChangeLog so
I wonder where is this comming from???
Cheers
Zoran
|
|
From: Vlad S. <vl...@cr...> - 2006-08-13 15:21:55
|
i am suspecting this happened after encoding patches
Zoran Vasiljevic wrote:
> Hi!
>
> Please(!) update ChangeLog when commiting new changes.
>
> There is a whole lotta new errors in the test suite
> in the form of:
>
> ---- Result was:
> 200 {} {1
> x
> 0
>
> }
> ---- Result should have been (exact matching):
> 200 1 x
> ==== tclresp-4.2 FAILED
>
> Now, what IS this? I cannot remember seeing those
> errors couple of days ago when I last commited.
> Since then I saw no entries in the ChangeLog so
> I wonder where is this comming from???
>
> Cheers
> Zoran
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
>
|
|
From: Zoran V. <zv...@ar...> - 2006-08-13 15:24:10
|
On 13.08.2006, at 17:21, Vlad Seryakov wrote: > i am suspecting this happened after encoding patches WHAT encoding patches??? I see nothing in the ChangeLog! Zoran |
|
From: Vlad S. <vl...@cr...> - 2006-08-13 15:28:26
|
I remember my last chnage in keepalive but i am sure i ran make test at that time, it was month ago, after that not sure about other changes really, not many last time Zoran Vasiljevic wrote: > On 13.08.2006, at 17:21, Vlad Seryakov wrote: > > >> i am suspecting this happened after encoding patches >> > > WHAT encoding patches??? I see nothing in the ChangeLog! > Zoran > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > naviserver-devel mailing list > nav...@li... > https://lists.sourceforge.net/lists/listinfo/naviserver-devel > > |
|
From: Zoran V. <zv...@ar...> - 2006-08-13 15:36:41
|
On 13.08.2006, at 17:28, Vlad Seryakov wrote: > I remember my last chnage in keepalive but i am sure i ran make > test at > that time, it was month ago, after that not sure about other changes > really, not many last time Your last change was: 2006-07-21 Vlad Seryakov <ser...@us...> * tcl/file.tcl: Use wrapped Tcl proc aroun d.tcl file instead of not exactly as needed global ns_cache. Since then it was only me fiddling with the ns_log and ns_proxy arround. But, I'm sure I did not get thos errors some times ago, actually at after 2006-08-04 as this was the last time I checked in something, NOT counting the last nsproxylib.c change which is totally unrelated to the core. That means: somebody did change the code (or the testsuite) w/o updating the ChangeLog file. I think I will have to go to the mail list archive and check the commit maillist as I did not get any emails about what was changed since.... I would ask the commiter (we will found who that is soon) to always update ChangeLog file so we know what is going on. Otherwise it is just a waste of time and nerves... Cheers Zoran > > Zoran Vasiljevic wrote: >> On 13.08.2006, at 17:21, Vlad Seryakov wrote: >> >> >>> i am suspecting this happened after encoding patches >>> >> >> WHAT encoding patches??? I see nothing in the ChangeLog! >> Zoran >> >> --------------------------------------------------------------------- >> ---- >> Using Tomcat but need to do more? Need to support web services, >> security? >> Get stuff done quickly with pre-integrated technology to make your >> job easier >> Download IBM WebSphere Application Server v.1.0.1 based on Apache >> Geronimo >> http://sel.as-us.falkag.net/sel? >> cmd=lnk&kid=120709&bid=263057&dat=121642 >> _______________________________________________ >> naviserver-devel mailing list >> nav...@li... >> https://lists.sourceforge.net/lists/listinfo/naviserver-devel >> >> > > > ---------------------------------------------------------------------- > --- > Using Tomcat but need to do more? Need to support web services, > security? > Get stuff done quickly with pre-integrated technology to make your > job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache > Geronimo > http://sel.as-us.falkag.net/sel? > cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > naviserver-devel mailing list > nav...@li... > https://lists.sourceforge.net/lists/listinfo/naviserver-devel |
|
From: Vlad S. <vl...@cr...> - 2006-08-13 17:01:25
|
i think i found it, it is the change about switching into chunked mode if encoding is used, thus returning the result with length prepended, so all test are broken now. when i disabled it in the sources, some tests are fine now, http.test, but still others are broken Zoran Vasiljevic wrote: > On 13.08.2006, at 17:28, Vlad Seryakov wrote: > > >> I remember my last chnage in keepalive but i am sure i ran make >> test at >> that time, it was month ago, after that not sure about other changes >> really, not many last time >> > > Your last change was: > > 2006-07-21 Vlad Seryakov <ser...@us...> > > * tcl/file.tcl: Use wrapped Tcl proc aroun d.tcl file instead of > not exactly as needed global ns_cache. > > Since then it was only me fiddling with the ns_log and ns_proxy > arround. But, I'm sure I did not get thos errors some times > ago, actually at after 2006-08-04 as this was the last time > I checked in something, NOT counting the last nsproxylib.c change > which is totally unrelated to the core. > > That means: somebody did change the code (or the testsuite) > w/o updating the ChangeLog file. I think I will have to go > to the mail list archive and check the commit maillist as I > did not get any emails about what was changed since.... > > I would ask the commiter (we will found who that is soon) to always > update ChangeLog file so we know what is going on. Otherwise it > is just a waste of time and nerves... > > Cheers > Zoran > > >> Zoran Vasiljevic wrote: >> >>> On 13.08.2006, at 17:21, Vlad Seryakov wrote: >>> >>> >>> >>>> i am suspecting this happened after encoding patches >>>> >>>> >>> WHAT encoding patches??? I see nothing in the ChangeLog! >>> Zoran >>> >>> --------------------------------------------------------------------- >>> ---- >>> Using Tomcat but need to do more? Need to support web services, >>> security? >>> Get stuff done quickly with pre-integrated technology to make your >>> job easier >>> Download IBM WebSphere Application Server v.1.0.1 based on Apache >>> Geronimo >>> http://sel.as-us.falkag.net/sel? >>> cmd=lnk&kid=120709&bid=263057&dat=121642 >>> _______________________________________________ >>> naviserver-devel mailing list >>> nav...@li... >>> https://lists.sourceforge.net/lists/listinfo/naviserver-devel >>> >>> >>> >> ---------------------------------------------------------------------- >> --- >> Using Tomcat but need to do more? Need to support web services, >> security? >> Get stuff done quickly with pre-integrated technology to make your >> job easier >> Download IBM WebSphere Application Server v.1.0.1 based on Apache >> Geronimo >> http://sel.as-us.falkag.net/sel? >> cmd=lnk&kid=120709&bid=263057&dat=121642 >> _______________________________________________ >> naviserver-devel mailing list >> nav...@li... >> https://lists.sourceforge.net/lists/listinfo/naviserver-devel >> > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > naviserver-devel mailing list > nav...@li... > https://lists.sourceforge.net/lists/listinfo/naviserver-devel > > |
|
From: Zoran V. <zv...@ar...> - 2006-08-13 17:09:06
|
On 13.08.2006, at 17:55, Vlad Seryakov wrote: > i think i found it, it is the change about switching into chunked mode > if encoding is used, thus returning the result with length > prepended, so > all test are broken now. > > when i disabled it in the sources, some tests are fine now, http.test, > but still others are broken As I see from the CVS, there were no changes to the C-code there? Did you have to change the C-code?? I guess, Bernd will have to explain what he ment to do with that... Cheers, Zoran |
|
From: Michael L. <mic...@gm...> - 2006-08-13 17:38:28
|
> As I see from the CVS, there were no changes to the C-code there? > Did you have to change the C-code?? > I guess, Bernd will have to explain what he ment to do with that... Bernd is on holiday right now. But i think i can explain the problem. Bernd submitted a patch (or better workaround) to prevent Naviserver from sending wrong content-length-headers. He simply made naviserver send all data, that had to be converted to a different encoding, in chunked transfer-encoding. Unfortunately nstest_http doesn't understand chunked content. This patch was comitted on July, 13th. When I worked on the encoding-tests, I had to add an OutputCharset configuration parameter. So now Naviserver has to convert the content and sends it in chunked mode, which nstest_http doesn't understand. And the tests fail. If you remove this workaround, some of my new encoding-tests still fail because of a wrong content-length. But this is really a bug. Michael |
|
From: Zoran V. <zv...@ar...> - 2006-08-13 17:41:52
|
On 13.08.2006, at 19:38, Michael Lex wrote: >> As I see from the CVS, there were no changes to the C-code there? >> Did you have to change the C-code?? >> I guess, Bernd will have to explain what he ment to do with that... > > Bernd is on holiday right now. But i think i can explain the problem. > Bernd submitted a patch (or better workaround) to prevent Naviserver > from sending wrong content-length-headers. He simply made naviserver > send all data, that had to be converted to a different encoding, in > chunked transfer-encoding. Unfortunately nstest_http doesn't > understand chunked content. This patch was comitted on July, 13th. > When I worked on the encoding-tests, I had to add an OutputCharset > configuration parameter. So now Naviserver has to convert the content > and sends it in chunked mode, which nstest_http doesn't understand. > And the tests fail. If you remove this workaround, some of my new > encoding-tests still fail because of a wrong content-length. But this > is really a bug. Michael, I do not want to remove anything if this breaks anybody's code. I just want to understand what is going on and how is this going to hit me (or not) if I start to use this code? Do I have to know somethind special? We do not set any special encodings: we always serve utf8. Should I care? Cheers, Zoran > > Michael > > ---------------------------------------------------------------------- > --- > Using Tomcat but need to do more? Need to support web services, > security? > Get stuff done quickly with pre-integrated technology to make your > job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache > Geronimo > http://sel.as-us.falkag.net/sel? > cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > naviserver-devel mailing list > nav...@li... > https://lists.sourceforge.net/lists/listinfo/naviserver-devel |
|
From: Michael L. <mic...@gm...> - 2006-08-13 18:03:48
|
> I do not want to remove anything if this breaks anybody's code. This chunked-transfer-encoding thing is only a workaround until the Content-Length bug is fixed. > I just want to understand what is going on and how is this > going to hit me (or not) if I start to use this code? > Do I have to know somethind special? We do not set any special > encodings: we always serve utf8. Should I care? It's only going to hit you if you use some other encoding than utf-8 and then only if you use a HTTP-client, that does not understand chunked encoding. But every web-browser and virtually every other client-software should understand this (It's a MUST part of the rfc). Michael |
|
From: Zoran V. <zv...@ar...> - 2006-08-14 10:00:26
|
On 13.08.2006, at 20:03, Michael Lex wrote: > > This chunked-transfer-encoding thing is only a workaround until the > Content-Length bug is fixed. I've been reading various posts from you, Bernd, Vlad and Stephen about that... I must say: the problem is that the *correct* content-length header is/would-be difficult (or costly in terms of memory and time) to compute for dynamic content. The way how the things are handled now, the caller passes the equivalent to [string length] bytes and an encoding to the underlying routine(s). They encode the content *on-the-fly* so the caller cannot possibly know in advance what to put in the Content-Length header as the *real* output length may/will change depending on the selected encoding. After all, the chunked encoding IS defined exactly for this reason: to allow the recepient to verify the number of bytes received, when it is impossible or not feasible for the generator to specify exact number of bytes sent. If you looking it from that perspective, the "workaround" Bernd has made is actually pretty much OK. I wonder how you can set a correct Content-Length if you use ADP streaming... The above case (using explicit output encoding) is (admitently) not exactly the same as streaming, as you COULD encode the string before sending and then set the correct content-length, but this would mean memory bloat as you will have to transform the entire content beforehand... Therefore, if you ask me, the "workaround" should be left there. Another thing... I guess if we do not return the Content-Length at all, and we do not use chunked transfer encoding, the browser will slurp everything until EOF, right? But this can only be done for non-keepalive connections... Hm... Wrong turn.... If you ask me, I think we must make sure our test routines also know how to decode chunked encodings and we should leave the "workaround" in place. If we all agree that this is the way to go we can check other places in the code and make sure they do the same: if output encoding set, turn-on the chunked transfer encoding. What do others think? Cheers Zoran |
|
From: Zoran V. <zv...@ar...> - 2006-08-14 10:19:10
|
On 14.08.2006, at 12:00, Zoran Vasiljevic wrote: > I must say: the problem is that the *correct* content-length > header is/would-be difficult (or costly in terms of memory > and time) to compute for dynamic content. Hm... I was too fast, as usual. If I read Bernds email *carefully*, he says: <quote> In my test case, 'string length' on the parsed adp string gives me 7109 bytes, 'string bytelength' 7147 bytes, in the Header 'Content-Length' is 7147 and wget stops after byte 7109 (e.g. Opera requests the page twice, haha, I lost one day to figure out why): string length: 7109 bytes = bytes returned string bytelength: 7147 = Content-Length header </quote> This would mean that the Content-Length is actually defined as number of characters in the given content encoding and NOT number of bytes! This is of course something completely different.... Putting the streamin mode aside, the caller will usually know how many characters he's returning. He may not know how many bytes this will yield in the given encodong allright but this is not important then, as we will do this correctly during the send. So the "bug" you are reffering to is us setting the content-length on the basis of [string bytelength] instead on the basis of [string length] equivalent? If this is so, then the bug should be of course fixed and the "workaround" should be removed. That would leave us to turn on the chunked encoding ONLY when we serve streaming content. Do I see this right? Cheers Zoran |
|
From: Michael L. <mic...@gm...> - 2006-08-14 10:53:26
|
I think you get Bernd wrong: The problem was, that Bernd wanted naviserver to return the content in iso-8859-1 encoding. So the number of bytes and the number of characters should be equal. The Content-Length has to be the number of bytes returned, but naviserver computed the value with string bytelength of an utf-8 string, which was, in Bernds, case greater than the bytelength of the iso8859-1 string. So it seems that chunked encoding is the best possible solution. But as Gustav said, chunked transfer-encoding is only part of HTTP/1.0 and some clients don't understand it. Btw: Aolserver doesn't encode "on-the-fly", but in memory. So they know the content-length before the content is sent to the recipient. Michael |
|
From: Zoran V. <zv...@ar...> - 2006-08-14 11:52:00
|
On 14.08.2006, at 12:53, Michael Lex wrote:
> I think you get Bernd wrong: The problem was, that Bernd wanted
> naviserver to return the content in iso-8859-1 encoding. So the number
> of bytes and the number of characters should be equal.
> The Content-Length has to be the number of bytes returned, but =20
> naviserver
> computed the value with string bytelength of an utf-8 string, which
> was, in Bernds, case greater than the bytelength of the iso8859-1
> string.
I believe the best way is to peek at the standard (RFC 2616):
14.13 Content-Length
The Content-Length entity-header field indicates the size of the
entity-body, in decimal number of OCTETs, sent to the recipient or,
in the case of the HEAD method, the size of the entity-body that
would have been sent had the request been a GET.
Content-Length =3D "Content-Length" ":" 1*DIGIT
An example is
Content-Length: 3495
Applications SHOULD use this field to indicate the transfer-=20
length of
the message-body, unless this is prohibited by the rules in section
4.4.
This all means that content-length gives total number of *bytes*
in the response, regardless of any encoding applied. This also
means that in the case of UTF8 encoded string "m=FC" it will be 3
and not 2. If the "m=FC" is sent with ISO8859-1 then the content
length wold be 2. Allright. I think I get it now.
If this is so, then this means that we cannot possibly give the
correct content-length UNLESS we apply the encoding BEFORE sending
any headers and body, as we would have to either give the correct
value in content-length header OR would need to OMIT the content-length
and turn off the keepalive for that response.
>
> So it seems that chunked encoding is the best possible solution. But
> as Gustav said, chunked transfer-encoding is only part of HTTP/1.0 and
> some clients don't understand it.
Yes, chunked encoding seems feasible there. For clients not supporting
the chunked responses, we could convert the entire message beforehand
burning some memory and cycles. As there are quite a few of them out
there, this may not be of much importance anyways.
OK, this makes sense.
>
> Btw: Aolserver doesn't encode "on-the-fly", but in memory. So they
> know the content-length before the content is sent to the recipient.
>
On the fly I mean that the message is not encoded in its *entirety*
beforehand, rather it is converted piece-by-piece (hence on-the-fly)
in Ns_ConnWriteVChars().
So, what do we have now?
A. For HTTP 1.0 clients only, we could/should/must either:
a. omit content-length and turn keepalive off leaving
the browser to drain the connection until EOF.
b. calculate the content-length in advance by
performing the conversion of the message
in its entirety in the memory using the given
output encoding
B. For HTTP 1.1 clients we can turn on chunked encoding
if the output encoding is specified, and is not UTF8
(basically, this is what Bernd's workaround does).
Is this right? Are there any other options we may have?
Zoran
> Michael
>
> ----------------------------------------------------------------------=20=
> ---
> Using Tomcat but need to do more? Need to support web services, =20
> security?
> Get stuff done quickly with pre-integrated technology to make your =20
> job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache =20
> Geronimo
> http://sel.as-us.falkag.net/sel?=20
> cmd=3Dlnk&kid=3D120709&bid=3D263057&dat=3D121642
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
|
|
From: Gustaf N. <ne...@wu...> - 2006-08-14 12:24:40
|
Zoran Vasiljevic schrieb: > > On the fly I mean that the message is not encoded in its *entirety* > beforehand, rather it is converted piece-by-piece (hence on-the-fly) > in Ns_ConnWriteVChars(). > there are not much options, when the output encoding changes length. > So, what do we have now? > > A. For HTTP 1.0 clients only, we could/should/must either: > > a. omit content-length and turn keepalive off leaving > the browser to drain the connection until EOF. > HTTP/1.0 does not say anything about keepalive (http://www.ietf.org/rfc/rfc1945.txt) but sending a "connection: close" does not hurt, since some nonstandard clients use it. > b. calculate the content-length in advance by > performing the conversion of the message > in its entirety in the memory using the given > output encoding > the calculation is only needed in cases, where the output is not raw (e.g. delivering images, i would not call it UTF8 encoded (see below), rather "raw") > B. For HTTP 1.1 clients we can turn on chunked encoding > if the output encoding is specified, and is not UTF8 > (basically, this is what Bernd's workaround does). > ... or there is for some other reasons no translation going on (see above) Without looking into the code, i would assume that the simplest case would be to distinguish between cases, where the content-length is unknown and known. unknown means: transformation or dynamic content. for unknown cases i would omit in HTTP/1.0 the content-length and use in HTTP/1.1 the chunked encoding the known-case is a non-brainer. -gustaf > |
|
From: Zoran V. <zv...@ar...> - 2006-08-14 12:47:26
|
On 14.08.2006, at 14:24, Gustaf Neumann wrote: > Without looking into the code, i would assume that the simplest case > would be to distinguish between cases, where the content-length is > unknown and known. > > unknown means: transformation or dynamic content. > > for unknown cases i would omit in HTTP/1.0 the content-length and use > in HTTP/1.1 the chunked encoding This also makes sense as it is most trivial to implement (less work, more fun), and we basically already have it that way (more or less). The only trouble is the ton of error messages in the test suite, as we alone cannot handle chunked encoding as consumers! I yet have to see if [ns_http] can do that (I don't think so). Cheers, Zoran |
|
From: Gustaf N. <ne...@wu...> - 2006-08-14 13:57:33
|
Zoran Vasiljevic schrieb: > The only trouble is the ton of error messages in the > test suite, as we alone cannot handle chunked encoding > as consumers! I yet have to see if [ns_http] can do that > (I don't think so). > well, implementing chunked encoding in tcl is no rocket science (but work). however, this might be useful for other purposes as well. The xotcl HTTP client library has chunked encoding implemented (but has a different interace than ns_http). One could use this, or snarf the code. I am sure, there are other implementations as well. -gustaf |
|
From: Bernd E. <eid...@we...> - 2006-08-21 07:20:33
|
> I would ask the commiter (we will found who that is soon) to always > update ChangeLog file so we know what is going on. Otherwise it > is just a waste of time and nerves... Sorry for not updating the ChangeLog! :-( Guess I should consider the medical usage part of http://en.wikipedia.org/wiki/Ginkgo Bernd. |
|
From: Andrew P. <at...@pi...> - 2006-08-13 16:04:13
|
On Sun, Aug 13, 2006 at 05:36:33PM +0200, Zoran Vasiljevic wrote: > That means: somebody did change the code (or the testsuite) > w/o updating the ChangeLog file. I think I will have to go > to the mail list archive and check the commit maillist as I > did not get any emails about what was changed since.... But Zoran, CVS is the master copy of such information, so ask CVS. The "cvs2cl.pl" script is useful for getting a by-date changelog style list of all commits: http://www.red-bean.com/cvs2cl/ Using it shows that between 7/21 and 8/13, there was only one commit to Naviserver (ignoring its various modules) by anyone other than "vasiljevic", this one: 2006-08-08 12:07 eide * tests/: encoding.test, tclresp.test, test.nscfg, testserver/modules/testhttp.tcl, testserver/pages/encoding.iso2iso_adp, testserver/pages/encoding.iso2utf_adp, testserver/pages/encoding.iso_adp, testserver/pages/encoding.utf2iso_adp, testserver/pages/encoding.utf2utf_adp, testserver/pages/encoding.utf_adp: * tests/testserver/pages/encoding.*: Various helper files. * tests/testserver/modules/testhttp.tcl: nstest_http is now encoding aware. * tests/test.nscfg: Sections for mimetypes and encodings added, along with parameters URLCharset, OutputCharset and HackContentType. * tests/encoding.test: tests for ns_return and ns_write, changing charsets and verifying URL and ADP encoding. * tests/tclresp.test: minor modification in tests 3.1 and 4.2 after change in test.nscfg. Thanks to Michael Lex for writing the tests. (Unfortunately the ancient cvs2cl.pl version 1.2 I'm using tends to mis-format the the cvs commit messages.) -- Andrew Piskorski <at...@pi...> http://www.piskorski.com/ |
|
From: Zoran V. <zv...@ar...> - 2006-08-13 16:15:25
|
On 13.08.2006, at 18:04, Andrew Piskorski wrote: > > Using it shows that between 7/21 and 8/13, there was only one commit > to Naviserver (ignoring its various modules) by anyone other than > "vasiljevic", this one: > > 2006-08-08 12:07 eide > Ah, Bernd! > * tests/: encoding.test, tclresp.test, test.nscfg, > testserver/modules/testhttp.tcl, > testserver/pages/encoding.iso2iso_adp, > testserver/pages/encoding.iso2utf_adp, > testserver/pages/encoding.iso_adp, > testserver/pages/encoding.utf2iso_adp, > testserver/pages/encoding.utf2utf_adp, > testserver/pages/encoding.utf_adp: * > tests/testserver/pages/encoding.*: Various helper files. * > tests/testserver/modules/testhttp.tcl: nstest_http is now encoding > aware. * tests/test.nscfg: Sections for mimetypes and encodings > added, along with parameters URLCharset, OutputCharset and > HackContentType. * tests/encoding.test: tests for ns_return and > ns_write, changing charsets and verifying URL and ADP encoding. * > tests/tclresp.test: minor modification in tests 3.1 and 4.2 after > change in test.nscfg. Thanks to Michael Lex for writing the > tests. Thanks for getting this one! So, whatever this means: "nstest_http is now encoding aware" it either shows that it is (itself) broken or that our core code is broken :-( Bernd, which one of those it is? Cheers Zoran |
|
From: Gustaf N. <ne...@wu...> - 2006-08-13 19:44:52
|
Michael Lex schrieb: > It's only going to hit you if you use some other encoding than utf-8 > and then only if you use a HTTP-client, that does not understand > chunked encoding. But every web-browser and virtually every other > client-software should understand this (It's a MUST part of the rfc). > ... to be more precise, of HTTP/1.1, not HTTP/1.0 do you have any statistics, what your clients use? we have still a lot of HTTP/1.0 requests (around 5%). we would not use this currently on our production server. -gustaf |
|
From: Bernd E. <eid...@we...> - 2006-08-21 07:34:23
|
Am Sonntag, 13. August 2006 21:44 schrieb Gustaf Neumann: > Michael Lex schrieb: > > It's only going to hit you if you use some other encoding than utf-8 > > and then only if you use a HTTP-client, that does not understand > > chunked encoding. But every web-browser and virtually every other > > client-software should understand this (It's a MUST part of the rfc). > > ... to be more precise, of HTTP/1.1, not HTTP/1.0 > > do you have any statistics, what your clients use? we have still a lot > of HTTP/1.0 > requests (around 5%). On one customer server with about 13 Million requests this month I have also 5,17% of http/1.0 requests... This one will be replaced at the end of the year with naviserver, it still runs AS. Bernd. |
|
From: Zoran V. <zv...@ar...> - 2006-08-14 14:13:34
|
On 14.08.2006, at 15:58, Gustaf Neumann wrote: > no rocket science (but work) This is precisely what I was afraid of: work. Not that I don't like, I just do not have time for that now. Moreover, the ns_http is a C-level implementation, which means some more work and much more instability as when you screw something at that level, everything goes down the drain. At the first glance, the nstest_http from the test-suite needs to be rewritten but as this is entirely written in tcl and affects only the test suite, this is less of a problem. More problematic is ns_http as this is a public call and is in the core server. This might affect everybody, so the stability is important. OTOH, this need not be done to get the test-suite pass. As I know Bernd is very fluent in Tcl and as he's now on the holiday (means he has a chance to relax and recover) I vote him to fix the test suite and rewrite the nstest_http :-) Cheers, zoran |
|
From: Stephen D. <sd...@gm...> - 2006-08-14 20:43:10
|
On 8/14/06, Zoran Vasiljevic <zv...@ar...> wrote: > > On 14.08.2006, at 15:58, Gustaf Neumann wrote: > > > no rocket science (but work) > > This is precisely what I was afraid of: work. > Not that I don't like, I just do not have time > for that now. > > Moreover, the ns_http is a C-level implementation, > which means some more work and much more instability > as when you screw something at that level, everything > goes down the drain. > > At the first glance, the nstest_http from the test-suite > needs to be rewritten but as this is entirely written in > tcl and affects only the test suite, this is less of > a problem. > > More problematic is ns_http as this is a public call and > is in the core server. This might affect everybody, so the > stability is important. OTOH, this need not be done to get > the test-suite pass. > > As I know Bernd is very fluent in Tcl and as he's now > on the holiday (means he has a chance to relax and recover) > I vote him to fix the test suite and rewrite the nstest_http :-) > Hello all.. Thanks Michael and Bernd for writing the new encoding tests! I actually picked them up on Friday and have been looking through the code this weekend. So with it all fresh in my mind, it was painful to catch up with the mailing list and see y'all struggle through it! I learnt a new phrase: Yak Shaving. You know when you start something, and them you realise it depends on something else so you need to do that first, and then you discover some other thing that *that* depends on so you start to work on that, and then another thing, and so on...? I've been Yak Shaving. The content-length bug is my fault. I busted it with the vectored IO changes, and it didn't come to light because we weren't testing character set conversion. Vlad suggested the chunked-encoding patch, and Bernd applied it (forgot the ChageLog, oops), but the tests didn't start failing until Michael added the encoding tests which changed the default output character set to iso-8859-1. I'd like to roll back that change. IIRC, the vectored IO changes are new since the last release so the existing tarball should be OK. It's also a one line patch so it's easy to apply if for some reason your dependent on HEAD. It does mess up a whole bunch of other tests and it's not the right way to fix this problem. I don't think nstest_http needs to be chunked encoding aware. There's no way a 5 byte response needs to be chunked! Besides, it's just a simple testing harness that needs to be somewhat not-conformant. I wrote it originally, rathern than use an existing client, because I needed to inject faults and see the server's response. Anyway. Encoding -- it's a bitch. As far as I can tell there's 3 situations you can find yourself in: * All your clients understand utf-8. You can serve them in whatever language you want using the one encoding. This should (be made to) work by default. * Your clients do not all understand utf-8, but you only support one other language. In this case, you can change the server default output encoding, and all should again, work by defaul (ignoring the current bugs). * Your clients don't understand utf-8, and you serve content in multiple languages which don't share a common character set. Sucks to be you. Rob Mayoff wrote the original encoding patches and wrote this helpful document to describe the situation: http://dqd.com/~mayoff/encoding-doc.html This was turn of the century and I'm wondering how much of this still applies today. How many clients do not support utf-8? I've been working on some patches to fix the various bugs so don't worry about it too much. But I'd appreciate feedback on how you actually use the encoding support. I'd explain more but I'm getting kicked off the wi-fi... Tomorrow. |
|
From: Zoran V. <zv...@ar...> - 2006-08-14 21:16:50
|
On 14.08.2006, at 22:43, Stephen Deasey wrote: > > * Your clients don't understand utf-8, and you serve content in > multiple languages which don't share a common character set. Sucks to > be you. > I think the whole purpose of that encooding mess is this above. With time this will be less and less important, so how much should we really care? From the technical perspective, it is nice to have universal and general sulution, but from the practical side: it costs time and money to keep it arround... > > I've been working on some patches to fix the various bugs so don't > worry about it too much. But I'd appreciate feedback on how you > actually use the encoding support. I use it this way: leave everything as-is. I never had to tweak any of the existing encoding (encoding) knobs. And I never had anybody complaining. And we do serve japanese, chinese and european languages. Allright, the client is always either IE or Mozilla or Safari (prerequisite) so mine is perhaps not a good example. Apropos chunked encoding: I still believe that the vectorized IO is OK and the way you transform UTF8 on the fly is also OK. So, if any content encoding has to take place, you can really only do it with the chunked encoding OR by converting the whole content in memory prior sending it, and giving the correct content length OR by just omitting the content length all together. I do not think there are other options. I'm curious what you will come up with ;-) Cheers Zoran |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X