Re: [TCLCORE] Possible issue with http 2.7+

[Alexandre F. <ale...@gm...>]

On Tue, Aug 18, 2015 at 4:24 PM, Pan Chai <Pan...@na...> wrote:
> Hello Tcl Core Team,
>
> I am having an issue with http 2.7+, with Tcl 8.5+.
>
> First, if you put this url string in browser (Chrome, firefox, IE, etc,):
> “http://skyview.gsfc.nasa.gov/cgi-bin/pskcall?VCOORD=ETA%20Carinae&SURVEY=DSS&SCOORD=Equatorial&SFACTR=Default&PIXELX=300&PIXELY=300”
>
> It will produce a data file (astronomical data file.) The data file is
> correct and being our test benchmark.
>
> But if you tried these commands under TCLSH:
>
> package require http
> set skyviewout [open “test.fits” w]
> set token [::http:: geturl “http://skyview.gsfc.nasa.gov/cgi-bin/pskcall”
> -channel $skyviewout -query
> “VCOORD=ETA%20Carinae&SURVEY=DSS&SCOORD=Equatorial&SFACTR=Default&PIXELX=300&PIXELY=300”]
> close $skyviewout
>
> It then produces a corrupted data file that has some in-deciphiable strings
> at beginning of the data file, a line consists of “400^M" and maybe
> somewhere else in the data file (like break line with “5000^M”) too.
>
> These TCLSH commands used to work fine. Just as soon as we upgrade to http
> package 2.7+, it suddenly produced strange results, and it is making reading
> those astronomical data files impossible.
>
> Can you help?

The issue is that when upgrading, the Tcl HTTP client selected
protocol version 1.1, which allowed the server to use chunked encoding
transfer. Those 400^M and 5000^M are chunk length prefixes.
Unfortunately, the chunk support in the HTTP package does not extend
to "-channel" mode; in this case the channel receives the
chunked-encoded flow unmodified.

As a workaround, avoid the -channel option; sure, the body will flow
into RAM instead, but will be correctly 'de-chunked':

% set token [::http::geturl
{http://skyview.gsfc.nasa.gov/cgi-bin/pskcall?VCOORD=ETA%20Carinae&SURVEY=DSS&SCOORD=Equatorial&SFACTR=Default&PIXELX=300&PIXELY=300}]
::http::1
% set x [http::data ::http::1];set y [string length $x]
374400
% string range $x 0 20
SIMPLE  =
%

(note also above that I didn't use -query, meaning a POST, since your
primary example in a browser is a GET)

Also note that when saving to  a binary file you might want to
[fconfigure -translation binary] to avoid unwanted translations.

Wrapping it up:

     package require http
     set token [::http::geturl
{http://skyview.gsfc.nasa.gov/cgi-bin/pskcall?VCOORD=ETA%20Carinae&SURVEY=DSS&SCOORD=Equatorial&SFACTR=Default&PIXELX=300&PIXELY=300}]
     set skyviewout [open “test.fits” w]
     fconfigure  $skyviewout -translation binary
     puts -nonewline $skyviewout [http::data $token]


-Alex