Re: [naviserver-devel] lurking bugs: file name encoding

[Stephen D. <sd...@gm...>]

On Wed, Oct 10, 2012 at 9:44 PM, Jeff Rogers <dv...@di...> wrote:
>
> ns_returnfile doesn't decode passed filenames from tcl correctly.  As a
> result, [file exists $file] could report that a file does exist but
> [ns_returnfile 200 text/html $file] could give a "not found" error.  The
> circumstances needed to make this happen are uncommon (changing tcl's
> system encoding, actually having files in strange encodings) but it is
> possible.  Fix should be to pass the argument to ns_returnfile through
> Tcl_UtfToExternalDString to get a 'native' file name.

There's been a bunch of stuff done to handle reading and writing bytes
to the connection in the right encoding. As part of that, naviserver
forces the system encoding to utf8 on start-up (this is usually keyed
off env variables). That makes Tcl_UtfToExternalDString a no-op. I
think it's the case that some code assumes Tcl's system encoding is
always utf8, and if it's not things will break. I may be
miss-remembering some of this.

You'll want a precise test case to nail down the circumstances of the
bug. There's some related tests in tests/encoding.test and
tests/tclresp.test.