#110 gzip encoding + Content-Length header

When I send file I now it's exact size so content_length() can be used to specify it. File's type may be html, in that case mime type is set to "text/html". In that combination of settings gzip encoding will be used by cppcms, but Content-Length header will not be updated, so client will receive compressed data with incorrect Content-Length header (most likely higher than actual) and will wait for request completion even if no more data available.

Possible fixes:
1. cppcms::http::response::need_gzip(): add check for Content-Length header similar to Content-Encoding.
2. If stream is gzipped, remove Content-Length or fix it to correct value if possible.
3. Something else (at least it should be documented).

response().content_type(mime_file.mime_type_by_name(filepath));
response().io_mode(cppcms::http::response::nogzip); ///< \note this is required otherwise data may be gzipped and content_length will be incorrect
response().set_header("Content-Disposition", "filename=\"" + filename + "\"");
response().content_length(boost::filesystem::file_size(filepath));
boost::filesystem::ifstream fin(filepath);
response().out() << fin.rdbuf();

Some maybe useful info:
cppcms=1.0.4
OS distrib=Archlinux
kernel=3.12.8-1-ARCH
machine=x86_64

[Artyom Beilis]

As you correctly noticed you do not know the content length - you can either disable gzip or not provide content length.

In general you are not required to provide content length some HTTP servers would do this for you. More than that sometimes it is just not possible - consider sending 100MB file - it is good to compress it but it is bad idea to store it as compression is done on the fly.

Good web servers would just set it in chunked encoding.

[Anonymous]

Example code sends data to nginx which compresses it on-the-fly and sends it chunked. I didn't find how to send chunked data using cppcms (am I supposed to use raw encoding and format chunks by hand?).

What I consider wrong behavior: cppcms::http::response::need_gzip() does not disable internal gzip compression when content_length() is set. It is known that content_length() can't work correctly with internal gzip (it is not possible to determine content_length with implicit gzip), so this behavior is unobvious and misleading. In general good interface should not provide an easy way to hurt yourself. This case is very hard to debug and found the source of the problem. Simptoms are continuous loading after content was rendered in the browser. It can lead to lots of wasted hours of developer who decided to notify client about content length and forget to disable gzip compression.

[Artyom Beilis]

1st you don't work over HTTP you work over FastCGI or SCGI with nginx. Nginx itself would create chunked output.

In general it is not your job to provide the content_length. HTTP handles this for you.

[Anonymous]

Does it matter? Library's user still have this trouble with default behavior: user sends file with known size, I think it is convenient to provide it's length for client. But library gzips it and content_length becomes invalid. So it is always necessary to disable gzip when using content_length() header. Why does library require that from user? It can easily disable gzip itself, remove content_length, or notify user about invalid composition of settings by exception (maybe it is the best option, verbose error is better than silent behavior change). All options are easily implemented and will save library's user from unnecessary troubleshooting.

If you consider this low-priority I can create a patch for review if we choose the way to fix that. I am interested in library's development and don't want anyone else to run into troubles with this behavior in the future.

About my usage: could you please explain what is the best practice to send file with cppcms? I use

response().set_header("Content-Disposition", "filename=\"" + filename + "\"");  
response().content_length(boost::filesystem::file_size(filepath));
response().out() << filestream.rdbuf();

I think it is a good idea to provide content length to enable downloading progress bar for user. It is not done automatically.

But in this example gzip will mess things up.

I know about xsendfile headers, but sometimes they can't be used so I need to send data over response object.