#3673 (ok 3.5.5) Data gzipped twice (mod_deflate)

[Jørgen Thomsen]

When exporting data using the gzip compression the exported file is gzipped twice

[J.M. Rütter]

Cannot confirm.

Please provide steps to reproduce your problem. Are you exporting a whole server, a database, a table or only some rows?

[Jørgen Thomsen]

Very reproducible problem.

- Display all 3 rows in a small test database
- select Export option below the displayed rows
- select user defined export options and select gzipped
- save the .gz file
- use gunzip twice to decompress the file

This occurs both with a local and a remote database

This does not happen when selecting zipped or bzipped

[Marc Delisle]

Jørgen,
are you running Apache with the mod_gzip module?

[Jørgen Thomsen]

no mod_gzip module

[Jørgen Thomsen]

However, mod_deflate is used and it appears to be a http header issue as a simple .gz file download does not cause the same problem (probably due to mod_deflate configuration for special file types).

Browsers (Firefox, Opera, Chrome) do not decompress the gzipped data as expected.

The http headers should probably be changed.

[Marc Delisle]

Could you attach your mod_deflate configuration details?

[Marc Delisle]

Patch for 3.5.3

[Marc Delisle]

Please try the attached patch (works for me in phpMyAdmin 3.5.3).

[Marc Delisle]

Any news?

[Jørgen Thomsen]

The patch is working for me, too

[Marc Delisle]

This bug was fixed in repository and will be part of a future release; thanks for reporting.

[Marc Delisle]

Jørgen,
I really need your mod_deflate configuration details.

The reason is that some people (include me on a new test machine) have problems with this patch: gzip export compression no longer works. I suspect that this Apache configuration:
<IfModule mod_deflate.c="">
# these are known to be safe with MSIE 6
AddOutputFilterByType DEFLATE text/html text/plain text/xml

      # everything else may cause problems with MSIE 6
      AddOutputFilterByType DEFLATE text/css
      AddOutputFilterByType DEFLATE application/x-javascript application/jav

ascript application/ecmascript
AddOutputFilterByType DEFLATE application/rss+xml
</IfModule>

means that mod_deflate only acts on certain MIME types, and that on your machine, more (or all) MIME types are passed thru mod_deflate.

[Jørgen Thomsen]

The only configuration about this I can find, is this

httpd.conf

AddType application/x-gzip .gz .tgz

<Location />
SetOutputFilter DEFLATE
SetInputFilter DEFLATE
SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary
SetEnvIfNoCase Request_URI \.(?:exe|t?gz|zip|bz2|sit|rar)$ no-gzip dont-vary
SetEnvIfNoCase Request_URI \.gz$ no-gzip dont-vary
    <IfModule mod_headers.c>
      # properly handle requests coming from behind proxies
      Header append Vary User-Agent
    </IfModule>
</Location>

DeflateFilterNote Ratio deflation
DeflateFilterNote Input instream
DeflateFilterNote Output outstream
DeflateCompressionLevel 7
DeflateBufferSize 65536

In php.ini

zlib.output_compression = Off

1) All files stored on server in /tmp
When testing now (4.0.4.2), gzipped files are not compressed at all, but getting the .gz postfix. However, zipped and b2zipped files are compressed as expected.

While testing I detected, that ticking off the option of overwriting an existing file had no effect. It was not allowed.

2) Files downloaded and stored locally
Compression appears to be working OK

[Marc Delisle]

Thanks. Starting with version 4.0.6, you'll probably find that the problem (gzipped twice) is back for you. We had to react, due to my fix causing other problems. See discussion on https://sourceforge.net/p/phpmyadmin/mailman/phpmyadmin-devel/thread/20130809084959.386d4515%40rincewind.suse.cz/#msg31268701

[Jørgen Thomsen]

Well, so be it, but I don't buy that the mod_deflate configuration is broken. It is very similar to the example displayed in the mod_deflate documentation.
http://httpd.apache.org/docs/2.2/mod/mod_deflate.html and .gz files are explicitly excepted from compression in exactly the same way as .zip and .bz2 files.

[Jørgen Thomsen]

I have exported an entire table in CSV format in all 4 formats (no compression, zipped, bzipped, gzipped) and both in /tmp and locally on the client.
This was done after applying the patch below, and I found no problems with compression.

--- phpMyAdmin-4.0.4.1-all-languages/export.php 2013-06-30 07:01:24.000000000 +0200
+++ phpMyAdmin-4.0.4.2-all-languages/export.php 2013-08-11 00:28:49.547475788 +0200
@@ -266,13 +266,15 @@
 function PMA_gzencodeNeeded()
 {
     if (@function_exists('gzencode')
-        && ! @ini_get('zlib.output_compression')
+        && ($GLOBALS['save_on_server'] || (
+        ! @ini_get('zlib.output_compression')
         // Here, we detect Apache's mod_deflate so we bet that
         // this module is active for this instance of phpMyAdmin
         // and therefore, will gzip encode the content
         && ! (function_exists('apache_get_modules')
             && in_array('mod_deflate', apache_get_modules()))
         && ! PMA_isGzHandlerEnabled()
+        ))
     ) {
         return true;
     } else {

[Michal Čihař]

The problem with filtering by Request_URI is that it can not work with dynamic content - in this case the request URI is export.php and we can not change that. We correctly set content type and filename in headers and we can not do more.

With your proposed code it breaks if mod_deflate is used with AddOutputFilterByType, which seems to be more widely used. And I don't see way to make the code work in both cases.