#2 Vary: being sent on too many requests

mod_gzip-1.3.19.2a
open-fixed
None
2
2002-10-22
2002-09-16
Todd Vierling
No

In .2a, Vary: is now sent by default, but it has a very
harsh interaction with requests. It seems that
*EVERYTHING* is being sent with a Vary: header, rather
than those things which match the includes/excludes
specified in httpd.conf.

Besides the fact that this will cause later Squid to
take up much more room for caching for an entire site,
this is causing some bizarre problems. (Among them,
MSIE5 apparently gets very confused about image/*
carrying a Vary: header, and ends up not capable of
right-click-and-save on images if this happens. Rather
than saving the image file, a BMP is saved...!)

Basically, it looks as if mod_gzip should be reversing
the checks for include/exclude and Accept-Encoding so
that Vary: can be sent properly (on *only* those URIs
which would elicit varying behavior thanks to mod_gzip).

Discussion

  • Todd Vierling
    Todd Vierling
    2002-09-16

    • priority: 5 --> 7
     
  • Logged In: YES
    user_id=329823

    Yes, you're right. This is a bug which requires bigger code
    changes so this needs some time to solve.

     
  • Logged In: YES
    user_id=329823

    At the moment you can do a

    <Files ~ "\.(gif|jpg|png)$">
    mod_gzip_send_vary Off
    </Files>

    for the files you want to exclude from the vary headers

     
  • Logged In: YES
    user_id=211909

    I am not sure about the exact mechanism how mod_gzip is
    currently setting the "Vary:" header, but I thing it will
    set a "note" when invoked and never remove this note.

    There might be a way to reduce the requests with a "Vary:"
    header a lot if the rules checking routine would do
    something before returning a Boolean to its caller. The rule
    checking routine is the one to know
    a) whether it is actually checking "reqheader"! rules or
    static rules and
    b) whether it is DECLINEing because of a missing INCLUDE or
    a firind EXCLUDE.
    Maybe there can be some logic to decide about the above, and
    in some cases (like a static exclude rule firing) simply
    remove the "note", thus suppress the "Vary:" header from
    being sent at all?

     
    • status: open --> open-accepted
     
    • status: open-accepted --> open-fixed
     
  • Logged In: YES
    user_id=329823

    fixed. mod_gzip now sends no Vary:-header when an item is
    excluded by 'file' or 'uri'.

     
    • status: open-fixed --> closed-fixed
     
  • Todd Vierling
    Todd Vierling
    2002-10-02

    Logged In: YES
    user_id=596653

    The fix still needs work (as noted on the mailing list).
    MIME types that are excluded still get a "Vary:" header, as
    are all requests not explicitly included if restricting to a
    specific set of "include" directives. Evidence the config
    fragment:

    mod_gzip_on Yes
    mod_gzip_item_include mime ^text/

    This configuration never compresses images (MIME image/*),
    yet they still get a Vary: header.

     
  • Todd Vierling
    Todd Vierling
    2002-10-02

    • status: closed-fixed --> open-remind
     
  • Todd Vierling
    Todd Vierling
    2002-10-02

    Logged In: YES
    user_id=596653

    Lowering priority because a workaround is known: use
    <Files> in conjunction with "Header unset Vary". (Use of
    "mod_gzip_send_vary" doesn't work properly due to another
    known bug....)

     
  • Todd Vierling
    Todd Vierling
    2002-10-02

    • priority: 7 --> 2
     
    • assigned_to: nobody --> ckruse
     
  • Todd Vierling
    Todd Vierling
    2002-10-03

    • status: open-remind --> open-postponed
     
    • status: open-postponed --> open-fixed
     
  • Logged In: YES
    user_id=329823

    I hope, that this bug is now fixed... MIME, URI, FILE and
    RSPHEADER should have no Vary headers.

     
  • Logged In: YES
    user_id=524482

    This do not actually work, at least as mod_gzip.c v1.35 and
    mod_gzip.h v1.16 (which, according to CVS claim to have it
    fixed).

    I use following config:
    mod_gzip_can_negotiate No
    mod_gzip_minimum_file_size 400
    mod_gzip_maximum_file_size 33554432
    mod_gzip_maximum_inmem_size 60000
    mod_gzip_temp_dir /home/httpd/gzip_tmp
    mod_gzip_keep_workfiles No
    mod_gzip_dechunk Yes
    mod_gzip_add_header_count Yes

    mod_gzip_item_include file .*

    mod_gzip_item_include mime ^text/(html|plain)
    mod_gzip_item_include mime ^application/(msword|vnd.ms-
    (excel|powerpoint)|x-shoc
    kwave-flash)

    mod_gzip_item_exclude mime ^image/

    <FilesMatch "\.(cgi|doc|epl|htm|html|p(hp|hp3|html|l|pt|ps)
    |shtml|swf|txt|xls)$"
    >
    mod_gzip_on Yes
    </FilesMatch>

    When I try to download simple CGI, which sets Content-type
    to image/gif, I get following response:

    monster:~/tmp# wget -S
    http://www.maupa.telvinet.pl/test2.cgi
    [ ... ]
    1 HTTP/1.1 200 OK
    2 Date: Sat, 15 Mar 2003 21:30:23 GMT
    3 Server: Apache
    4 Vary: Accept-Encoding
    5 Connection: close
    6 Content-Type: image/gif

    There should be no 'Vary: Accept-Encoding', as such result
    would never be compressed anyway. By the way, wget does
    not support compression, but this should not make any
    difference.
    Test script test2.cgi is just:

    #!/bin/bash

    echo Content-type: image/gif
    echo
    echo abc

    (it does not produce any valid image, but this shouldn't be of
    mod_gzip concern).