ziproxy-users — discussion regarding ziproxy usage

Re: [Ziproxy-users] Re: Exclude Image file types possible?

[Will <ws...@99...>]

I'm not saying it should be removed all together, but rather to give people
the option to configure ziproxy the way that is most pleasing to them.

My reason for requesting this feature is this:

Think about a typical .gif file; it's pretty small with the exception of
animated gifs.  When you recompress a .gif file you don't generally gain a
whole lot. Now think about how cruddy a page looks when the transparency is
removed.  You could argue that the site should use PNG files but PNG files
can actually crash some browsers on Macintosh computers if they are over a
certain size.   I found that out the hard way when I used png files for an
logo and ended up converting them back to gifs in order to fix compatibility
problems.  While you can barely tell (quality wise) when a jpeg is
recompressed it is really noticeable when a .gif suddenly becomes fully
solid and ruins the look of a page.

I did a test:
a gif file 233 x122 pixels (a "larger" .gif file)
7257 bytes Original size with transparency
5762 bytes after ziproxy, lost transparency now the gif background is
different from web page background
1495 bytes difference

I'd rather keep the theme and look of the page and take half a second longer
to load.

You can modify how much compression you get with a jpg and alter whether or
not it should recompress it at all, why not allow the same capability to
choose whether or not to recompress .gif files?  I don't see any reason
_not_ to include some kind of exclusion list; it would empty by default and
ultimately it would be up to the user whether to activate the feature or
not.

Keep in mind that ziproxy also recompresses jpegs, bmp, html, text,
JavaScript and a whole slew of other web components which will help keep it
snappy and responsive.  By allowing the individual to surf they way they
want to surf, it increases both the flexibility and value of the program.

Will


----- Original Message ----- 
From: "Knute Johnson" <kn...@fr...>
To: <zip...@li...>
Sent: Friday, November 28, 2003 1:26 PM
Subject: [Ziproxy-users] Re: Exclude Image file types possible?


If I can put in an opinion, the whole purpose is to decrease
bandwidth requirements.  If you stop compressing certain types of
images there is no reason to use Ziproxy.  There are a couple of
issues with transparent gifs and animated gifs but not compressing
them brings you back to the question of why use Ziproxy at all.

I'm helping a friend evaluate Ziproxy for use on his network.  The
issue we are looking at right now is how to easily change the browser
selection from one proxy to another.  We had looked at using a script
to change the Windows registry settings, an applet to do the same or
write to the hosts file, a small user application to write to the
hosts file, or a local proxy with a user selection control.  This is
to somewhat address the same issue and that is to easily get the full
content of the page without compression and quickly switch back to
the Ziproxy proxy.

knute...

>Hi,
>
>this should be well discussed first. Whether there couldn't be found
>more general (configuration) rule as to which images to compress and
>which to keep. You may find yourself adding new items into Exclude
>list very often. And what about pages with "dynamically" generated
>images, like /img.php?id=something ?
>
>Proxy server "wwwoffle" allows nearly everything to be set on per-URL
>basis. But its C code is then much more complex and hard to get into,
>despite its good style/readability. I don't feel like this, maybe
>after few years of further C programming practice.....
>
>On Wednesday 26 November 2003 21:49, wspigel wrote:
>> I think this would be useful for multiple reasons.  First reason
>> being that animated gif files are everywhere on the internet and
>> they add a good deal of environment to it.  By compressing .gif
>> files ziproxy not only removes their transparancy but also their
>> animation.  Some sites that rely on the transparency of .gif files
>> look bad when they are converted.
>>
>> Any exclude list could look like the "Compressible" list that is
>> used currently:
>>
>> Exclude = { .gif, .tiff, .psp, .dwg, .tga }
>>
>> I think this will increase the flexibility of the program and allow
>> a higher variation in variety of configurations.
>
>
>
>
>-------------------------------------------------------
>This SF.net email is sponsored by: SF.net Giveback Program.
>Does SourceForge.net help you be more productive?  Does it
>help you create better code?  SHARE THE LOVE, and help us help
>YOU!  Click Here: http://sourceforge.net/donate/
>_______________________________________________
>Ziproxy-users mailing list
>Zip...@li...
>https://lists.sourceforge.net/lists/listinfo/ziproxy-users
>

-- 
Knute Johnson
Molon Labe...





-------------------------------------------------------
This SF.net email is sponsored by: SF.net Giveback Program.
Does SourceForge.net help you be more productive?  Does it
help you create better code?  SHARE THE LOVE, and help us help
YOU!  Click Here: http://sourceforge.net/donate/
_______________________________________________
Ziproxy-users mailing list
Zip...@li...
https://lists.sourceforge.net/lists/listinfo/ziproxy-users

[Ziproxy-users] Re: Exclude Image file types possible?

[Knute J. <kn...@fr...>]

If I can put in an opinion, the whole purpose is to decrease 
bandwidth requirements.  If you stop compressing certain types of 
images there is no reason to use Ziproxy.  There are a couple of 
issues with transparent gifs and animated gifs but not compressing 
them brings you back to the question of why use Ziproxy at all.

I'm helping a friend evaluate Ziproxy for use on his network.  The 
issue we are looking at right now is how to easily change the browser 
selection from one proxy to another.  We had looked at using a script 
to change the Windows registry settings, an applet to do the same or 
write to the hosts file, a small user application to write to the 
hosts file, or a local proxy with a user selection control.  This is 
to somewhat address the same issue and that is to easily get the full 
content of the page without compression and quickly switch back to 
the Ziproxy proxy.

knute...

>Hi,
>
>this should be well discussed first. Whether there couldn't be found
>more general (configuration) rule as to which images to compress and
>which to keep. You may find yourself adding new items into Exclude
>list very often. And what about pages with "dynamically" generated
>images, like /img.php?id=something ?
>
>Proxy server "wwwoffle" allows nearly everything to be set on per-URL
>basis. But its C code is then much more complex and hard to get into,
>despite its good style/readability. I don't feel like this, maybe
>after few years of further C programming practice.....
>
>On Wednesday 26 November 2003 21:49, wspigel wrote:
>> I think this would be useful for multiple reasons.  First reason
>> being that animated gif files are everywhere on the internet and
>> they add a good deal of environment to it.  By compressing .gif
>> files ziproxy not only removes their transparancy but also their
>> animation.  Some sites that rely on the transparency of .gif files
>> look bad when they are converted.
>>
>> Any exclude list could look like the "Compressible" list that is
>> used currently:
>>
>> Exclude = { .gif, .tiff, .psp, .dwg, .tga }
>>
>> I think this will increase the flexibility of the program and allow
>> a higher variation in variety of configurations.
>
>
>
>
>-------------------------------------------------------
>This SF.net email is sponsored by: SF.net Giveback Program.
>Does SourceForge.net help you be more productive?  Does it
>help you create better code?  SHARE THE LOVE, and help us help
>YOU!  Click Here: http://sourceforge.net/donate/
>_______________________________________________
>Ziproxy-users mailing list
>Zip...@li...
>https://lists.sourceforge.net/lists/listinfo/ziproxy-users
>

-- 
Knute Johnson
Molon Labe...

[Ziproxy-users] Re: Exclude Image file types possible?

[Juraj V. <va...@na...>]

Hi,

this should be well discussed first. Whether there couldn't be found 
more general (configuration) rule as to which images to compress and which to 
keep. You may find yourself adding new items into Exclude list very often. 
And what about pages with "dynamically" generated images, like 
/img.php?id=something ?

Proxy server "wwwoffle" allows nearly everything to be set on per-URL basis. 
But its C code is then much more complex and hard to get into, despite its 
good style/readability. I don't feel like this, maybe after few years of 
further C programming practice.....

On Wednesday 26 November 2003 21:49, wspigel wrote:
> I think this would be useful for multiple reasons.  First reason being that
> animated gif files are everywhere on the internet and they add a good deal
> of environment to it.  By compressing .gif files ziproxy not only removes
> their transparancy but also their animation.  Some sites that rely on the
> transparency of .gif files look bad when they are converted.
>
> Any exclude list could look like the "Compressible" list that is used
> currently:
>
> Exclude = { .gif, .tiff, .psp, .dwg, .tga }
>
> I think this will increase the flexibility of the program and allow a
> higher variation in variety of configurations.

Re: [Ziproxy-users] Exclude Image file types possible?

[wspigel <ws...@99...>]

I think this would be useful for multiple reasons.  First reason being th=
at
animated gif files are everywhere on the internet and they add a good dea=
l
of environment to it.  By compressing .gif files ziproxy not only removes
their transparancy but also their animation.  Some sites that rely on the
transparency of .gif files look bad when they are converted.

Any exclude list could look like the "Compressible" list that is used
currently:

Exclude =3D { .gif, .tiff, .psp, .dwg, .tga }

I think this will increase the flexibility of the program and allow a hig=
her
variation in variety of configurations.

Will


----- Original Message -----=20
From: "Juraj Variny" <va...@na...>
To: <zip...@li...>
Cc: <"\"\""@naex.sk>
Sent: Wednesday, November 26, 2003 10:14 AM
Subject: Re: [Ziproxy-users] Exclude Image file types possible?


> Sorry, there isn't such option. Please suggest why is it necesssary and
how
> should that list look.
>
> Juraj
>
> Cit=E1t wspigel <ws...@99...>:
>
> > Is there any known way to exclude certain types of files and images f=
rom
> > being compressed?  For example, say I want .gif files to be passed
through
> > without being recompressed, can this be done with an "exclude list"
> > somewhere in the ziproxy.conf file?
> >
> > Will
> >
> >
> >
> >
> > -------------------------------------------------------
> > This SF.net email is sponsored by: SF.net Giveback Program.
> > Does SourceForge.net help you be more productive?  Does it
> > help you create better code?  SHARE THE LOVE, and help us help
> > YOU!  Click Here: http://sourceforge.net/donate/
> > _______________________________________________
> > Ziproxy-users mailing list
> > Zip...@li...
> > https://lists.sourceforge.net/lists/listinfo/ziproxy-users
> >
>
>
> --=20
>
>
>
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: SF.net Giveback Program.
> Does SourceForge.net help you be more productive?  Does it
> help you create better code?  SHARE THE LOVE, and help us help
> YOU!  Click Here: http://sourceforge.net/donate/
> _______________________________________________
> Ziproxy-users mailing list
> Zip...@li...
> https://lists.sourceforge.net/lists/listinfo/ziproxy-users
>
>

Re: [Ziproxy-users] Exclude Image file types possible?

[Juraj V. <va...@na...>]

There isn't such option. Currently, you can set images not to be compresssed 
only based on their dimensions.

Juraj

wspigel <ws...@99...> wrote:

> Is there any known way to exclude certain types of files and images from
> being compressed?  For example, say I want .gif files to be passed through
> without being recompressed, can this be done with an "exclude list"
> somewhere in the ziproxy.conf file?
> 
> Will
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.net email is sponsored by: SF.net Giveback Program.
> Does SourceForge.net help you be more productive?  Does it
> help you create better code?  SHARE THE LOVE, and help us help
> YOU!  Click Here: http://sourceforge.net/donate/
> _______________________________________________
> Ziproxy-users mailing list
> Zip...@li...
> https://lists.sourceforge.net/lists/listinfo/ziproxy-users
> 


-- 

Re: [Ziproxy-users] Exclude Image file types possible?

[Juraj V. <va...@na...>]

Sorry, there isn't such option. Please suggest why is it necesssary and h=
ow=20
should that list look.

Juraj

Cit=E1t wspigel <ws...@99...>:

> Is there any known way to exclude certain types of files and images fro=
m
> being compressed?  For example, say I want .gif files to be passed thro=
ugh
> without being recompressed, can this be done with an "exclude list"
> somewhere in the ziproxy.conf file?
>=20
> Will
>=20
>=20
>=20
>=20
> -------------------------------------------------------
> This SF.net email is sponsored by: SF.net Giveback Program.
> Does SourceForge.net help you be more productive?  Does it
> help you create better code?  SHARE THE LOVE, and help us help
> YOU!  Click Here: http://sourceforge.net/donate/
> _______________________________________________
> Ziproxy-users mailing list
> Zip...@li...
> https://lists.sourceforge.net/lists/listinfo/ziproxy-users
>=20


--=20

[Ziproxy-users] Exclude Image file types possible?

[wspigel <ws...@99...>]

Is there any known way to exclude certain types of files and images from
being compressed?  For example, say I want .gif files to be passed through
without being recompressed, can this be done with an "exclude list"
somewhere in the ziproxy.conf file?

Will

[Ziproxy-users] Re: ziproxy 1.3 bug

[Juraj V. <va...@na...>]

Hi all,

I humbly admit I'm sometimes just too lazy to test or inspect bugs. I'll 
release 1.3b today. "1.3.1" would make the numbering even more inconsistent.

Juraj

P.S. I found myself connecting to ziproxy even from my new workplace, where 
there is a DSL connection. And ... it helped! I should really force myself to 
be more responsible.

On Monday 24 November 2003 02:37, Cheuk-san Edward Wang wrote:
> I just fixed a bad typo in forward_content. This bug causes all http
> POST requests to fail. I noticed that you've just released 1.3, but I
> think we should make another new release (1.3.1 maybe) quickly because
> of this bug.
>
> Cheuksan Wang

Re: [Ziproxy-users] timeout bug fixed

[wspigel <ws...@99...>]

Both issues seem to be fixed.  Nice job!  Now I'll have to find something
else to help improve the program :)


----- Original Message ----- 
From: "Juraj Variny" <va...@na...>
To: <zip...@li...>
Sent: Saturday, November 22, 2003 12:19 PM
Subject: [Ziproxy-users] timeout bug fixed


> Hi,
>
> the testing package is at:
>
> http://web.naex.sk/jv/ziproxy-1.3beta22nov2003.tar.gz
>
> Moreover, I tried to guess what could caused the wrong Gzip default value.
> Please test whether it is right now.
>
> Juraj
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: SF.net Giveback Program.
> Does SourceForge.net help you be more productive?  Does it
> help you create better code?  SHARE THE LOVE, and help us help
> YOU!  Click Here: http://sourceforge.net/donate/
> _______________________________________________
> Ziproxy-users mailing list
> Zip...@li...
> https://lists.sourceforge.net/lists/listinfo/ziproxy-users
>
>

[Ziproxy-users] timeout bug fixed

[Juraj V. <va...@na...>]

Hi,

the testing package is at:
 
http://web.naex.sk/jv/ziproxy-1.3beta22nov2003.tar.gz

Moreover, I tried to guess what could caused the wrong Gzip default value. 
Please test whether it is right now.

Juraj

Re: [Ziproxy-users] Re: Re: Re: Ziproxy 1.3 Issues and Suggestion

[Juraj V. <jv...@de...>]

Hi,

On Friday 14 November 2003 06:38, you wrote:
> It is the version that came with RedHat 7.3. I don't know what version it
> is, I've never updated it as far as I know.  How would I find out what
> version it is?

=46rom full name of the RPM file, just the first numbers from the beginning=
=2E=20
I've looked both at redhat mirror and at freshrpms, but nowhere saw any=20
libconfuse RPM.

>Color changing GIFs:
>You also asked what places I've seen gif file backgrounds change colors.
>Attached is one such gif, in original form, that changes when passed throu=
gh
>ziproxy, the URL is here:
>http://mail.99main.com/cgi-bin/openwebmail/openwebmail.pl

This picture was subject to GIF->JPEG (not an GIF->PNG) conversion. With=20
default quality setting, PNG would have about 5700 bytes, and JPEG about 30=
00=20
bytes only. The FAQ item "Some pictures have inapropriate background color"=
=20
then applies here.

> *** end large EXE timeout ***
> Trying to compress downloads:
> I also mentioned that ziproxy tries to compress download files such as
> executables, if the download doesn't finish before ziproxy times out the
> file transfer fails and there doesn't seem to be a way to tell ziproxy to
> not compress them.  Are you aware of that issue?  I explained it more in =
my
> last e-mail.

I've reproduced this and will fix. This is not related to compression, why =
you=20
got that impression?

Juraj

[Ziproxy-users] Re: Re: Re: Ziproxy 1.3 Issues and Suggestion

[Juraj V. <jv...@de...>]

Hi,

On Wed, 12 Nov 2003, will spigel 000727 wrote:
> Yes, that compiled just fine and seems to work great!  The only thing I
> noticed is that it doesn't seem to compress text/html pages unless you put
> "Gzip = true" in the ziproxy.conf even though the docs mention it is on by
> default.  Image compression is still working, that hasn't had a problem so far.  

I wasn't able to reproduce this. Try to send again a bugreport, or at
least the URL.

> The only glaring problem I see at this point is the random background color
> changes on some gif and png images when they are recompressed.  There doesn't
> seem to be a way to "turn off" compression of them either like there is with
> jpg files.  

There may be sufficient if you send a URL of the picture. There's no
reason why GIF->PNG compression should change the background.  

> Thanks for all your hard work,
> Will
Thanks for your patience, too.

Juraj

> 
> ---------- Original Message -----------
> From: Juraj Variny <va...@na...>
> To: "" <zip...@li...>
> Sent: Wed, 12 Nov 2003 18:16:56 +0100
> Subject: [Ziproxy-users] Re: Re: Ziproxy 1.3 Issues and Suggestion
> 
> > Hi,
> > 
> > I tried copying new files into 1.3beta source directory as you did,
> >  but everything was OK then. Here is the package:
> > 
> > http://web.naex.sk/jv/ziproxy-1.3beta12nov2003.tar.gz
> > 
> > Juraj
> > -- 
> > 
> > -------------------------------------------------------
> > This SF.Net email sponsored by: ApacheCon 2003,
> > 16-19 November in Las Vegas. Learn firsthand the latest
> > developments in Apache, PHP, Perl, XML, Java, MySQL,
> > WebDAV, and more! http://www.apachecon.com/
> > _______________________________________________
> > Ziproxy-users mailing list
> > Zip...@li...
> > https://lists.sourceforge.net/lists/listinfo/ziproxy-users
> ------- End of Original Message -------
> 
> 
> ______________________________________________
> 99main Internet Services http://www.99main.com
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email sponsored by: ApacheCon 2003,
> 16-19 November in Las Vegas. Learn firsthand the latest
> developments in Apache, PHP, Perl, XML, Java, MySQL,
> WebDAV, and more! http://www.apachecon.com/
> _______________________________________________
> Ziproxy-users mailing list
> Zip...@li...
> https://lists.sourceforge.net/lists/listinfo/ziproxy-users
> 
> 

Re: [Ziproxy-users] Re: Re: Ziproxy 1.3 Issues and Suggestion

[will s. 0. <ws...@99...>]

Yes, that compiled just fine and seems to work great!  The only thing I
noticed is that it doesn't seem to compress text/html pages unless you put
"Gzip = true" in the ziproxy.conf even though the docs mention it is on by
default.  Image compression is still working, that hasn't had a problem so far.  

The only glaring problem I see at this point is the random background color
changes on some gif and png images when they are recompressed.  There doesn't
seem to be a way to "turn off" compression of them either like there is with
jpg files.  

Thanks for all your hard work,
Will


---------- Original Message -----------
From: Juraj Variny <va...@na...>
To: "" <zip...@li...>
Sent: Wed, 12 Nov 2003 18:16:56 +0100
Subject: [Ziproxy-users] Re: Re: Ziproxy 1.3 Issues and Suggestion

> Hi,
> 
> I tried copying new files into 1.3beta source directory as you did,
>  but everything was OK then. Here is the package:
> 
> http://web.naex.sk/jv/ziproxy-1.3beta12nov2003.tar.gz
> 
> Juraj
> -- 
> 
> -------------------------------------------------------
> This SF.Net email sponsored by: ApacheCon 2003,
> 16-19 November in Las Vegas. Learn firsthand the latest
> developments in Apache, PHP, Perl, XML, Java, MySQL,
> WebDAV, and more! http://www.apachecon.com/
> _______________________________________________
> Ziproxy-users mailing list
> Zip...@li...
> https://lists.sourceforge.net/lists/listinfo/ziproxy-users
------- End of Original Message -------


______________________________________________
99main Internet Services http://www.99main.com

[Ziproxy-users] Re: Re: Ziproxy 1.3 Issues and Suggestion

[Juraj V. <va...@na...>]

Hi,

I tried copying new files into 1.3beta source directory as you did, but 
everything was OK then. Here is the package:

http://web.naex.sk/jv/ziproxy-1.3beta12nov2003.tar.gz

Juraj
-- 

[Ziproxy-users] Re: Ziproxy 1.3 Issues and Suggestion

[Juraj V. <va...@na...>]

Hi all,

it seems now fixed ... it was dire programming error -- unmatched function 
parameters :-(

I will like to get some feedback:

* whether anonymous CVS service at sf.net is now reliable. Look into ChangeLog 
to see if you have the latest version. 

* whether my fixes work. If they do, I will release second beta. There may be 
more such errors and I'm unable to test all configuration option 
combinations. Thank you, really.

After downloading from CVS, you have to run ./bootstrap before configure. It 
requires automake and autoconf packages installed. If this is problem, ask me 
for prepared files.

Juraj

On Sunday 09 November 2003 21:33, Juraj Variny wrote:
> Hi,
>
> > My first test was getting a HTML only page,
> > http://www.99main.com/files/test.html
>
> 1.3beta really behaves poorly here with Gzip=true. Thanks for the bug
> report. I will drop a note when fix will be in CVS.
>
> Juraj
>
>
> -------------------------------------------------------
> This SF.Net email sponsored by: ApacheCon 2003,
> 16-19 November in Las Vegas. Learn firsthand the latest
> developments in Apache, PHP, Perl, XML, Java, MySQL,
> WebDAV, and more! http://www.apachecon.com/
> _______________________________________________
> Ziproxy-users mailing list
> Zip...@li...
> https://lists.sourceforge.net/lists/listinfo/ziproxy-users

Re: [Ziproxy-users] Re: Ziproxy 1.3 Issues and Suggestion

[Juraj V. <va...@na...>]

Hi,

> My first test was getting a HTML only page,
> http://www.99main.com/files/test.html

1.3beta really behaves poorly here with Gzip=true. Thanks for the bug report. 
I will drop a note when fix will be in CVS.

Juraj

[Ziproxy-users] Re: Ziproxy 1.3 Issues and Suggestion

[Juraj V. <va...@na...>]

On Saturday 08 November 2003 01:44, Will wrote:
> I have a few questions to ask and also a suggestion.
>
> First, to test ziproxy's compression I took a 108K text file and FTPed it
> to a server that I know does not have any kind of mod_gzip/mod_deflate
> apache module.  Then I tried retreiving it in IE with ziproxy both enabled
> and disabled.  The result was there was no compression of the text file.  I
> used a program called Netmeter to monitor the actual bandwidth used, both
> came out nearly the same.  Ziproxy does not seem to compress straight txt
> files.  What am I doing wrong here?

Please retry with logging turned on (set LogFile option to some filename). 
Then look into log file/send it here. BTW, how did you measure file size? 
Browsers always display uncompressed size in page properties.

> Second, It mentions in the Changelog that it can convert GIF images to
> lossless PNG including preserving transparency.  This doesn't seem to be
> active either.  Do I have to turn this on somehow?

This function should be always active, though it applies to minority of GIF 
images only. Most possible candidates are large screenshots with few colors. 
Try to find such one and send its address, possibly with ziproxy logging 
output, too.

> Third, (possibly a bug?) I know "Gzip = true" by default, but if I put Gzip
> = true in the ziproxy.conf it sends a lot of garbage to the browser.

Well, this really smells like a bug. Do as above, send whole ziproxy.conf and 
OS version info, too.

> Finally, a suggestion:  Is there a way you could make ziproxy look in a
> browser header for a token to trigger compression strength?  For example
> say my header was:
>
> "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"
>
> Is it possible to add something to it which ziproxy reads and adjusts
> compression based on that, example:
>
> "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; Ziproxy 75,75,75,75)"
>
> This would tell ziproxy to compress everything a certain way.  A simple
> client could be created to adjust the header to whatever compression levels
> the person wants.

It would be helpful to know more what you're trying to achieve with this. 
Maybe it will be sufficient to create, say, three configuration files both to 
xinetd (each listening on another port) and ziproxy (each with another 
quality settings) and let users choose this way.

Juraj

Re: [Ziproxy-users] Ziproxy 1.3 Issues and Suggestion

[Cheuk-san E. W. <wa...@ai...>]

On Fri, Nov 07, 2003 at 07:44:20PM -0500, Will wrote:
> 
> Finally, a suggestion:  Is there a way you could make ziproxy look in a browser header for a token to trigger compression strength?  For example say my header was:
> 
> "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"
> 
> Is it possible to add something to it which ziproxy reads and adjusts compression based on that, example:
> 
> "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; Ziproxy 75,75,75,75)"
> 
> This would tell ziproxy to compress everything a certain way.  A simple client could be created to adjust the header to whatever compression levels the person wants.

That'd be a good feature to have. However, I don't see how you can do
it. Are you proposing to change Mozilla, Konqueror, or IE? If you
don't change the browser, how can you change the header?

Cheuksan Wang

[Ziproxy-users] Ziproxy 1.3 Issues and Suggestion

[Will <ws...@99...>]

I have a few questions to ask and also a suggestion.

First, to test ziproxy's compression I took a 108K text file and FTPed =
it to a server that I know does not have any kind of =
mod_gzip/mod_deflate apache module.  Then I tried retreiving it in IE =
with ziproxy both enabled and disabled.  The result was there was no =
compression of the text file.  I used a program called Netmeter to =
monitor the actual bandwidth used, both came out nearly the same.  =
Ziproxy does not seem to compress straight txt files.  What am I doing =
wrong here?

Second, It mentions in the Changelog that it can convert GIF images to =
lossless PNG including preserving transparency.  This doesn't seem to be =
active either.  Do I have to turn this on somehow?

Third, (possibly a bug?) I know "Gzip =3D true" by default, but if I put =
Gzip =3D true in the ziproxy.conf it sends a lot of garbage to the =
browser.


Finally, a suggestion:  Is there a way you could make ziproxy look in a =
browser header for a token to trigger compression strength?  For example =
say my header was:

"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"

Is it possible to add something to it which ziproxy reads and adjusts =
compression based on that, example:

"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; Ziproxy =
75,75,75,75)"

This would tell ziproxy to compress everything a certain way.  A simple =
client could be created to adjust the header to whatever compression =
levels the person wants.  

Re: [Ziproxy-users] Re: Timeout problems

[Knute J. <kn...@fr...>]

>This problem was fixed similar way (thanks to Cheuksan) in CVS version
>few weeks ago. You may try it, if you desire. I will release it as
>1.3-beta soon, when we'll update documentation (not any big changes
>from users' view), JPEG2000 issues (which shouldn't affect
>non-JPEG2000 compilation) and yet untested 'make install'. You will
>need to install automake and autoconf packages, to be able to use
>./bootstrap script.
>
>The HTTPS code was not changed much, so it's likely that the bug is
>still present. What is your configuration? Is ziproxy between client
>and Squid or between Squid and Internet?
>
>Juraj

Juraj:

Thanks.  I'll take a look at the CVS version.  The setup we are 
testing is Ziproxy - Squid - Internet.  Ziproxy is running on a 
separate Linux 9 machine.  It appears to be working perfectly except 
for the https.  I'm going to play with is some more in the next 
couple of days, if I figure anything out I'll post.



-- 
Knute Johnson
Molon Labe...

[Ziproxy-users] Re: Timeout problems

[Juraj V. <va...@na...>]

This problem was fixed similar way (thanks to Cheuksan) in CVS version few 
weeks ago. You may try it, if you desire. I will release it as 1.3-beta soon, 
when we'll update documentation (not any big changes from users' view), 
JPEG2000 issues (which shouldn't affect non-JPEG2000 compilation) and yet 
untested 'make install'. You will need to install automake and autoconf 
packages, to be able to use ./bootstrap script.

The HTTPS code was not changed much, so it's likely that the bug is still 
present. What is your configuration? Is ziproxy between client and Squid or 
between Squid and Internet?

Juraj

On Saturday 11 October 2003 20:28, Knute Johnson wrote:
> In testing Ziproxy we discovered some problems with it timing out.
> It would time out even if data was being downloaded.  The timeout
> feature is critical to cleaning up left over instances of ziproxy if
> it doesn't terminate normally or the data is not delivered in a
> timely manner.  We left the time out setting in the .conf file at 240
> seconds but added some code to ziproxy.c to reset the timer.
>
> The complete listing for ziproxy.c is below with the additions I made
> to solve our timeout problems.  I have marked them all with comments,
> search for `knute'.  If anybody has a better idea on how to solve
> this issue, please let us know.
>
> We have been running tests with Ziproxy pointed at Squid and it
> appears to work flawlessly except that we can't seem to get and https
> connections to function.  I'm not sure why that is but it may have
> something to do with the NextProxy NextPort not pointing to the right
> place for https.
>
> Thanks very much,
>
> knute...
>
> 	/* ziproxy.c

[Ziproxy-users] Timeout problems

[Knute J. <kn...@fr...>]

In testing Ziproxy we discovered some problems with it timing out.  
It would time out even if data was being downloaded.  The timeout 
feature is critical to cleaning up left over instances of ziproxy if 
it doesn't terminate normally or the data is not delivered in a 
timely manner.  We left the time out setting in the .conf file at 240 
seconds but added some code to ziproxy.c to reset the timer.

The complete listing for ziproxy.c is below with the additions I made 
to solve our timeout problems.  I have marked them all with comments, 
search for `knute'.  If anybody has a better idea on how to solve 
this issue, please let us know.

We have been running tests with Ziproxy pointed at Squid and it 
appears to work flawlessly except that we can't seem to get and https 
connections to function.  I'm not sure why that is but it may have 
something to do with the NextProxy NextPort not pointing to the right 
place for https.

Thanks very much,

knute...

	/* ziproxy.c
	 * 2002 Juraj Variny<va...@na...>
	 *
	 * Released subject to GNU General Public License.
	 * 
	 * Based on mwp_proxy.c from Mark Williams<mw...@au...>
	 */
	
	#include <unistd.h>
	#include <stdlib.h>
	#include <stdio.h>
	#include <string.h>
	#include <sys/types.h>
	#include <time.h>
	#include <sys/time.h>
	#include <sys/resource.h>
	#include <signal.h>
	#include <sys/socket.h>
	#include <netinet/in.h>
	#include <netdb.h>
	#include <fcntl.h>
	#include <syslog.h>
	
	
	#include "image.h"
	#include "htmlmodify.h"
	#include "cfgfile.h"
	
	#define SERVER_NAME "ziproxy"
	#define PROTOCOL "HTTP/1.1"
	#define LOGGING (LogFileName || LogPipe)
	
	#define MAX_LINELEN 4096
	#define MAX_HEADERS 50
	#define RFC1123FMT "%a, %d %b %Y %H:%M:%S GMT"
	
	typedef char *headers[MAX_HEADERS];
	
	enum {IMG_NONE, IMG_PNG, IMG_GIF, IMG_JPEG};
	
	#define WILLGZIP (1<<1)
	#define SUFF_MODIFIED (1<<2)
	#define MODIFY_HTML (1<<3)
	#define CONN_KEEPALIVE (1<<4)
	
	/* Forwards. */
	static int open_client_socket( char* hostname, short Port );
	static headers* get_client_headers(int*, int*);
	static void proxy_http( headers *lines, char* method, char* path, 
char* protocol, 
					FILE* sockrfp, FILE* sockwfp, int content_length, int flags);
	static void proxy_ssl( char* method, char* host, int Port, char* 
protocol, FILE* sockrfp, FILE* sockwfp );
	static int check_trim( char* line );
	static void send_error( int status, char* title, char* extra_header, 
char* text );
	static void send_headers( int status, char* title, char* 
extra_header, char* mime_type, int length, time_t mod );
	static long timeval_subtract_us ( struct timeval *x, struct timeval 
*y);
	static char* allocstr(size_t size);
	static void sigcatch( int sig );
	
	//logging file
	FILE *logfile;
	struct timeval newtime, oldtime;
	
	int logpid, is_sending_data = 0;
	static char line[MAX_LINELEN];
	
	int
	main( int argc, char** argv )
	    {
	    char *method, *url, *protocol, *host, *path;
	    short Port;
	    int sockfd;
	    int ssl, flags = 0;
		int linelen,content_length;
	    FILE* sockrfp;
	    FILE* sockwfp;
	    time_t tm;
		headers* req_hdrs;
	
	    logpid=getpid();
	
	    if(argc > 1) ReadCfgFile(argv[1]);
	    else ReadCfgFile(DefaultCfgLocation);
		
	if(LOGGING){
	    time(&tm);
	    //open logging file
		if(LogPipe)
			logfile = stderr;
		else if(LogFileName)
	{
		struct tm *btime;
		time_t cas;
			char logname[50];

			time(&cas);
			btime = localtime(&cas);
			strftime(logname,sizeof(logname),LogFileName,btime);
		    logfile = fopen(logname, "a");
	}
		
		//set line buffered mode for logfile
		setvbuf(logfile,NULL,_IOLBF,BUFSIZ);
		//setlinebuf(logfile);
		
		fprintf(logfile, "[%d] Starting - %s", logpid, ctime(&tm));
	
	    gettimeofday(&oldtime,NULL);
	}
		
	if(ZTimeout){
	    /* Get ready to catch timeouts.. */
	    (void) signal( SIGALRM, sigcatch );
	
	    (void) alarm( ZTimeout );
	}

		/* Read the first line of the request. */
	    if ((fgets(line, sizeof(line), stdin ) ==  NULL) ||	
						(linelen = check_trim(line)) == 0)
			send_error( 400, "Bad Request", (char*) 0, 
							"No request found or request too long." );
		
	    method=allocstr (5*linelen);
		url = method + linelen;
		protocol = url + linelen;
		host = protocol + linelen;
		path = host + linelen;
		
	    /* Parse it. */
	    if ( sscanf( line, "%[^ ] %[^ ] %[^ ]", method, url, protocol ) 
!= 3 )
			send_error( 400, "Bad Request", (char*) 0, "Can't parse request." 
);
	
	    if ( url == (char*) 0 )
			send_error( 400, "Bad Request", (char*) 0, "Null URL." );
	
	    if ( strncasecmp( url, "http://", 7 ) == 0 )
		{
	
			(void) strncpy( url, "http", 4 );	/* make sure it's lower case */
			if ( sscanf( url, "http://%[^:/]:%hd%s", host, &Port, path ) == 3 
);
			else if ( sscanf( url, "http://%[^/]%s", host, path ) == 2 )
			{
			    Port = 80;
			}
			else if ( sscanf( url, "http://%[^:/]:%hd", host, &Port ) == 2 )
			{
				path=url;
			}
			else if ( sscanf( url, "http://%[^/]", host ) == 1 )
			{
			Port = 80;
				path=url;
		    }
		else
		    send_error( 400, "Bad Request", (char*) 0, "Can't parse URL." 
);
		ssl = 0;
		
		if(NextProxy)	path = url;

		    //log URL
		if(LOGGING) fprintf(logfile, "[%d] URL - %s\n",logpid , url);
	
		if(undo_name(path)) flags |= SUFF_MODIFIED;
		}
	else if ( strcmp( method, "CONNECT" ) == 0 )
		{
		if ( sscanf( url, "%[^:]:%hd", host, &Port ) == 2 )
		    ;
		else if ( sscanf( url, "%s", host ) == 1 )//equivalent to strcpy 
until first whitespace?
		    Port = 443;
		else
		    send_error( 400, "Bad Request", (char*) 0, "Can't parse URL." 
);
		ssl = 1;

		if(LOGGING) fprintf(logfile, "[%d] URL - %s\n",logpid , url);
		}
	else
		send_error( 400, "Bad Request", (char*) 0, "Unknown URL type." );
	
		
		if(!ssl)
		{
				req_hdrs=get_client_headers(&content_length, &flags);
	if(LOGGING){
		gettimeofday(&newtime,NULL);
		fprintf(logfile, "[%d] getting, parsing headers took %ld us 
\n",logpid,
				timeval_subtract_us(&newtime, &oldtime));
		fprintf(logfile, "[%d] Method = %s Protocol = %s\n",logpid, method, 
protocol);
		gettimeofday(&oldtime,NULL);
	}
		}
	
	    /* Open the client socket to the real web server. */
	    sockfd = open_client_socket( host, Port );
	
	    /* Open separate streams for read and write, r+ doesn't always 
work. 
		 * What about "a+" ? */
	    sockrfp = fdopen( sockfd, "r" );
	    sockwfp = fdopen( sockfd, "w" );
	
	    if ( ssl )
		proxy_ssl( method, host, Port, protocol, sockrfp, sockwfp );
	    else
		proxy_http( req_hdrs, method, path, protocol, sockrfp, sockwfp, 
content_length, flags);
	
	    /* Done. */
	if(LOGGING){
	    fprintf(logfile, "[%d] Finished.\n",logpid);
	    fclose(logfile);
	}
	    (void) close( sockfd );
	    exit( 0 );
	    }
	
	
	#if defined(AF_INET6) && defined(IN6_IS_ADDR_V4MAPPED)
	#define USE_IPV6
	#endif
	
	static int
	open_client_socket( char* hostname, short Port )
	    {
	#ifdef USE_IPV6
	    struct addrinfo hints;
	    char Portstr[10];
	    int gaierr;
	    struct addrinfo* ai;
	    struct addrinfo* ai2;
	    struct addrinfo* aiv4;
	    struct addrinfo* aiv6;
	    struct sockaddr_in6 sa;
	#else /* USE_IPV6 */
	    struct hostent *he;
	    struct sockaddr_in sa;
	#endif /* USE_IPV6 */
	    int sa_len, sock_family, sock_type, sock_protocol;
	    int sockfd;
	
	    memset( (void*) &sa, 0, sizeof(sa) );
	
	if(NextProxy){
		hostname = NextProxy;
		Port = NextPort;
	}
	
	
	#ifdef USE_IPV6
	    (void) memset( &hints, 0, sizeof(hints) );
	    hints.ai_family = PF_UNSPEC;
	    hints.ai_socktype = SOCK_STREAM;
	    (void) snprintf( Portstr, sizeof(Portstr), "%d", Port );
	    if ( (gaierr = getaddrinfo( hostname, Portstr, &hints, &ai )) != 
0 )
		send_error( 404, "Not Found", (char*) 0, "Unknown host." );
	
	    /* Find the first IPv4 and IPv6 entries. */
	    aiv4 = (struct addrinfo*) 0;
	    aiv6 = (struct addrinfo*) 0;
	    for ( ai2 = ai; ai2 != (struct addrinfo*) 0; ai2 = ai2->ai_next 
)
		{
		switch ( ai2->ai_family )
		    {
		    case AF_INET: 
		    if ( aiv4 == (struct addrinfo*) 0 )
			aiv4 = ai2;
		    break;
		    case AF_INET6:
		    if ( aiv6 == (struct addrinfo*) 0 )
			aiv6 = ai2;
		    break;
		    }
		}
	
	    /* If there's an IPv4 address, use that, otherwise try IPv6. */
	    if ( aiv4 != (struct addrinfo*) 0 )
		{
		if ( sizeof(sa) < aiv4->ai_addrlen )
		    {
		    (void) fprintf(stderr, "%s - sockaddr too small (%lu < %lu)\n",
			hostname, (unsigned long) sizeof(sa),
			(unsigned long) aiv4->ai_addrlen );
		    exit( 1 );
		    }
		sock_family = aiv4->ai_family;
		sock_type = aiv4->ai_socktype;
		sock_protocol = aiv4->ai_protocol;
		sa_len = aiv4->ai_addrlen;
		(void) memmove( &sa, aiv4->ai_addr, sa_len );
		goto ok;
		}
	    if ( aiv6 != (struct addrinfo*) 0 )
		{
		if ( sizeof(sa) < aiv6->ai_addrlen )
		    {
		    (void) fprintf(
			stderr, "%s - sockaddr too small (%lu < %lu)\n",
			hostname, (unsigned long) sizeof(sa),
			(unsigned long) aiv6->ai_addrlen );
		    exit( 1 );
		    }
		sock_family = aiv6->ai_family;
		sock_type = aiv6->ai_socktype;
		sock_protocol = aiv6->ai_protocol;
		sa_len = aiv6->ai_addrlen;
		(void) memmove( &sa, aiv6->ai_addr, sa_len );
		goto ok;
		}
	
	    send_error( 404, "Not Found", (char*) 0, "Unknown host." );
	
	    ok:
	    freeaddrinfo( ai );
	
	#else /* USE_IPV6 */
	
	    he = gethostbyname( hostname );
	    if ( he == (struct hostent*) 0 )
		send_error( 404, "Not Found", (char*) 0, "Unknown host." );
	    sock_family = sa.sin_family = he->h_addrtype;
	    sock_type = SOCK_STREAM;
	    sock_protocol = 0;
	    sa_len = sizeof(sa);
	    (void) memmove( &sa.sin_addr, he->h_addr, he->h_length );
	    sa.sin_port = htons( Port );
	
	#endif /* USE_IPV6 */
	
	    sockfd = socket( sock_family, sock_type, sock_protocol );
	    if ( sockfd < 0 )
		send_error( 500, "Internal Error", (char*) 0, "Couldn't create 
socket." );
	
	    if ( connect( sockfd, (struct sockaddr*) &sa, sa_len ) < 0 )
		send_error( 503, "Service Unavailable", (char*) 0, "Connection 
refused." );
	
	    return sockfd;
	    }
	
	static headers * get_client_headers(int *content_length, int *flags)
	{
		//char data[MAX_LINELEN];
		int was_via=0;
		int linelen,i=0;
		headers * hdrs;
	
		
		hdrs=(headers*)malloc(sizeof(headers));
		*content_length=-1;
	
	if(LOGGING)
		fprintf(logfile, "[%d] Headers from client:\n",logpid);
	
		
		while (fgets(line, sizeof(line), stdin) != NULL)
		{
			if (strcmp(line, "\n") == 0 || strcmp(line, "\r\n") == 0)
				break;
			
			if((linelen=check_trim(line))==0)
				send_error( 400, "Bad request", (char*) 0, "Line too long.");
	
	//		if (strncasecmp(line,"User-Agent:", 11) == 0)
			if(LOGGING) fprintf(logfile,"[%d]   %s\n",logpid, line);
			if (strncasecmp(line,"Connection:",11) == 0)
					continue;
	
			if (strncasecmp(line,"Proxy-Connection:",17) == 0)
					continue;
		
			if (strncasecmp(line, "Range:", 6) == 0) continue; 
	
			//get content length
			if (strncasecmp(line, "Content-Length:", 15 ) == 0)
				*content_length = atol(&(line[15]));
	
				//can accept gzip?
			else if (strncasecmp(line, "Accept-Encoding:", 16) == 0)
				//we don't know how to uncompress HTML 
				//to modify it -> don't pass this to server
			{
				if(DoGzip) *flags |= WILLGZIP;
				continue; 
			}
	
			if (ServHost && strncasecmp(line,"Via:",4) == 0)
			{
				if(strstr(line,ServHost) && strstr(line, SERVER_NAME)) 
						/*prevent infinite recursion*/
					send_error( 503, "Service Unavailable", (char*) 0, "Connection 
refused." );
				
				if (i<MAX_HEADERS){
					(*hdrs)[i] = malloc(linelen + strlen(ServHost) + 
sizeof(SERVER_NAME) + 10);
					sprintf((*hdrs)[i], "%s, 1.1 %s (%s)", line, ServHost, 
SERVER_NAME);
					i++;
					was_via=1;
					continue;
				}
				else 
					send_error( 400, "Bad request", (char*) 0, "Too many headers.");
				
			}
			if(i<MAX_HEADERS) (*hdrs)[i++]=strdup(line);
			else send_error( 400, "Bad request", (char*) 0, "Too many 
headers.");
			
		}
	
		if(ServHost && !was_via)
		{
			if(i<MAX_HEADERS)
			{
				(*hdrs)[i] = malloc(strlen(ServHost) + sizeof(SERVER_NAME) + 15);
				sprintf((*hdrs)[i],"Via: 1.1 %s (%s)",ServHost , SERVER_NAME);
				i++;
			}
			else	send_error( 400, "Bad request", (char*) 0, "Too many 
headers.");
		}
		for(;i<MAX_HEADERS;i++) (*hdrs)[i]=NULL;
	
		return hdrs;
	}
	
	static void proxy_http( headers* hdrs, char* method, char* path, 
char* protocol,
					FILE* sockrfp, FILE* sockwfp , int content_length, int flags)
	{
		//char data[MAX_LINELEN];
		size_t linelen;
               int lines = 0, status = 0, http_status = 0, ich;
		
		int where_content_type = -1,where_content_length = -1,where_chunked 
= -1;
		int where_content_encoding = -1, where_etag = -1;
		
		long i;
	
		char *tempp;
		int compress = 0;
		int quality = -1;
		int image = IMG_NONE,modify =0;
	
		char *inbuf;
		int inlen;
	
		int bpp,width,height;
	
		unsigned int chunklen;
		unsigned int chunkpos;
		
		is_sending_data = 0;
		// Send request
		fprintf(sockwfp, "%s %s %s\r\n", method, path, protocol);
	
	if(LOGGING) fprintf(logfile, "[%d] Headers sent to server:\n[%d]   
%s %s %s\n",
				logpid, logpid, method, path, protocol);
	
		// Forward the remainder of the request from the client
		for(i=0;((*hdrs)[i]) && i<MAX_HEADERS;i++)
		{
			fputs((*hdrs)[i],sockwfp);		
			if(LOGGING)
				fprintf(logfile, "[%d]   %s\n",logpid, (*hdrs)[i]);
	
			free((*hdrs)[i]);
			fputs("\r\n",sockwfp);
		}
		fputs("\r\n",sockwfp);
	
		// If there's content, forward that too
		if (content_length != -1)
			for (i = 0; i < content_length && (ich = getchar()) != EOF; ++i)
				putc(ich, sockwfp);
		fflush(sockwfp);
	
		if(LOGGING){
			gettimeofday(&newtime,NULL);
			fprintf(logfile, "[%d] connecting, forwarding headers took %ld us 
\n",logpid,
				timeval_subtract_us(&newtime, &oldtime));
			gettimeofday(&oldtime,NULL);
		}
	
	
		// Forward the response back to the client
		content_length = -1;
		status = -1;
	
		//get header
		while (fgets(line, sizeof(line), sockrfp) != 0)
		{
			if (strcmp(line, "\n") == 0 || strcmp(line, "\r\n") == 0)
				break;
	
			if ((linelen = check_trim(line)) == 0)
					send_error(500,"Internal Error",NULL,
							"Too long line in response from server");
			//get content length
			if (strncasecmp(line, "Content-Length:", 15) == 0)
			{
				content_length = atol(&(line[15]));
				where_content_length = lines;
			}
			else if (strncasecmp(line, "Content-Encoding:", 17) == 0)
				where_content_encoding = lines;
			
			else if (strncasecmp(line, "Transfer-Encoding:", 18) == 0)
			{//check for chunking
				tempp = line + 19;
				if (strncasecmp(tempp, "chunked", 7) == 0)
				where_chunked = lines;
			}
			else if (strncasecmp(line, "Content-Type:", 13) == 0)
				where_content_type = lines;
			
			else if (strncasecmp(line, "Location:", 9) == 0)
			{//change suffix in Location: header
				tempp = strrchr(line, '.');
				if(tempp && (tempp + 12 < line + MAX_LINELEN) &&  
								(!strcasecmp(tempp,".gif") || !strcasecmp(tempp,".png")))
						strcpy(tempp + 4, "_or_jpg");
			}
			else if (strncasecmp(line, "Accept-Ranges:", 14) == 0)
					strcpy(&line[14], " none");
					
                       else if (strncasecmp(line, "ETag:", 5) == 0)
				where_etag = lines;

                       else if ((strncasecmp(line, "HTTP/1.1", 8) == 
0) ||
                                (strncasecmp(line, "HTTP/1.0", 8) == 
0))
                           http_status = atoi(&line[8]);

				//store header info
			if (lines < MAX_HEADERS)
			{
				if(strncasecmp(line, "Connection:", 11) && 
				 strncasecmp(line, "Proxy-Connection:", 17))
				{
				(*hdrs)[lines]=strdup(line);
				lines++;
				}
			}else send_error( 500, "Internal Error",(char*)0,"Too many headers 
in response from server");
	
		}
	
		if(where_content_type > 0)
		{
				tempp = (*hdrs)[where_content_type] + 14;
	
				if (!strncasecmp(tempp,"text",4))
				{
						if(DoGzip) compress = 1;
						tempp += 4;
						if(!strncasecmp(tempp,"/html",5))
										modify = 1;
				}
				else if(content_length >= 600 || content_length < 0)
				{
					if (!strncasecmp(tempp, "image/jpeg", 10) ||
						!strncasecmp(tempp, "image/pjpeg", 11) ||
						!strncasecmp(tempp, "image/jpg", 9) ||
						!strncasecmp(tempp, "image/pjpg", 10) 
						)
						image = IMG_JPEG;
					else if (!strncasecmp(tempp, "image/gif", 9))
						image = IMG_GIF;
					else if (!strncasecmp(tempp, "image/png", 9))
						image = IMG_PNG;
				}
				else if(DoGzip && Compressible &&
						(strncasecmp(tempp, "application/",12)==0) && 
						(strlen(tempp) > 15))
				{
				tempp += 12;
					if(strncasecmp(tempp,"x-",2) == 0) tempp+=2;
	
					for (i=0; Compressible[i]; i++)
						if (strncasecmp(tempp,Compressible[i],sizeof(Compressible[i]) - 
1)==0)
						{
							compress = 1;
							break;
						}
				}
		}
	
		if(image && !(flags & SUFF_MODIFIED))
		{
				tempp = strrchr(path,'.');
				if(tempp && (!strcasecmp(tempp,".gif") || 
!strcasecmp(tempp,".png")))
						image =IMG_NONE;
				//we cease to do anything with image. It will confuse browser if
				//content type is other than file suffix.
		}
		
		//Dont gzip, if either:
		//  content is encoded (e.g already compressed)
		//  browser doesn't accept gzip
		if(compress && ((where_content_encoding > 0) || !(flags & 
WILLGZIP)))
		{
			compress = 0;
		}
		
		//Dont modify HTMLs if content is encoded
	if(modify && (where_content_encoding > 0))
		modify = 0;
				
	//log so far
	if(LOGGING){
		gettimeofday(&newtime,NULL);
		fprintf(logfile, "[%d] In Headers:\n",logpid);
		for (i = 0; i < lines; i++)
			fprintf(logfile, "[%d]   %s\n",logpid, (*hdrs)[i]);
		fprintf(logfile, "[%d] Image = %d, Modify = %d, Chunked = %d\n",
				logpid, image, modify, (where_chunked > 0));
	

		if(DoGzip) fprintf(logfile, "[%d] WillGZip = %d, Compress = %d\n",
			logpid, flags & WILLGZIP, compress);

		gettimeofday(&oldtime,NULL);
	}

	//if no data only forward header and exit
       if ((strcasecmp(method, "HEAD") == 0) || (http_status == 304) 
|| (http_status == 302))
	{
               if(LOGGING) fprintf(logfile, "[%d] Forwarding header 
only.\n",logpid);

		//no data, only forward header
		for (i = 0; i < lines; i++)
		{
			fputs((*hdrs)[i], stdout);
			fputs("\r\n",stdout);
		}
		fputs("Connection: close\r\nProxy-Connection: close\r\n\r\n", 
stdout);
		
		//exit
		fflush(stdout);
		return;
	}
	
	//not not workable, stream data and exit	
	if (!compress && !image && !modify)
	{
               if(LOGGING) fprintf(logfile, "[%d] Forwarding header 
and streaming data.\n",logpid);
		//output header
		for (i = 0; i < lines; i++)
		{
			fputs((*hdrs)[i], stdout);
			fputs("\r\n",stdout);
		}
		fputs("Connection: close\r\nProxy-Connection: close\r\n\r\n", 
stdout);

		is_sending_data = 1;
		//stream data through
		for (i = 0; (content_length == -1 || i < content_length) && (ich = 
getc(sockrfp)) != EOF; ++i) {
			fputc(ich, stdout);
			// this code added by knute to prevent timeout when
			// data still coming
			if (!(i % 4096) && ZTimeout)
				alarm(ZTimeout);
		}

		//done
		fflush(stdout);
		return;
	}

	//This error message is used in several branches, so prepare it now.
	snprintf(line, MAX_LINELEN, "Too big file.<br>"
			"Maximal file size allowed for compression/modification"
			"with this proxy is %d kB.", MaxSize/1024);

	//get data and store for compression
	if (MaxSize && (content_length > MaxSize))
		send_error( 500, "Internal Error",(char*)0,line);
			
	if (content_length == -1) inlen = 1024;
	else inlen = content_length;
	
	inbuf=allocstr(inlen + 1);
	
	// unchunk if needed
	if (where_chunked > 0)
	{
		//chunked
		chunkpos = 0;
		fscanf(sockrfp, "%x", &chunklen);
		fgetc(sockrfp); fgetc(sockrfp);

		while (chunklen > 0)
		{
			// this code added by knute to prevent timeout when
			// data still coming
			alarm(ZTimeout);

			if (MaxSize && (chunklen + chunkpos > MaxSize))
				send_error( 500, "Internal Error",(char*)0,line);
					
			for (i = 0; i < chunklen; i++){
				
				if (chunkpos + i >= inlen)
				{
					inbuf=realloc(inbuf,2*inlen + 1);
					inlen*=2;
				}
#ifdef getc_unlocked
				ich = getc_unlocked(sockrfp);
#else
				ich = getc(sockrfp);
#endif
				if (ich == EOF) break;
				
				inbuf[chunkpos + i] = ich;
			}
			chunkpos += i;

			if (ich == EOF) break;

			chunklen = 0;
			fgetc(sockrfp); fgetc(sockrfp);
			fscanf(sockrfp, "%x", &chunklen);
			fgetc(sockrfp); fgetc(sockrfp);
		}

		inlen = chunkpos;
	}
	else
	{
		//not chunked
		if (content_length == -1)
		{
			i = 0;
#ifdef getc_unlocked
			while ((ich = getc_unlocked(sockrfp)) != EOF)
#else
			while ((ich = getc(sockrfp)) != EOF)
#endif
			{
				if (i >= MaxSize)
					send_error( 500, "Internal Error",(char*)0,line);
					
				if (i >= inlen)
				{
					inbuf=realloc(inbuf,2*inlen + 1);
					inlen*=2;
				}
				
				inbuf[i] = ich;
				i++;

				// this code added by knute to prevent timeout
				// when data still coming
				if (!(i % 4096) && ZTimeout)
					alarm(ZTimeout);
			}
			inlen = i;
		}
		else 
		{
			for (i = 0; i < content_length; i++)
			{
#ifdef getc_unlocked
				ich = getc_unlocked(sockrfp);
#else
				ich = getc(sockrfp);
#endif
				
				if(ich == EOF) break;
				
				inbuf[i] = ich;
				//should use recv() or read()
				
				// this code added by knute to prevent timeout
				// when data still coming
				if (!(i % 4096) && ZTimeout)
					alarm(ZTimeout);
			} 
			inlen = i;
		}
	}
if(LOGGING){
	gettimeofday(&newtime,NULL);

	if(inlen != content_length) fprintf(logfile, "[%d] In Content-
Length: %d\n",logpid, inlen);

	fprintf(logfile, "[%d] getting data took %ld us\n",logpid,
			timeval_subtract_us(&newtime,&oldtime));
	oldtime=newtime;
}

	/*if(inlen > 50000) // really needed?
		setpriority(PRIO_PROCESS, logpid, 15);
	else*/ if (inlen < 500)
		image = IMG_NONE; //not worth compressing

	if (image)
	{
		switch(image)
		{
				case IMG_PNG:status = png2bitmap(inbuf, inlen, &tempp, 
								 &width, &height, &bpp);
							 quality = 128;
							 break;
				case IMG_GIF:status = gif2bitmap(inbuf, inlen, &tempp, 
								 &width, &height, &bpp);
							 quality = 128;
							 break;
				case IMG_JPEG:status = jpg2bitmap(inbuf, inlen, &tempp, 
								 &width, &height, &bpp);
							 quality = 0;
							 break;
							 
		}
		if (status)
		{//error, forward unchanged
				outbuf = inbuf;
				outlen = inlen;
		}
		else
		{
			i = width * height;
			if (i < 5000)
				ich = ThumbQ;
			else if ((i < 50000)|| (width < 150) || (height < 150))
		                ich = SmallQ;
			else if (i < 250000)
				ich = MedQ;
		        else ich = LargeQ;

			if (ich == 0){//do nothing
				outbuf = inbuf;
				outlen = inlen;
			}else{
				if (ich < 0){
					ich = -ich;
					rgb2gray(tempp, height, width, &bpp);
				}
				quality += ich;

				//worsest compression ratio allowed 85%(large images) - 90% 
(small)
				outlen = inlen * 0.85 + 40;
					
				status = bitmap2jpg(tempp, width, height, bpp, quality);
				if (status)
				{
					outbuf = inbuf;
					outlen = inlen;
				}
				else
				{//everything OK
					free((*hdrs)[where_content_type]);
					(*hdrs)[where_content_type] = "Content-Type: image/jpeg";
				}
			}
		}
					
	}

	if(modify)
	{
		inbuf[inlen]='\0';
		if((inlen > 10000) && ((protocol[7] == '1') || !UseContentLength))
		{
			int out_type;
			is_sending_data = 1;
			if((protocol[7] == '1') && !compress){ //use chunked
			//some browsers doesn't like chunked with HTTP/1.0
				if((*hdrs)[0][7] == '0') (*hdrs)[0][7]='1';
				out_type = OChunked;		
			}else if(compress)
				out_type = OGzipStream;
				
			else out_type = OStream;
					
			fputs((*hdrs)[0], stdout);

			if(out_type == OChunked) fputs("\r\nTransfer-Encoding: chunked", 
stdout);
			if(out_type == OGzipStream) fputs("\r\nContent-Encoding: gzip", 
stdout);

			fputs("\r\nConnection: close\r\nProxy-Connection: 
close\r\n",stdout);

			if(LOGGING){
				fprintf(logfile, "[%d] While modification, streaming "
						"content. Out Headers:\n",logpid);
				fprintf(logfile, "[%d]   %s\n",logpid, (*hdrs)[0]);
			if(out_type == OChunked)
				fprintf(logfile, "[%d]   Transfer-Encoding: chunked\n", logpid);
			if(out_type == OGzipStream)
				fprintf(logfile, "[%d]   Content-Encoding: gzip\r\n",logpid);
			
				fprintf(logfile, "[%d]   Connection: close\n",logpid);
				fprintf(logfile, "[%d]   Proxy-Connection: close\n",logpid);
			}

			//forward remaining headers
			for (i = 1; i < lines; i++)
			{
				if ((i != where_content_length) &&
				(i != where_chunked)){
					
					if(LOGGING) fprintf(logfile, "[%d]   %s\n",logpid, (*hdrs)[i]);
					
					fputs((*hdrs)[i], stdout);
					fputs("\r\n",stdout);
				}
			}
			fputs("\r\n", stdout);
				
			fflush(stdout);

			status = htmlmodify(inbuf,&outbuf, out_type);
			
			
			if(LOGGING){
			fflush(stdout);
			fsync(1);
			gettimeofday(&newtime,NULL);
			fprintf(logfile, "[%d] mod+streaming took %ld us\n",logpid,
				timeval_subtract_us(&newtime, &oldtime));
			}
			return;
		}else{
			status = htmlmodify(inbuf,&outbuf, ONormal);
			outlen = strlen(outbuf);
		}
	}
	
	if(compress)
	{
		if(modify) {
			inbuf = outbuf;
			inlen = outlen;
		}
		is_sending_data = 1;
		if(UseContentLength || (inlen < 10000)) status += gzip (inbuf, 
inlen, &outbuf, &outlen, ONormal);
		else{
			fputs((*hdrs)[0], stdout);

			fputs("\r\nContent-Encoding: gzip\r\nConnection: close\r\nProxy-
Connection: close\r\n",stdout);

			if(LOGGING){
				fprintf(logfile, "[%d] While compression, streaming "
						"content. Out Headers:\n",logpid);
				fprintf(logfile, "[%d]   Content-Encoding: gzip\r\n",logpid);
				fprintf(logfile, "[%d]   Connection: close\n",logpid);
				fprintf(logfile, "[%d]   Proxy-Connection: close\n",logpid);
			}

			//forward remaining headers
			for (i = 1; i < lines; i++)
			{
				if ((i != where_content_length) &&
				(i != where_chunked)){
					
					if(LOGGING) fprintf(logfile, "[%d]   %s\n",logpid, (*hdrs)[i]);
					fputs((*hdrs)[i], stdout);
					fputs("\r\n",stdout);
				}
			}
			fputs("\r\n", stdout);
				
			fflush(stdout);

			status = gzip(inbuf, inlen, &outbuf, &outlen, OGzipStream);
			
			if(LOGGING){
			fsync(1);
			gettimeofday(&newtime,NULL);
			fprintf(logfile, "[%d] compr+streaming took %ld us\n",logpid,
				timeval_subtract_us(&newtime, &oldtime));
			}
			return;
	
		}
	}
				
	if(LOGGING){
		gettimeofday(&newtime,NULL);
		fprintf(logfile, 
			"[%d] Content modification/compression returned %d. It took %ld 
us.\n",
			logpid, status, timeval_subtract_us(&newtime, &oldtime));
		fprintf(logfile, "[%d] Forwarding header and modified 
content.\n",logpid);
		fprintf(logfile, "[%d] Out Headers:\n",logpid);
		gettimeofday(&oldtime,NULL);
	}
	
	is_sending_data = 1;
//Print first header
	fputs((*hdrs)[0], stdout);
	fputs("\r\n",stdout);

	if((where_content_length > 0) || (outlen > 0))
		fprintf(stdout, "Content-Length: %d\r\n", outlen);
	if(compress) fputs("Content-Encoding: gzip\r\n", stdout);
	fputs("Connection: close\r\nProxy-Connection: close\r\n", stdout);
	
	if(LOGGING){
		fprintf(logfile, "[%d]   %s\n",logpid, (*hdrs)[0]);
		if((where_content_length > 0) || (outlen > 0))
			fprintf(logfile, "[%d]   Content-Length: %d\n",logpid, outlen);
	
	if(compress) fprintf(logfile, "[%d]   Content-Encoding: 
gzip\r\n",logpid);
	
	fprintf(logfile, "[%d]   Connection: close\n", logpid);
	fprintf(logfile, "[%d]   Proxy-Connection: close\n",logpid);
	}
	
	//forward remaining headers
	for (i = 1; i < lines; i++)
	{
		if (	(i != where_content_length) &&
			(i != where_chunked)/* &&
			(!modify || (i != where_etag))//if content was modified, don't 
forward ETag: header*/
			)
		{
			if(LOGGING)fprintf(logfile, "[%d]   %s\n",logpid, (*hdrs)[i]);
			fputs((*hdrs)[i], stdout);
			fputs("\r\n",stdout);
		}
	}
	fputs("\r\n", stdout);

	//forward content
	tempp = outbuf;
	if (inlen != 0)
		do{
			i = fwrite(tempp, 1, outlen, stdout);
			outlen -= i;
			tempp += i;
		}while((i > 0) && outlen);
	fflush(stdout);
	if(LOGGING && (inlen != 0))
	{
		if(outlen == 0)
		{
			fsync(1);
			gettimeofday(&newtime,NULL);
			fprintf(logfile, "[%d] forwarding took %ld us\n",logpid,
				timeval_subtract_us(&newtime, &oldtime));
		}
		else fprintf(logfile,
			"[%d] Error - Last %d bytes of content could not be written\n",
			logpid, outlen);
	}
}


static void
proxy_ssl( char* method, char* host, int Port, char* protocol, FILE* 
sockrfp, FILE* sockwfp )
    {
    int client_read_fd, server_read_fd, client_write_fd, 
server_write_fd;
    struct timeval *timeout = NULL;
    fd_set fdset;
    int maxp1, r;
    char buf[10000];

	free(method);
    /* Return SSL-proxy greeting header. */
    (void) fputs( "HTTP/1.0 200 Connection established\r\n\r\n", 
stdout );
    (void) fflush( stdout );
    /* Now forward SSL packets in both directions until done. */
    client_read_fd = fileno( stdin );
    server_read_fd = fileno( sockrfp );
    client_write_fd = fileno( stdout );
    server_write_fd = fileno( sockwfp );
    
    if(ZTimeout) timeout = malloc(sizeof(struct timeval));
    
    if ( client_read_fd >= server_read_fd )
	maxp1 = client_read_fd + 1;
    else
	maxp1 = server_read_fd + 1;
    (void) alarm( 0 );
    for (;;)
	{
	if(ZTimeout){
		timeout->tv_sec = ZTimeout;
	    	timeout->tv_usec = 0;
	}
	FD_ZERO( &fdset );
	FD_SET( client_read_fd, &fdset );
	FD_SET( server_read_fd, &fdset );

	r = select( maxp1, &fdset, (fd_set*) 0, (fd_set*) 0, timeout );
	if ( r == 0 )
	    send_error( 408, "Request Timeout", (char*) 0, "Request timed 
out." );
	else if ( FD_ISSET( client_read_fd, &fdset ) )
	    {
	    r = read( client_read_fd, buf, sizeof( buf ) );
	    if ( r <= 0 )
		break;
	    r = write( server_write_fd, buf, r );
	    if ( r <= 0 )
		break;
	    }
	else if ( FD_ISSET( server_read_fd, &fdset ) )
	    {
	    r = read( server_read_fd, buf, sizeof( buf ) );
	    if ( r <= 0 )
		break;
	    r = write( client_write_fd, buf, r );
	    if ( r <= 0 )
		break;
	    }
	}
    }


static void
sigcatch( int sig )
    {
    send_error( 408, "Request Timeout", (char*) 0, "Request timed 
out." );
    }

//If it returns line length > 0, line was correctly ended with 
newline - use to check output
//from fgets(). 
static int
check_trim( char* line )
    {
    int k,l;

    k = l = strlen( line );
	
    while (l > 0 && (line[l-1] == '\n' || line[l-1] == '\r'))
		line[--l] = '\0';

	return (k != l)? l : 0;
    }


static void
send_error( int status, char* title, char* extra_header, char* text )
    {
	    if(is_sending_data){
	    //if already sending data, sending error headers is pointless
		    status = -status;
	    }else{
    send_headers( status, title, extra_header, "text/html", -1, -1 );
    (void) printf( "<HTML><HEAD><TITLE>%d %s</TITLE></HEAD>\n<BODY 
BGCOLOR=\"#cc9999\"><H4>%d %s</H4>\n", status, title, status, title 
);
    (void) printf( "%s\n", text );
    (void) printf( "<HR>\n<ADDRESS><A 
HREF=\"%s\">%s</A></ADDRESS>\n</BODY></HTML>\n", ServUrl, SERVER_NAME 
);
    (void) fflush( stdout );
	    }

    //log
if(LOGGING){
	    fprintf(logfile, "[%d] ERROR - %d %s ( %s )\n", logpid, status, 
title, text);
	    fprintf(logfile, "[%d] Finished.\n",logpid);
	    fclose(logfile);
	}
    exit( 1 );
    }


static void
send_headers( int status, char* title, char* extra_header, char* 
mime_type, int length, time_t mod )
    {
    time_t now;
    char timebuf[100];

    (void) printf( "%s %d %s\r\n", PROTOCOL, status, title );
    (void) printf( "Server: %s\r\n", SERVER_NAME );
    now = time( (time_t*) 0 );
    (void) strftime( timebuf, sizeof(timebuf), RFC1123FMT, gmtime( 
&now ) );
    (void) printf( "Date: %s\r\n", timebuf );
    if ( extra_header != (char*) 0 )
	(void) printf( "%s\r\n", extra_header );
    if ( mime_type != (char*) 0 )
	(void) printf( "Content-Type: %s\r\n", mime_type );
    if ( length >= 0 )
	(void) printf( "Content-Length: %d\r\n", length );
    if ( mod != (time_t) -1 )
	{
	(void) strftime( timebuf, sizeof(timebuf), RFC1123FMT, gmtime( &mod 
) );
	(void) printf( "Last-Modified: %s\r\n", timebuf );
	}
    (void) printf( "Connection: close\r\nProxy-Connection: 
close\r\n\r\n" );
    }

char *allocstr(size_t size)
{
	char* str=(char*)malloc(size);
	if(!str) {
		fputs("Virtual memory exhausted!\n",stderr);
		exit(3);
	}
	return str;
}
 

long timeval_subtract_us (struct timeval *x, struct timeval *y)
{
	struct timeval result;
	
        /* Perform the carry for the later subtraction by updating Y. 
*/
        if (x->tv_usec < y->tv_usec) {
          int nsec = (y->tv_usec - x->tv_usec) / 1000000 + 1;
          y->tv_usec -= 1000000 * nsec;
          y->tv_sec += nsec;
        }
        if (x->tv_usec - y->tv_usec > 1000000) {
          int nsec = (x->tv_usec - y->tv_usec) / 1000000;
          y->tv_usec += 1000000 * nsec;
          y->tv_sec -= nsec;
        }
      
        /* Compute the time remaining to wait.
           `tv_usec' is certainly positive. */
        result.tv_sec = x->tv_sec - y->tv_sec;
        result.tv_usec = x->tv_usec - y->tv_usec;
      
        return 1000000*result.tv_sec + result.tv_usec;
}

	
			
-- 
Knute Johnson
Molon Labe...

[Ziproxy-users] squid + ziproxy

[Agustin De I. <aig...@li...>]

Hi list, I have a squid server 2.6-DEVEL-20021219 and i am interested that works in conjuntion with ziproxy, I installed ziproxy at the same server and works fine alone, anybody knows what modifications I must to do in squid.conf? 
-- 
______________________________________________
Check out the latest SMS services @ http://www.linuxmail.org 
This allows you to send and receive SMS through your mailbox.


Powered by Outblaze

[Ziproxy-users] test 2

[Juraj V. <no...@sa...>]

Did it accepted mail from nospam@... ?

[Ziproxy-users] list test

[Juraj V. <va...@na...>]

=A9t okt  2 19:04:17 CEST 2003