rabbit-proxy-users — Normal usage and help

Re: [Rabbit-proxy-users] Resource Growth...

[Robert O. <ro...@kh...>]

Samuel Hill wrote:

>How do we find what is not leaving memory?
>It looks like something happens but it stays in memory.
>That builds and builds until the proxy is OOM.
>  
>
Do you have anything interesting in the error_log?
Do you have anything interesting in the access_log?

What is the current problem is it OOM do it hang?
(you have claimed both at different times in the past).

Can you try it on linux?

Do you have lsof installed? does it say anything?
Do "netstat -a" say anything interesting?

If you can get a GUI up on the machine you can use jmp
or any other profiler (but I do not know of any other that runs on 
FreeBSD).

/robo

RE: [Rabbit-proxy-users] Resource Growth...

[Samuel H. <Sam...@Co...>]

So I guess in my threads thinking out loud...

How do we find what is not leaving memory?
It looks like something happens but it stays in memory.
That builds and builds until the proxy is OOM.


Sam



-----Original Message-----
From: Samuel Hill [mailto:Sam...@Co...] 
Sent: Friday, April 23, 2004 3:18 PM
To: 'Samuel Hill'; 'Robert Olofsson'
Cc: rab...@li...
Subject: RE: [Rabbit-proxy-users] Resource Growth... OOOPS


Ignore that.

Running an older version did that.
The latest stuff it does not do that.
Confusion and I had an older version of Rabbit running.

That's good to know that some improvements have certainly been made.

New version the size does not grow.
It stays at 24616 the whole time.

So the problem is something with usage. 
My mistake.

Sam


-----Original Message-----
From: rab...@li...
[mailto:rab...@li...] On Behalf Of
Samuel Hill
Sent: Friday, April 23, 2004 2:51 PM
To: 'Robert Olofsson'
Cc: rab...@li...
Subject: [Rabbit-proxy-users] Resource Growth...


I have a proxy running on another box with freebsd.
I just wanted to make sure that if it just runs it does not run itself
into the ground if no one is using it. Meaning if there is a leak it has
to do with usage, not simply the proxy just running...

I found that a box not connected to the network and just running the
proxy has the same problem. This means it is not doing anything, just
running the proxy, with nobody connecting to it, it has the same
problem.

ANYWAY...
How can I figure out what is causing the process to grow?

Starts..................
  PID USERNAME PRI NICE   SIZE    RES STATE  C   TIME   WCPU    CPU
COMMAND
51625 root      96    0   274M 23408K select 0   0:18  3.27%  3.27% java

And after exactly 30 minutes...
  PID USERNAME PRI NICE   SIZE    RES STATE  C   TIME   WCPU    CPU
COMMAND
51625 root      97    0   286M 46672K select 0   1:32  4.25%  4.25% java

About double in size.
As you can guess after one hour the proxy needs restarted.
This is an exact number, exactly 30 minutes within a few seconds.

Sam

__________________________________________
Samuel Hill
Network/Systems WestPA.net Inc.
20 Market Street Plaza, Warren, PA 16365
Voice - 877-726-9462    Fax - 814-723-9585
 


-----Original Message-----
From: rab...@li...
[mailto:rab...@li...] On Behalf Of
Robert Olofsson
Sent: Friday, April 23, 2004 2:38 PM
To: Samuel Hill
Cc: rab...@li...
Subject: Re: [Rabbit-proxy-users] Convert and Transparent Images


Samuel Hill wrote:

>Okay, bad idea. PNG is not the way to go, it really is the same as a 
>gif.
>  
>
Well mostly, png are usually a few percent smaller. Png also have full
alpha channels where gif only have one color that may be transparent.
Gif is (up to) 256 (8 bit)colors, png have up to 32 bits (or is that 48 
bits I'm not
sure).

>I guess there is little that can be done.
>You could make the background (fill) of the area the same as the 
>background but would be complex.
>  
>
That would be _really_ complex. What about pages that have images as
background. What about the browsers default background (seems to be
mostly white today, but can be changed depending on browser). What about
the user that only watches an image (as opposed to an image that is part
of a web page).

>Trick being how do you know it was a transparent gif, etc etc.
>  
>
So I do not think that there is much to do about this, provide a link to
the unfiltered page seems to do it for me.

It is not something that I will implement this century anyway.

As for converting images into png files: "convert infile.gif
outfile.png" yes it is that simple.  You could also try something like
"convert infile.jpeg png:outfile.P" if you want any other extension than

png.

Reading the man page for convert will reveal many more nice things that
it can do.

/robo



-------------------------------------------------------
This SF.net email is sponsored by: The Robotic Monkeys at ThinkGeek For
a limited time only, get FREE Ground shipping on all orders of $35 or
more. Hurry up and shop folks, this offer expires April 30th!
http://www.thinkgeek.com/freeshipping/?cpg=12297
_______________________________________________
Rabbit-proxy-users mailing list Rab...@li...
https://lists.sourceforge.net/lists/listinfo/rabbit-proxy-users



-------------------------------------------------------
This SF.net email is sponsored by: The Robotic Monkeys at ThinkGeek For
a limited time only, get FREE Ground shipping on all orders of $35 or
more. Hurry up and shop folks, this offer expires April 30th!
http://www.thinkgeek.com/freeshipping/?cpg=12297
_______________________________________________
Rabbit-proxy-users mailing list Rab...@li...
https://lists.sourceforge.net/lists/listinfo/rabbit-proxy-users

RE: [Rabbit-proxy-users] Resource Growth... OOOPS

[Samuel H. <Sam...@Co...>]

Ignore that.

Running an older version did that.
The latest stuff it does not do that.
Confusion and I had an older version of Rabbit running.

That's good to know that some improvements have certainly been made.

New version the size does not grow.
It stays at 24616 the whole time.

So the problem is something with usage. 
My mistake.

Sam


-----Original Message-----
From: rab...@li...
[mailto:rab...@li...] On Behalf Of
Samuel Hill
Sent: Friday, April 23, 2004 2:51 PM
To: 'Robert Olofsson'
Cc: rab...@li...
Subject: [Rabbit-proxy-users] Resource Growth...


I have a proxy running on another box with freebsd.
I just wanted to make sure that if it just runs it does not run itself
into the ground if no one is using it. Meaning if there is a leak it has
to do with usage, not simply the proxy just running...

I found that a box not connected to the network and just running the
proxy has the same problem. This means it is not doing anything, just
running the proxy, with nobody connecting to it, it has the same
problem.

ANYWAY...
How can I figure out what is causing the process to grow?

Starts..................
  PID USERNAME PRI NICE   SIZE    RES STATE  C   TIME   WCPU    CPU
COMMAND
51625 root      96    0   274M 23408K select 0   0:18  3.27%  3.27% java

And after exactly 30 minutes...
  PID USERNAME PRI NICE   SIZE    RES STATE  C   TIME   WCPU    CPU
COMMAND
51625 root      97    0   286M 46672K select 0   1:32  4.25%  4.25% java

About double in size.
As you can guess after one hour the proxy needs restarted.
This is an exact number, exactly 30 minutes within a few seconds.

Sam

__________________________________________
Samuel Hill
Network/Systems WestPA.net Inc.
20 Market Street Plaza, Warren, PA 16365
Voice - 877-726-9462    Fax - 814-723-9585
 


-----Original Message-----
From: rab...@li...
[mailto:rab...@li...] On Behalf Of
Robert Olofsson
Sent: Friday, April 23, 2004 2:38 PM
To: Samuel Hill
Cc: rab...@li...
Subject: Re: [Rabbit-proxy-users] Convert and Transparent Images


Samuel Hill wrote:

>Okay, bad idea. PNG is not the way to go, it really is the same as a
>gif.
>  
>
Well mostly, png are usually a few percent smaller. Png also have full
alpha channels where gif only have one color that may be transparent.
Gif is (up to) 256 (8 bit)colors, png have up to 32 bits (or is that 48 
bits I'm not
sure).

>I guess there is little that can be done.
>You could make the background (fill) of the area the same as the
>background but would be complex.
>  
>
That would be _really_ complex. What about pages that have images as
background. What about the browsers default background (seems to be
mostly white today, but can be changed depending on browser). What about
the user that only watches an image (as opposed to an image that is part
of a web page).

>Trick being how do you know it was a transparent gif, etc etc.
>  
>
So I do not think that there is much to do about this, provide a link to
the unfiltered page seems to do it for me.

It is not something that I will implement this century anyway.

As for converting images into png files: "convert infile.gif
outfile.png" yes it is that simple.  You could also try something like
"convert infile.jpeg png:outfile.P" if you want any other extension than

png.

Reading the man page for convert will reveal many more nice things that
it can do.

/robo



-------------------------------------------------------
This SF.net email is sponsored by: The Robotic Monkeys at ThinkGeek For
a limited time only, get FREE Ground shipping on all orders of $35 or
more. Hurry up and shop folks, this offer expires April 30th!
http://www.thinkgeek.com/freeshipping/?cpg=12297
_______________________________________________
Rabbit-proxy-users mailing list Rab...@li...
https://lists.sourceforge.net/lists/listinfo/rabbit-proxy-users



-------------------------------------------------------
This SF.net email is sponsored by: The Robotic Monkeys at ThinkGeek For
a limited time only, get FREE Ground shipping on all orders of $35 or
more. Hurry up and shop folks, this offer expires April 30th!
http://www.thinkgeek.com/freeshipping/?cpg=12297
_______________________________________________
Rabbit-proxy-users mailing list Rab...@li...
https://lists.sourceforge.net/lists/listinfo/rabbit-proxy-users

[Rabbit-proxy-users] Resource Growth...

[Samuel H. <Sam...@Co...>]

I have a proxy running on another box with freebsd.
I just wanted to make sure that if it just runs it does not run itself
into the ground if no one is using it.
Meaning if there is a leak it has to do with usage, not simply the proxy
just running...

I found that a box not connected to the network and just running the
proxy has the same problem.
This means it is not doing anything, just running the proxy, with nobody
connecting to it, it has the same problem.

ANYWAY...
How can I figure out what is causing the process to grow?

Starts..................
  PID USERNAME PRI NICE   SIZE    RES STATE  C   TIME   WCPU    CPU
COMMAND
51625 root      96    0   274M 23408K select 0   0:18  3.27%  3.27% java

And after exactly 30 minutes...
  PID USERNAME PRI NICE   SIZE    RES STATE  C   TIME   WCPU    CPU
COMMAND
51625 root      97    0   286M 46672K select 0   1:32  4.25%  4.25% java

About double in size.
As you can guess after one hour the proxy needs restarted.
This is an exact number, exactly 30 minutes within a few seconds.

Sam

__________________________________________
Samuel Hill
Network/Systems WestPA.net Inc.
20 Market Street Plaza, Warren, PA 16365
Voice - 877-726-9462    Fax - 814-723-9585
 


-----Original Message-----
From: rab...@li...
[mailto:rab...@li...] On Behalf Of
Robert Olofsson
Sent: Friday, April 23, 2004 2:38 PM
To: Samuel Hill
Cc: rab...@li...
Subject: Re: [Rabbit-proxy-users] Convert and Transparent Images


Samuel Hill wrote:

>Okay, bad idea. PNG is not the way to go, it really is the same as a 
>gif.
>  
>
Well mostly, png are usually a few percent smaller. Png also have full
alpha channels where gif only have one color that may be transparent.
Gif is (up to) 256 (8 bit)colors, png have up to 32 bits (or is that 48 
bits I'm not
sure).

>I guess there is little that can be done.
>You could make the background (fill) of the area the same as the 
>background but would be complex.
>  
>
That would be _really_ complex. What about pages that have images as
background. What about the browsers default background (seems to be
mostly white today, but can be changed depending on browser). What about
the user that only watches an image (as opposed to an image that is part
of a web page).

>Trick being how do you know it was a transparent gif, etc etc.
>  
>
So I do not think that there is much to do about this, provide a link to
the unfiltered page seems to do it for me.

It is not something that I will implement this century anyway.

As for converting images into png files: "convert infile.gif
outfile.png" yes it is that simple.  You could also try something like
"convert infile.jpeg png:outfile.P" if you want any other extension than

png.

Reading the man page for convert will reveal many more nice things that
it can do.

/robo



-------------------------------------------------------
This SF.net email is sponsored by: The Robotic Monkeys at ThinkGeek For
a limited time only, get FREE Ground shipping on all orders of $35 or
more. Hurry up and shop folks, this offer expires April 30th!
http://www.thinkgeek.com/freeshipping/?cpg=12297
_______________________________________________
Rabbit-proxy-users mailing list Rab...@li...
https://lists.sourceforge.net/lists/listinfo/rabbit-proxy-users

Re: [Rabbit-proxy-users] Convert and Transparent Images

[Robert O. <ro...@kh...>]

Samuel Hill wrote:

>Okay, bad idea. PNG is not the way to go, it really is the same as a gif.
>  
>
Well mostly, png are usually a few percent smaller. Png also have full
alpha channels where gif only have one color that may be transparent.
Gif is (up to) 256 (8 bit)colors, png have up to 32 bits (or is that 48 
bits I'm not
sure).

>I guess there is little that can be done.
>You could make the background (fill) of the area the same as the
>background but would be complex.
>  
>
That would be _really_ complex. What about pages that have images as
background. What about the browsers default background (seems to be
mostly white today, but can be changed depending on browser).
What about the user that only watches an image (as opposed to an image
that is part of a web page).

>Trick being how do you know it was a transparent gif, etc etc.
>  
>
So I do not think that there is much to do about this, provide a link to
the unfiltered page seems to do it for me.

It is not something that I will implement this century anyway.

As for converting images into png files: "convert infile.gif outfile.png"
yes it is that simple.  You could also try something like
"convert infile.jpeg png:outfile.P" if you want any other extension than 
png.

Reading the man page for convert will reveal many more nice
things that it can do.

/robo

RE: [Rabbit-proxy-users] name lookups and dnsjava

[Samuel H. <Sam...@Co...>]

Hmmmmm...

Well, I have it in a folder called "class".
My proxy is in /usr/local/proxy and class is /usr/local/proxy/class

more classpath
CLASSPATH=.:/usr/local/proxy/class/dnsjava-1.6.2.jar

I actually had to manually specify the following in jr, jmake, common,
and classpath.
Something weird, maybe the freebsd issue.
Works now but for some reason there is not an inclusion properly of the
classpath in every file.
I think this is that -f switch that you have in there which must be a
linux thing.


OKAY....
That seems to improve that.
It seems to have improved response time :-)



ANYWAY...
How can I figure out what is causing the process to grow?

Starts..................
  PID USERNAME PRI NICE   SIZE    RES STATE  C   TIME   WCPU    CPU
COMMAND
51625 root      96    0   274M 23408K select 0   0:18  3.27%  3.27% java

And after exactly 30 minutes...
  PID USERNAME PRI NICE   SIZE    RES STATE  C   TIME   WCPU    CPU
COMMAND
51625 root      97    0   286M 46672K select 0   1:32  4.25%  4.25% java

About double in size.
As you can guess after one hour the proxy needs restarted.
This is an exact number, exactly 30 minutes within a few seconds.

Sam




------------------------------------------------------------------------
------------------------

500 Internal Server Error
You have found a bug in RabbIT please report this(together with the URL
you tried to visit) to the RabbIT crew.

Connection status

------------------------------------------------------------------------
--------
status: Handling request
started: Fri Apr 23 09:35:41 EDT 2004
keepalive: true
meta: false
mayusecache: false
maycache: true
mayfilter: true
requestline: GET http://www.westpa.net/ HTTP/1.1
statuscode: 200
extrainfo: null
contentlength: -

Proxy status

------------------------------------------------------------------------
--------
proxy version: RabbIT proxy version 2.0.31
proxy identity: ..__-WestPAnet SpeedLink DialUp-__..
server host: speedlinkdialup-proxy1.westpa.net/65.110.97.42
server port: 500
accessfilters: rabbit.filter.AccessFilter
httpinfilters: rabbit.filter.HTTPBaseFilter 
httpoutfilters:rabbit.filter.HTTPBaseFilter

System properties

------------------------------------------------------------------------
--------
java.version: 1.4.2-p6
java.vendor: Sun Microsystems Inc.
os.name: FreeBSD
os.version: 5.2.1-RELEASE
os.arch: i386
error is:

java.lang.NoClassDefFoundError: org/xbill/DNS/Address
	at rabbit.proxy.Proxy.getInetAddress(Proxy.java:651)
	at
rabbit.io.ConnectionHandler.getConnection(ConnectionHandler.java:136)
	at rabbit.proxy.Proxy.getWebConnection(Proxy.java:242)
	at
rabbit.proxy.Connection.setupWebConnection(Connection.java:1340)
	at rabbit.proxy.Connection.handleRequest(Connection.java:1576)
	at
rabbit.proxy.Connection.filterAndHandleRequest(Connection.java:199)
	at rabbit.proxy.Connection.doWork(Connection.java:293)
	at rabbit.util.RestartableThread.run(RestartableThread.java:28)


-----Original Message-----
From: rab...@li...
[mailto:rab...@li...] On Behalf Of
Robert Olofsson
Sent: Friday, April 23, 2004 9:25 AM
To: Samuel Hill
Cc: rab...@li...
Subject: Re: [Rabbit-proxy-users] name lookups and dnsjava


Samuel Hill wrote:

>I got the following errors on compile...
>
>src/rabbit/proxy/Proxy.java:16: package org.xbill.DNS does not exist 
>import org.xbill.DNS.*;
>  
>
So you have not managed to put the dnsjava-1.6.2.jar in the classpath.
Did you update the classpath file to point out the jar?

You can also do it manually like this: robo@ghoul:~/src/RabbIT2$ javac
-d . -classpath /tmp/dnsjava-1.6.2.jar 
src/rabbit/*/*.java

(depending on where you have put dnsjava-1.6.2.jar of course).

Starting it, there may again be issues with the classpath since you seem

to have
trouble using the jr command (and I do not think that your fix to it was

correct),
I suggest you start it manually like this:

robo@ghoul:~/src/RabbIT2$ java -classpath .:/tmp/dnsjava-1.6.2.jar 
rabbit.proxy.Proxy

If I choose to use dnsjava I will of course make it easier, but for now 
it is still in
testing mode.

/robo


-------------------------------------------------------
This SF.net email is sponsored by: The Robotic Monkeys at ThinkGeek For
a limited time only, get FREE Ground shipping on all orders of $35 or
more. Hurry up and shop folks, this offer expires April 30th!
http://www.thinkgeek.com/freeshipping/?cpg=12297
_______________________________________________
Rabbit-proxy-users mailing list Rab...@li...
https://lists.sourceforge.net/lists/listinfo/rabbit-proxy-users

Re: [Rabbit-proxy-users] name lookups and dnsjava

[Robert O. <ro...@kh...>]

Samuel Hill wrote:

>I got the following errors on compile...
>
>src/rabbit/proxy/Proxy.java:16: package org.xbill.DNS does not exist
>import org.xbill.DNS.*;
>  
>
So you have not managed to put the dnsjava-1.6.2.jar in the classpath.
Did you update the classpath file to point out the jar?

You can also do it manually like this:
robo@ghoul:~/src/RabbIT2$ javac -d . -classpath /tmp/dnsjava-1.6.2.jar 
src/rabbit/*/*.java

(depending on where you have put dnsjava-1.6.2.jar of course).

Starting it, there may again be issues with the classpath since you seem 
to have
trouble using the jr command (and I do not think that your fix to it was 
correct),
I suggest you start it manually like this:

robo@ghoul:~/src/RabbIT2$ java -classpath .:/tmp/dnsjava-1.6.2.jar 
rabbit.proxy.Proxy

If I choose to use dnsjava I will of course make it easier, but for now 
it is still in
testing mode.

/robo

RE: [Rabbit-proxy-users] name lookups and dnsjava

[Samuel H. <Sam...@Co...>]

I got the following errors on compile...

src/rabbit/proxy/Proxy.java:16: package org.xbill.DNS does not exist
import org.xbill.DNS.*;
^
src/rabbit/proxy/Proxy.java:651: cannot resolve symbol
symbol  : variable Address 
location: class rabbit.proxy.Proxy
        return Address.getByName (url.getHost ());
               ^
Note: src/rabbit/handler/MultiPartHandler.java uses or overrides a
deprecated API.
Note: Recompile with -deprecation for details.
2 errors

------------------------------------------------

Then I get bad stuff when I try to start

/usr/local/bin/java -cp /class/dnsjava-1.6.2.jar rabbit.proxy.Proxy
Exception in thread "main" java.lang.NoClassDefFoundError:
rabbit/proxy/Proxy


Sam

-----Original Message-----
From: rab...@li...
[mailto:rab...@li...] On Behalf Of
Robert Olofsson
Sent: Thursday, April 22, 2004 5:24 PM
To: Robert Olofsson
Cc: Samuel Hill; rab...@li...
Subject: Re: [Rabbit-proxy-users] name lookups and dnsjava


Robert Olofsson wrote:

> Hmm. That depends on the name resolver you have. It may be that the
> standard
> version is single-threaded. Then only one java thread can do name 
> lookups at the
> same time (meaning that some request may be finished but no new are 
> started).
>
> It is probably possible to insert dnsjava or a similar lib to get the
> dns lookup to
> be more well behaved.

Ok, using dnsjava is trivial.
I have attached a patch to Proxy.java to enable it if you care to try.

For it to work you also need to download dnsjava:
http://www.dnsjava.org/download/dnsjava-1.6.2.jar

and put it in the classpath:

robo@ghoul:~/src/RabbIT2$ more classpath
CLASSPATH=.:/tmp/dnsjava-1.6.2.jar
robo@ghoul:~/src/RabbIT2$ make code

Then restart it either with the jr command or with something else that
sets the classpath "java -cp /tmp/dnsjava-1.6.2.jar rabbit.proxy.Proxy"

/robo

[Rabbit-proxy-users] ./jr errors Common - FreeBSD

[Samuel H. <Sam...@Co...>]

There are some flags used in common that are not correct for FreeBSD.
The common file should look like this with freebsd...
Just in case someone else has trouble :-)

Just removing the -f (or whatever) switches works
#------------------------------------------------

#!/bin/sh

# set up normal european timezone.
TZ=ECT
export TZ

DOMAIN=`domainname `
HOST=`hostname`

# suppress "no such file..."
SERV=`ls *.$DOMAIN 2>/dev/null`

if [ -f classpath ]
then
    . ./classpath
else
    echo classpath not found.
    CLASSPATH=.
fi



#-----------------------------------------

Re: [Rabbit-proxy-users] name lookups and dnsjava

[Robert O. <ro...@kh...>]

Robert Olofsson wrote:

> Hmm. That depends on the name resolver you have. It may be that the 
> standard
> version is single-threaded. Then only one java thread can do name 
> lookups at the
> same time (meaning that some request may be finished but no new are 
> started).
>
> It is probably possible to insert dnsjava or a similar lib to get the 
> dns lookup to
> be more well behaved.

Ok, using dnsjava is trivial.
I have attached a patch to Proxy.java to enable it if you care to try.

For it to work you also need to download dnsjava:
http://www.dnsjava.org/download/dnsjava-1.6.2.jar

and put it in the classpath:

robo@ghoul:~/src/RabbIT2$ more classpath
CLASSPATH=.:/tmp/dnsjava-1.6.2.jar
robo@ghoul:~/src/RabbIT2$ make code

Then restart it either with the jr command or with something else
that sets the classpath "java -cp /tmp/dnsjava-1.6.2.jar rabbit.proxy.Proxy"

/robo

[Rabbit-proxy-users] Re: Others wait after error...

[Robert O. <ro...@kh...>]

Samuel Hill wrote:

>I noticed the following problem.
>When there is some sort of exception, like below, Rabitt will hold of
>all other request until the retries are completed.
>....
>[19/Apr/2004:20:19:51 GMT][WARN][Could not find the host:
>'java.net.UnknownHostException: www.fafsa.edu.gov: www.fafsa.edu.gov':
>  
>
Hmm. That depends on the name resolver you have. It may be that the 
standard
version is single-threaded. Then only one java thread can do name 
lookups at the
same time (meaning that some request may be finished but no new are 
started).

It is probably possible to insert dnsjava or a similar lib to get the 
dns lookup to
be more well behaved.

/robo

RE: [Rabbit-proxy-users] Convert and Transparent Images

[Samuel H. <Sam...@Co...>]

Okay, bad idea.
PNG is not the way to go, it really is the same as a gif.

I guess there is little that can be done.
You could make the background (fill) of the area the same as the
background but would be complex.
You would have to "examine" tha page, get the color, and write that to
the transparent fill.
Trick being how do you know it was a transparent gif, etc etc.

Sam


-----Original Message-----
From: rab...@li...
[mailto:rab...@li...] On Behalf Of
Samuel Hill
Sent: Thursday, April 22, 2004 12:04 PM
To: rab...@li...
Subject: RE: [Rabbit-proxy-users] Convert and Transparent Images


It appears that png format supports transparency and compression.

I so far have not figured out hor to get convert to convert the image
from one image to png format. I also am not too sure if it would keep
its transparency in the process.

Sam
 


-----Original Message-----
From: rab...@li...
[mailto:rab...@li...] On Behalf Of
Samuel Hill
Sent: Thursday, April 22, 2004 11:47 AM
To: rab...@li...
Subject: [Rabbit-proxy-users] Convert and Transparent Images


When using Rabbit and convert gets a transparent image it will show up
funny on most web pages. This is because when the conversion happens it
uses the background of the original creation, usually white.

I am wondering if there is a way to not convert transparent gifs? I
cannot see any way to detect if it is transparent or not.

Sam



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
Rabbit-proxy-users mailing list Rab...@li...
https://lists.sourceforge.net/lists/listinfo/rabbit-proxy-users



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
Rabbit-proxy-users mailing list Rab...@li...
https://lists.sourceforge.net/lists/listinfo/rabbit-proxy-users

RE: [Rabbit-proxy-users] Convert and Transparent Images

[Samuel H. <Sam...@Co...>]

It appears that png format supports transparency and compression.

I so far have not figured out hor to get convert to convert the image
from one image to png format.
I also am not too sure if it would keep its transparency in the process.

Sam
 


-----Original Message-----
From: rab...@li...
[mailto:rab...@li...] On Behalf Of
Samuel Hill
Sent: Thursday, April 22, 2004 11:47 AM
To: rab...@li...
Subject: [Rabbit-proxy-users] Convert and Transparent Images


When using Rabbit and convert gets a transparent image it will show up
funny on most web pages. This is because when the conversion happens it
uses the background of the original creation, usually white.

I am wondering if there is a way to not convert transparent gifs? I
cannot see any way to detect if it is transparent or not.

Sam



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
Rabbit-proxy-users mailing list Rab...@li...
https://lists.sourceforge.net/lists/listinfo/rabbit-proxy-users

[Rabbit-proxy-users] Convert and Transparent Images

[Samuel H. <Sam...@Co...>]

When using Rabbit and convert gets a transparent image it will show up
funny on most web pages.
This is because when the conversion happens it uses the background of
the original creation, usually white.

I am wondering if there is a way to not convert transparent gifs?
I cannot see any way to detect if it is transparent or not.

Sam

[Rabbit-proxy-users] RE: Image Compression Choices

[Samuel H. <Sam...@Co...>]

We are talking dial-up.
2000 seems cool, maybe even a bit bigger like 8,000-12,000.
I may set that higher myself because the concept makes perfect sense.

Sam
 


-----Original Message-----
From: Robert Olofsson [mailto:ro...@kh...] 
Sent: Thursday, April 22, 2004 10:44 AM
To: Samuel Hill
Subject: Re: [Rabbit-proxy-users] Memory/Proxy problems


Samuel Hill wrote:

>Well I turned the cache back on.
>I set it to 2 gigs and we will see what happens.
>
>I did notice, and this is an old and different problem, that some 
>images are not being compressed, and I do not see why.
>  
>
There are three common cases where rabbit will not convert images:

1) Rabbit will download it and see that it is less than 2000 bytes and
that means that it is to small. So rabbit will let it be. (The redhat
logo 
falls into this category)

2) When converting images rabbit will send the smallest image. Some 
images get
bigger when converted to jpegs and in thoose cases rabbit will serve the

original.

3) There are also some sites that send odd content types for images and 
rabbit will
not know what to do with them.

Case 3 is not very common for what I know, but it depends on the
users...

qvc is probably a mixed case of 2 and 3 (as far as I can see by looking 
at the first page).

Why?
case 1: converting images introduces latencies and we want them to be 
small.
case2: sending more bytes is dumb! If the image also has lower quality 
we are being double dumb!!

The number 2000 was selected a long time ago, maybe it ought to be 
changed, but I think it is quite ok.
Maybe it ought to be a little bigger (to fit nicely into 2
tcp-packages).

/robo

Re: [Rabbit-proxy-users] Memory/Proxy problems

[Robert O. <ro...@kh...>]

Samuel Hill wrote:

>What exactly do you mean by this?
>"Can you open the monitor window and check if many threads seems to be
>waiting on the cache?"
>  
>
If you did run the GUI version of jmp it would be quite obvious.
Since you are not doing that it may be a bit harder.

If you look at this picture (jmp main window):
http://www.khelekore.org/jmp/sc-main-0.36.png

You will see that there is a button named "Monitors"  pressing that
button opens a window that shows all monitors in the jvm. An example is:

http://www.khelekore.org/jmp/sc-monitors-0.36.png

Doing this also checks for deadlocks.

Now since you do not run a gui-version this is hard to do.

/robo

RE: [Rabbit-proxy-users] Memory/Proxy problems

[Samuel H. <Sam...@Co...>]

What exactly do you mean by this?
"Can you open the monitor window and check if many threads seems to be
waiting on the cache?"

I do not understand how to do that?
I could not figure out how to gather that data.

Sam

-----Original Message-----
From: rab...@li...
[mailto:rab...@li...] On Behalf Of
Robert Olofsson
Sent: Monday, April 19, 2004 4:46 PM
To: Samuel Hill
Cc: rab...@li...
Subject: Re: [Rabbit-proxy-users] Memory/Proxy problems


Samuel Hill wrote:

>500 is my maxconnections setting.
>I will increase that though.
>  
>
That ought to be enought.

>I know when I go to status it does not generally show more than 40-50.
>  
>
Yes, from what I can see in the thread dumps, that seems to be the case.

>If there is a wait on cache then it may be the kernel limitations. I 
>will have to look at that.
>  
>
I do not think that you have a kernel limit there, I think it is my code

that
is stupid (probably looping somewhere it should not or something like
that).
 
The cache is synchronized (and needs to be) but perhaps it hold locks 
for to
long, that would fit the thread dump and method dump.

/robo



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
Rabbit-proxy-users mailing list Rab...@li...
https://lists.sourceforge.net/lists/listinfo/rabbit-proxy-users

Re: [Rabbit-proxy-users] Memory/Proxy problems

[Robert O. <ro...@kh...>]

Samuel Hill wrote:

>500 is my maxconnections setting.
>I will increase that though.
>  
>
That ought to be enought.

>I know when I go to status it does not generally show more than 40-50.
>  
>
Yes, from what I can see in the thread dumps, that seems to be the case.

>If there is a wait on cache then it may be the kernel limitations.
>I will have to look at that.
>  
>
I do not think that you have a kernel limit there, I think it is my code 
that
is stupid (probably looping somewhere it should not or something like that).
 
The cache is synchronized (and needs to be) but perhaps it hold locks 
for to
long, that would fit the thread dump and method dump.

/robo

RE: [Rabbit-proxy-users] Memory/Proxy problems

[Samuel H. <Sam...@Co...>]

500 is my maxconnections setting.
I will increase that though.

I know when I go to status it does not generally show more than 40-50.

The server is a dual Zeon 3ghz server with a perc controller (high speed
scsi).
It has plenty of horsepower for what we are doing.

If there is a wait on cache then it may be the kernel limitations.
I will have to look at that.

I will gather more info for you.
I am not running the GUI version of jmp because I do not have any GUI
set up on the server.

Sam


-----Original Message-----
From: rab...@li...
[mailto:rab...@li...] On Behalf Of
Robert Olofsson
Sent: Monday, April 19, 2004 4:28 PM
To: Samuel Hill
Cc: rab...@li...
Subject: Re: [Rabbit-proxy-users] Memory/Proxy problems


Samuel Hill wrote:

>I am still seeing a leak....
>  
>
>I think we can see about 1 MB a minute growth in ram usage.
>As it goes on and on response times from the proxy go way down. 
>Eventually, when the RAM limit is reached, there will be 5-10 seconds 
>response times for pages to be delivered.
>  
>
Ok, noted.

One question what is your "maxconnections" setting?
I guess that most users use IE/5 or IE/6 and last time I checked each IE

will open 8
concurrent connections to each host. The default value is 500, and the
thread dumps I have seen does not go that high. If you have to few 
connections your
clients will have to wait until a connection has timed out.

Anyway looking at the jmp_dump there seems to be a lot of threads that
are waiting for the cache, so it may be that the cache is to slow or to 
heavily
synchronized.

So one thing that would be interesting to get is a monitor dump (hmm,
jmp does not write such files, yet). Can you open the monitor window and
check if many threads seems to be waiting on the cache?

>I get stuff like below but what do you really need?
>jmp/0.41 initializing: (nomethods,nomonitors,noobjects,nogui):...
>  
>
That looks like normal output. Since jmp is still in development (note 
the 0 at the
beginning of the version) it still outputs some debug information on the

console.

The object dump that you did send me will be a good first start to look
at. Did you run GC before you generated that file?

 From the jmp-dump I see:
Total    80335    354882    5584184    1262985
char[]    13117    54480    2087760    206557
java.lang.String    13875    59323    333000    212557
rabbit.http.GeneralHeader$Header    9795    23007    156720    41357
rabbit.http.HTTPHeader    1100    2561    44000    4874

So the java heap seems to be ~5.5 MB, not very big.
And the biggest entry is char[] (as expected).

One thing that would be interesting is if you could genereate a string
dump. _NOTE_ this file may contain sensitive information so think twice
before 
you
release it. If you dont want to release it, can you at least see if 
there are many
duplicates? (I would guess that that is not an issue, but your ouput 
seems a bit odd in
regarding the number of strings, but if you did not do a GC before you 
generated this
file that is normal).

Looking at the methods we can see that addCacheStream and addEntry are 
the top most
entries (except wait, but wait is meant to take time). So this would 
also seem to
give the cache as the bottle neck.

I will look more on this tomorrow, I am to tired now (after a hard 
karate training session).
 
/robo



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
Rabbit-proxy-users mailing list Rab...@li...
https://lists.sourceforge.net/lists/listinfo/rabbit-proxy-users

Re: [Rabbit-proxy-users] Memory/Proxy problems

[Robert O. <ro...@kh...>]

Samuel Hill wrote:

>I am still seeing a leak....
>  
>
>I think we can see about 1 MB a minute growth in ram usage.
>As it goes on and on response times from the proxy go way down.
>Eventually, when the RAM limit is reached, there will be 5-10 seconds
>response times for pages to be delivered.
>  
>
Ok, noted.

One question what is your "maxconnections" setting?
I guess that most users use IE/5 or IE/6 and last time I checked each IE 
will open 8
concurrent connections to each host. The default value is 500, and the
thread dumps I have seen does not go that high. If you have to few 
connections your
clients will have to wait until a connection has timed out.

Anyway looking at the jmp_dump there seems to be a lot of threads that
are waiting for the cache, so it may be that the cache is to slow or to 
heavily
synchronized.

So one thing that would be interesting to get is a monitor dump (hmm,
jmp does not write such files, yet). Can you open the monitor window and
check if many threads seems to be waiting on the cache?

>I get stuff like below but what do you really need?
>jmp/0.41 initializing: (nomethods,nomonitors,noobjects,nogui):...
>  
>
That looks like normal output. Since jmp is still in development (note 
the 0 at the
beginning of the version) it still outputs some debug information on the 
console.

The object dump that you did send me will be a good first start to look at.
Did you run GC before you generated that file?

 From the jmp-dump I see:
Total    80335    354882    5584184    1262985
char[]    13117    54480    2087760    206557
java.lang.String    13875    59323    333000    212557
rabbit.http.GeneralHeader$Header    9795    23007    156720    41357
rabbit.http.HTTPHeader    1100    2561    44000    4874

So the java heap seems to be ~5.5 MB, not very big.
And the biggest entry is char[] (as expected).

One thing that would be interesting is if you could genereate a string dump.
_NOTE_ this file may contain sensitive information so think twice before 
you
release it. If you dont want to release it, can you at least see if 
there are many
duplicates? (I would guess that that is not an issue, but your ouput 
seems a bit odd in
regarding the number of strings, but if you did not do a GC before you 
generated this
file that is normal).

Looking at the methods we can see that addCacheStream and addEntry are 
the top most
entries (except wait, but wait is meant to take time). So this would 
also seem to
give the cache as the bottle neck.

I will look more on this tomorrow, I am to tired now (after a hard 
karate training session).
 
/robo

[Rabbit-proxy-users] Others wait after error...

[Samuel H. <Sam...@Co...>]

I noticed the following problem.
When there is some sort of exception, like below, Rabitt will hold of
all other request until the retries are completed.
For example, while the following was happening no other pages were being
served.
So the host could not be reached and Rabitt just hung until it spit out
the following errors and then continued.

Sam

[19/Apr/2004:20:19:51 GMT][WARN][Could not find the host:
'java.net.UnknownHostException: www.fafsa.edu.gov: www.fafsa.edu.gov':
GET http://www.fafsa.edu.gov/ HTTP/1.1
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg,
application/x-shockwave-flash, */*
Accept-Language: en-us
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows 98)
Host: www.fafsa.edu.gov
Via: HTTP/1.1 RabbIT

]
[19/Apr/2004:20:19:51 GMT][WARN][Failed to connect
socket:java.io.IOException: Could not connect to host:
java.net.UnknownHostException: www.fafsa.edu.gov: www.fafsa.edu.gov]
[19/Apr/2004:20:19:51 GMT][WARN][Could not find the host:
'java.net.UnknownHostException: www.fafsa.edu.gov':
GET http://www.fafsa.edu.gov/ HTTP/1.1
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg,
application/x-shockwave-flash, */*
Accept-Language: en-us
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows 98)
Host: www.fafsa.edu.gov
Via: HTTP/1.1 RabbIT

]
[19/Apr/2004:20:19:51 GMT][WARN][Failed to connect
socket:java.io.IOException: Could not connect to host:
java.net.UnknownHostException: www.fafsa.edu.gov]
[19/Apr/2004:20:19:51 GMT][WARN][Could not find the host:
'java.net.UnknownHostException: www.fafsa.edu.gov':
GET http://www.fafsa.edu.gov/ HTTP/1.1
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg,
application/x-shockwave-flash, */*
Accept-Language: en-us
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows 98)
Host: www.fafsa.edu.gov
Via: HTTP/1.1 RabbIT

]
[19/Apr/2004:20:19:51 GMT][WARN][Failed to connect
socket:java.io.IOException: Could not connect to host:
java.net.UnknownHostException: www.fafsa.edu.gov]
[19/Apr/2004:20:19:51 GMT][WARN][Could not find the host:
'java.net.UnknownHostException: www.fafsa.edu.gov':
GET http://www.fafsa.edu.gov/ HTTP/1.1
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg,
application/x-shockwave-flash, */*
Accept-Language: en-us
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows 98)
Host: www.fafsa.edu.gov
Via: HTTP/1.1 RabbIT

]
[19/Apr/2004:20:19:51 GMT][WARN][Failed to connect
socket:java.io.IOException: Could not connect to host:
java.net.UnknownHostException: www.fafsa.edu.gov]
[19/Apr/2004:20:19:51 GMT][WARN][Could not find the host:
'java.net.UnknownHostException: www.fafsa.edu.gov':
GET http://www.fafsa.edu.gov/ HTTP/1.1
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg,
application/x-shockwave-flash, */*
Accept-Language: en-us
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows 98)
Host: www.fafsa.edu.gov
Via: HTTP/1.1 RabbIT

]
[19/Apr/2004:20:19:51 GMT][WARN][Failed to connect
socket:java.io.IOException: Could not connect to host:
java.net.UnknownHostException: www.fafsa.edu.gov]
[19/Apr/2004:20:19:51 GMT][WARN][strange error setting up web
connection: java.io.IOException: Unable to connect to URL:
http://www.fafsa.edu.gov/]
[19/Apr/2004:20:19:52 GMT][WARN][expire date in the past: '0']
[19/Apr/2004:20:19:54 GMT][WARN][expire date in the past: 'Sun, 04 Apr
2004 21:06:22 GMT']

[Rabbit-proxy-users] Memory/Proxy problems

[Samuel H. <Sam...@Co...>]

I am still seeing a leak.
I do not get to an OOM error but the proxy runs slower and slower with
time.
The CPU usage does not go up but the RAM usage goes up.

I have jmp running...
/usr/local/bin/java -Xrunjmp:nomethods,nomonitors,noobjects,nogui
rabbit.proxy.Proxy
But am not sure what I am looking for.

I think we can see about 1 MB a minute growth in ram usage.
As it goes on and on response times from the proxy go way down.
Eventually, when the RAM limit is reached, there will be 5-10 seconds
response times for pages to be delivered.

I get stuff like below but what do you really need?
jmp/0.41 initializing: (nomethods,nomonitors,noobjects,nogui):...
    tracing objects: false
    tracing methods: false
    tracing monitors: false
    showing gui: false
    dump/reset by signal allowed: false
jmp: Enabling localization.
jmp: Loaded and registered correctly.
java.net.SocketTimeoutException: Read timed out
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.read(SocketInputStream.java:129)
        at
java.io.BufferedInputStream.fill(BufferedInputStream.java:183)
        at
java.io.BufferedInputStream.read1(BufferedInputStream.java:222)
        at
java.io.BufferedInputStream.read(BufferedInputStream.java:277)
        at java.io.DataInputStream.read(DataInputStream.java:170)
        at rabbit.proxy.Connection.handleSSL(Connection.java:1948)
        at
rabbit.proxy.Connection.checkAndHandleSSL(Connection.java:139)
        at rabbit.proxy.Connection.doWork(Connection.java:252)
        at rabbit.util.RestartableThread.run(RestartableThread.java:28)
arena_new: 1, Eden
arena_new: 2, Semi
arena_new: 3, Semi
arena_new: 1, Eden
arena_new: 2, Semi
arena_new: 3, Semi


Sam
 


-----Original Message-----
From: rab...@li...
[mailto:rab...@li...] On Behalf Of
Robert Olofsson
Sent: Saturday, April 17, 2004 4:12 PM
To: Sam
Cc: rab...@li...
Subject: Re: [Rabbit-proxy-users] FreeBSD 5.2 and resource problems


Sam wrote:

> The resource problem appears to be the linux compatability and linux
> java.

Yes making sure that programs work well on many systems is hard. Most of
all since it requires a lot of testing.

I take it that rabbit is running quite well for the time being then,
good! I am sure you will find some sites that cause more problems,
hopefully most of them can be fixed by not filtering the pages, but I
still want to know so I can make rabbit better.

> I also got jmp to work!

Ok, that is good to know.

> I will keep watching and see what else happens.
> If I get more web sites that cause problems or other oddities I will
> let you know.

Not if, when :-) and if you dont find any bugs then reporting success is

good too.

Have fun
/robo



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
Rabbit-proxy-users mailing list Rab...@li...
https://lists.sourceforge.net/lists/listinfo/rabbit-proxy-users

Re: [Rabbit-proxy-users] FreeBSD 5.2 and resource problems

[Robert O. <ro...@kh...>]

Sam wrote:

> The resource problem appears to be the linux compatability and linux 
> java.

Yes making sure that programs work well on many systems is hard. Most of
all since it requires a lot of testing.

I take it that rabbit is running quite well for the time being then, good!
I am sure you will find some sites that cause more problems, hopefully
most of them can be fixed by not filtering the pages, but I still want to
know so I can make rabbit better.

> I also got jmp to work!

Ok, that is good to know.

> I will keep watching and see what else happens.
> If I get more web sites that cause problems or other oddities I will 
> let you know.

Not if, when :-) and if you dont find any bugs then reporting success is 
good too.

Have fun
/robo

[Rabbit-proxy-users] FreeBSD 5.2 and resource problems

[Sam <Sam...@Co...>]

I think I may have figured out the rest.
The resource problem appears to be the linux compatability and linux java.
FreeBSD being truly UNIX is tough when many apps are written around 
linux, not UNIX.

The answer is to make sure the jdk14 from the ports is installed (Not 
linux-jdk14).
There is actually a bug in ports/jdk14 with FreeBSD 5.2 ports tree.
Remove the first line of ...
/usr/ports/java/jdk14/work/control/build/bsd-i586/gensrc/java/util/CurrencyData.java
and compile/install that way.
Then start rabbit and away ya go.

I was able to start out fresh and get the "pure" java that was for FreeBSD.
This was instead of using the linux compatability.
Removing all java programs that were compiled with linux compatability 
and then building java the pure FreeBSD way fixed the problem.
(Not that I don't like Linux)

I also got jmp to work!
It should not be needed right now but who knows.

I will keep watching and see what else happens.
If I get more web sites that cause problems or other oddities I will let 
you know.

Thank you.
Sam

Re: [Rabbit-proxy-users] Unable to open certain sites.

[Robert O. <ro...@kh...>]

Samuel Hill wrote:

>Even if I specify to not cache and not filter .js I still cannot get
>Rabbit to open the following pages.
>
>www.excite.com
>http://www.247realmedia.com/index.html
>  
>

Ok, I have found something.
www.excite.com sends some broken http. More specifically it sends a 
header that looks like:

Cache-control: must-revalidate"

(notice the quote character at the end!).

This causes rabbit to try to fill up the string so it reads the next 
line and appends that
until it can match an even number of quotes.
This does not happen for this page....

Attached is a patch that ignores appending http headers unless rabbit is 
run in
strict mode.
With this patch both of the given sites seems to load correct (as far as 
I can tell).

/robo