[Apt-proxy-users] Further first impressions of apt-proxy-v2

[Craig F. <cr...@wi...>]

Okay, I've had a couple more evenings to sit down and play with apt-proxy-
v2.  I figured out a couple of things (such as the existence 
of /etc/init.d/apt-proxy-v2 (-; ), and I have also come across a few 
oddities.  Some of these I think I mentioned in my last email to the list, 
but I'll elaborate more here:

1)  apt-proxy-v2 does not seem to serve files from its own cache to clients.  
This happens for Packages.gz and Release files as well as packages.  The 
attached file (getpackages) contains the log messages for the second of 
two "apt-get update"'s.  The second "apt-get update" was executed immedately 
after the twistd process had gone idle from the first one, and 
min_refresh_delay is set to 1d in /etc/apt-proxy/apt-proxy-v2.conf.  
Therefore, apt-proxy-v2 should have served up Packages.gz and Release from 
its cache, but instead it downloaded them again from the backend and sent 
them to the client (again).  Lines which may be relevant or important include:

Line 10 (note the two very different dates):
06/08/2002 00:14 [AptProxy,0,127.0.0.1] [debug:9]If-Modified-Since: Fri,
19 Jul 2002 19:03:33 GMT

Lines 15-20:
06/08/2002 00:14 [AptProxy,0,127.0.0.1] [debug:9]CHECKING_CACHED
06/08/2002 00:14 [-] [verify:9]Process Status: -1
06/08/2002 00:14 [-] [verify:9]unknown file: not verified
06/08/2002 00:14 [-]
06/08/2002 00:14 [-] [verify:9]verication failed
06/08/2002 00:14 [-] [debug:9]NOT_CACHED

These messages were for the retrieval of Packages.gz.  Similar messages exist 
for Release as well.  Any idea of where to start looking to debug this one?


2)  apt-proxy-v2 gives 403 Forbidden responses for certain files.  For 
instance, trying to do an "apt-get install gs" always yields:

Err http://localhost woody/main gs 6.53-3
  403 Forbidden
Failed to fetch http://localhost:8000/debian/pool/main/g/gs/gs_6.53-
3_i386.deb  403 Forbidden

This behavior is the same for both sid and woody on multiple backend 
servers.  The file in question does exist and is accessible via HTTP on the 
backend servers.  The apt-proxy-v2 log shows the following:

06/08/2002 00:39 [AptProxy,2,127.0.0.1] [debug:9]Connection: keep-alive
06/08/2002 00:39 [AptProxy,2,127.0.0.1] [debug:9]User-Agent: Debian
APT-HTTP/1.3
06/08/2002 00:39 [AptProxy,2,127.0.0.1] [debug:9]
06/08/2002 00:39 [AptProxy,2,127.0.0.1] [debug:9]/../ in simplified uri
06/08/2002 00:39 [AptProxy,2,127.0.0.1] (harmless warning): discarding
zero-length data for request <GET
/debian/pool/main/g/gs/gs_6.53-3_i386.deb HTTP/1.1>
06/08/2002 00:39 [AptProxy,2,127.0.0.1] [debug:9]Client connection closed

Note that the "simplified uri" and "harmless warning" lines do not occur in 
the logs for successful transfers.


3)  apt-roxy-vs fails to start upon installation.  I didn't save the actual 
error message, but I seem to remember that there was a permissions error 
relating to the log file.  I also remember that the aptproxy user was created 
after that error.  Perhaps this order is reversed?  At any rate, doing 
an "/etc/init.d/apt-proxy-v2 start" after the install starts it up normally.  
I will try to take the time to do a complete uninstall and reinstall sometime 
soon to get the details of all of this.


4)  twistd eats a lot of CPU cycles for a rather long time after downloads 
are complete and the client has disconnected.  What is it doing?


5)  I see mention of memory leaks in the TODO file, so I assume everyone 
knows about this, but:  Memory usage by twistd gradually grows over time as 
apt-proxy-v2 serves more and more requests.  My system finally OOM'ed and the 
vm killed twistd after a couple of "apt-get install -d kde"'s on my firewall 
machine (which does not have any of X installed).


6)  Does apt-proxy-v2 really NEED to be a daemon that runs constantly (as 
opposed to a daemon controlled by inet.d).  I understand that there is 
maintenance to be done on the cache, but it would seem that all of that could 
be handled at the end of requests and/or by cron scripts.  Likewise, I 
understand that there may a slight performance degradation if the process 
needs to be started by inet.d, but for small sites, having the large daemon 
in memory all the time is overkill.  Perhaps there could be an option to 
either run apt-proxy-v2 as a daemon or from inet.d (as is done in the samba 
and apache packages).  Running apt-proxy-v2 from inet.d would also help 
lessen the effects of the apparent memory leak in apt-proxy-v2 since a new 
daemon would be started with each request and ended afterwards.  This is 
similar in principle to the way apache kills off child processes after they 
have served so many pages in order to keep memory leaks under control.  What 
am I missing here that necessitates having twistd running all the time?

-- 
Craig Foster
cr...@wi...