apt-proxy-users

[Apt-proxy-users] cache cleaning does not work

[Christian K. <ch...@ck...>]

Hello,

[please CC: me, I'm not on this list]

I started using 1.9.17 on Debian Sarge and later have updated to version
1.9.28. I used apt-proxy-import to include 5 GB of deb files. Apt-proxy
works so far and saves me traffic. Thanks!

But cache cleaning does not work. In my config I have

--->--->---
cleanup_freq = 2h
max_age = off
max_versions = 2
dynamic_backends = off
---<---<---

and in my log:

2005/04/09 13:52 CEST [-] [debug] Doing periodic cleaning up
2005/04/09 13:52 CEST [-] [debug] Periodic cleaning done

There are a lot multiple package versions in the cache, but from du I can
say, it does not delete any of them. For instance:

karl:/var/cache/apt-proxy# ls -l debian/pool/main/e/exim/
total 2994
-rw-r--r--  1 aptproxy root 758074 May  6  2004 exim_3.36-11_i386.deb
-rw-r--r--  1 aptproxy root 757532 Jan  9 13:32 exim_3.36-13_i386.deb
-rw-r--r--  1 aptproxy root 758742 Mar  3 23:02 exim_3.36-14_i386.deb
-rw-r--r--  1 aptproxy root 781630 Dec 11  2003 exim_3.36-9.1_i386.deb
karl:/var/cache/apt-proxy# ls -l --time=atime debian/pool/main/e/exim/
total 2994
-rw-r--r--  1 aptproxy root 758074 May  6  2004 exim_3.36-11_i386.deb
-rw-r--r--  1 aptproxy root 757532 Jan  9 13:32 exim_3.36-13_i386.deb
-rw-r--r--  1 aptproxy root 758742 Mar  3 23:02 exim_3.36-14_i386.deb
-rw-r--r--  1 aptproxy root 781630 Dec 11  2003 exim_3.36-9.1_i386.deb

(Rem: I had to use a backup at some point and correct the atimes manually)

From my understanding, it should only keep 2 versions of them and delete the
others. What's wrong here?

Christian

-- 
Christian Knoke            * * *            http://cknoke.de
* * * * * * * * *  Ceterum censeo Microsoft esse dividendum.

Re: [Apt-proxy-users] cache cleaning does not work

[Chris H. <ha...@de...>]

Hi, it took a while to get this fixed

On Saturday 09 Apr 2005 13:27, Christian Knoke wrote:
> [please CC: me, I'm not on this list]
>
> But cache cleaning does not work. In my config I have
>
> --->--->---
> cleanup_freq = 2h
> max_age = off
> max_versions = 2

max_versions was broken for a while, sorry.  I have uploaded 1.9.30 to 
unstable which fixes this; please upgrade.

Chris

Re: [Apt-proxy-users] cache cleaning does not work

[Christian K. <ch...@ck...>]

Hello,

[please CC: me, I'm not on this list]

On Fri, May 20, 2005 at 08:31:02AM +0100, Chris Halls wrote:

> On Saturday 09 Apr 2005 13:27, Christian Knoke wrote:
> >
> > But cache cleaning does not work. In my config I have
> >
> > --->--->---
> > cleanup_freq = 2h
> > max_age = off
> > max_versions = 2
> 
> max_versions was broken for a while, sorry.  I have uploaded 1.9.30 to 
> unstable which fixes this; please upgrade.

Thank you. I installed 1.9.30, but it still doesn't clean the cache (to tell
from du).

Here are the logs (with comments):

2005/05/20 12:07 CEST [-] Log opened.
2005/05/20 12:07 CEST [-] twistd 1.3.0rc1 (/usr/bin/python2.3 2.3.5) starting up
2005/05/20 12:07 CEST [-] reactor class: twisted.internet.default.SelectReactor
2005/05/20 12:07 CEST [-] Loading /usr/sbin/apt-proxy...
2005/05/20 12:07 CEST [-] Updating twisted's process module.
2005/05/20 12:07 CEST [-] No updating required.
2005/05/20 12:07 CEST [-] Loaded.
2005/05/20 12:07 CEST [-] apt_proxy.apt_proxy.Factory starting on 8086
2005/05/20 12:07 CEST [-] Starting factory <apt_proxy.apt_proxy.Factory instance at 0xb7b12c0c>
2005/05/20 12:07 CEST [-] set uid/gid 104/65534
2005/05/20 12:07 CEST [-] [debug] Verifying database: /var/cache/apt-proxy/.apt-proxy/db/access.db
2005/05/20 12:07 CEST [-] [db] /var/cache/apt-proxy/.apt-proxy/db/access.db could not be opened, moved to /var/cache/apt-proxy/.apt-proxy/db/access.db.error
2005/05/20 12:07 CEST [-] [db] Recreating /var/cache/apt-proxy/.apt-proxy/db/access.db
2005/05/20 12:07 CEST [-] [debug] Opening database /var/cache/apt-proxy/.apt-proxy/db/access.db
2005/05/20 12:07 CEST [-] [recycle] Adopting new file: /debian-non-US/dists/sarge/non-US/non-free/binary-i386/Packages.gz
2005/05/20 12:07 CEST [-] [recycle] Adopting new file: /debian-non-US/dists/sarge/non-US/non-free/binary-i386/Packages

[8530 lines skipped]

2005/05/20 14:52 CEST [-] [debug] Doing periodic cleaning up
2005/05/20 14:52 CEST [-] [debug] Periodic cleaning done

It doesn't seem to take any attempt to clean up.

[8 lines skipped]

2005/05/20 14:52 CEST [-] [recycle] Adopting new file: /debian/pool/main/a/apmd/libapm1_3.2.1-4_i386.deb
2005/05/20 14:52 CEST [-] [recycle] Adopting new file: /debian/pool/main/a/apmd/libapm1_3.2.2-3_i386.deb
2005/05/20 14:52 CEST [-] [recycle] Adopting new file: /debian/pool/main/a/apmd/libapm1_3.2.2-2_i386.deb
2005/05/20 14:52 CEST [-] [recycle] Adopting new file: /debian/pool/main/a/apmd/libapm1_3.2.2-1_i386.deb

2005/05/20 14:52 CEST [-] [recycle] Adopting new file: /debian/pool/main/a/a2ps/a2ps_4.13b-4.3_i386.deb
2005/05/20 14:52 CEST [-] [recycle] Adopting new file: /debian/pool/main/a/apt/apt-utils_0.5.28.1_i386.deb
2005/05/20 14:52 CEST [-] [recycle] Adopting new file: /debian/pool/main/a/apt/apt-utils_0.5.25_i386.deb
2005/05/20 14:52 CEST [-] [recycle] Adopting new file: /debian/pool/main/a/apt/apt_0.5.25_i386.deb
2005/05/20 14:52 CEST [-] [recycle] Adopting new file: /debian/pool/main/a/apt/apt_0.5.28.6_i386.deb
2005/05/20 14:52 CEST [-] [recycle] Adopting new file: /debian/pool/main/a/apt/apt-utils_0.5.24_i386.deb
2005/05/20 14:52 CEST [-] [recycle] Adopting new file: /debian/pool/main/a/apt/apt_0.5.24_i386.deb
2005/05/20 14:52 CEST [-] [recycle] Adopting new file: /debian/pool/main/a/apt/apt_0.5.28.1_i386.deb
2005/05/20 14:52 CEST [-] [recycle] Adopting new file: /debian/pool/main/a/apt/apt-utils_0.5.23_i386.deb
2005/05/20 14:52 CEST [-] [recycle] Adopting new file: /debian/pool/main/a/apt/apt-utils_0.5.27_i386.deb
2005/05/20 14:52 CEST [-] [recycle] Adopting new file: /debian/pool/main/a/apt/apt_0.5.23_i386.deb
2005/05/20 14:53 CEST [-] [recycle] Adopting new file: /debian/pool/main/a/apt/apt_0.5.27_i386.deb
2005/05/20 14:53 CEST [-] [recycle] Adopting new file: /debian/pool/main/a/apt/apt-utils_0.5.28.6_i386.deb
2005/05/20 14:53 CEST [-] [recycle] Adopting new file: /debian/pool/main/a/apt/apt-utils_0.5.26_i386.deb
2005/05/20 14:53 CEST [-] [recycle] Adopting new file: /debian/pool/main/a/apt/apt_0.5.26_i386.deb
2005/05/20 14:53 CEST [-] [recycle] Adopting new file: /debian/pool/main/a/acl/libacl1_2.2.23-1_i386.deb
2005/05/20 14:53 CEST [-] [recycle] Adopting new file: /debian/pool/main/3/3dchess/3dchess_0.8.1-11_i386.deb
2005/05/20 15:07 CEST [-] [debug] Doing periodic cleaning up
2005/05/20 15:07 CEST [-] [debug] Periodic cleaning done

[14 lines skipped]

At this time, the cache was still the same size. I decided to restart
apt-proxy. This seems to trigger some activity:

2005/05/20 17:05 CEST [-] Received SIGTERM, shutting down.
2005/05/20 17:05 CEST [-] (Port 8086 Closed)
2005/05/20 17:05 CEST [-] Stopping factory <apt_proxy.apt_proxy.Factory instance at 0xb7b12c0c>
2005/05/20 17:05 CEST [-] [debug] Verifying database: /var/cache/apt-proxy/.apt-proxy/db/update.db
2005/05/20 17:05 CEST [-] [debug] Opening database /var/cache/apt-proxy/.apt-proxy/db/update.db
2005/05/20 17:05 CEST [-] [debug] Verifying database: /var/cache/apt-proxy/.apt-proxy/db/packages.db
2005/05/20 17:05 CEST [-] [debug] Opening database /var/cache/apt-proxy/.apt-proxy/db/packages.db
2005/05/20 17:05 CEST [-] Main loop terminated.
2005/05/20 17:05 CEST [-] Server Shut Down.
2005/05/20 17:05 CEST [-] Log opened.
2005/05/20 17:05 CEST [-] twistd 1.3.0rc1 (/usr/bin/python2.3 2.3.5) starting up
2005/05/20 17:05 CEST [-] reactor class: twisted.internet.default.SelectReactor
2005/05/20 17:05 CEST [-] Loading /usr/sbin/apt-proxy...
2005/05/20 17:05 CEST [-] Updating twisted's process module.
2005/05/20 17:05 CEST [-] No updating required.
2005/05/20 17:05 CEST [-] Loaded.
2005/05/20 17:05 CEST [-] apt_proxy.apt_proxy.Factory starting on 8086
2005/05/20 17:05 CEST [-] Starting factory <apt_proxy.apt_proxy.Factory instance at 0xb7b12bec>
2005/05/20 17:05 CEST [-] set uid/gid 104/65534
2005/05/20 17:06 CEST [-] [debug] Verifying database: /var/cache/apt-proxy/.apt-proxy/db/access.db
2005/05/20 17:06 CEST [-] [debug] Opening database /var/cache/apt-proxy/.apt-proxy/db/access.db
2005/05/20 17:06 CEST [-] Received SIGTERM, shutting down.
2005/05/20 17:06 CEST [-] (Port 8086 Closed)
2005/05/20 17:06 CEST [-] Stopping factory <apt_proxy.apt_proxy.Factory instance at 0xb7b12bec>
2005/05/20 17:06 CEST [-] [db] =========================
2005/05/20 17:06 CEST [-] [db] Dumping update times
2005/05/20 17:06 CEST [-] [db] =========================
2005/05/20 17:06 CEST [-] [debug] Verifying database: /var/cache/apt-proxy/.apt-proxy/db/update.db
2005/05/20 17:06 CEST [-] [debug] Opening database /var/cache/apt-proxy/.apt-proxy/db/update.db
2005/05/20 17:06 CEST [-] [db] /debian/dists/sarge/contrib/binary-i386/Release: Fri May 20 11:49:53 2005
2005/05/20 17:06 CEST [-] [db] /debian/dists/sarge-proposed-updates/contrib/binary-i386/Release: Fri May 20 11:49:56 2005

[61 lines skipped]

2005/05/20 17:06 CEST [-] [db] =========================
2005/05/20 17:06 CEST [-] [db] Dumping access times
2005/05/20 17:06 CEST [-] [db] =========================
2005/05/20 17:06 CEST [-] [db] /security/dists/sarge/updates/contrib/binary-i386/Release: Fri May 20 12:02:12 2005
2005/05/20 17:06 CEST [-] [db] /security/pool/updates/main/x/xfree86/xfonts-base-transcoded_4.1.0-16woody3_all.deb: Mon Feb 16 10:00:00 2004

[.......]

2005/05/20 17:06 CEST [-] [db] /debian/pool/main/a/apmd/libapm1_3.2.1-4_i386.deb: Mon Feb 23 22:32:00 2004

[.......]

2005/05/20 17:06 CEST [-] [db] /debian/pool/main/a/apmd/libapm1_3.2.2-1_i386.deb: Sat May  1 05:17:00 2004

[........]

2005/05/20 17:06 CEST [-] [db] /debian/pool/main/a/apmd/libapm1_3.2.2-3_i386.deb: Sat May 14 18:41:47 2005

[........]

2005/05/20 17:06 CEST [-] [db] /debian/pool/main/a/apmd/libapm1_3.2.2-2_i386.deb: Mon Jun  7 15:47:00 2004

[.........]

2005/05/20 17:06 CEST [-] [db] =========================
2005/05/20 17:06 CEST [-] [db] Dumping packages
2005/05/20 17:06 CEST [-] [db] =========================
2005/05/20 17:06 CEST [-] [debug] Verifying database: /var/cache/apt-proxy/.apt-proxy/db/packages.db
2005/05/20 17:06 CEST [-] [debug] Opening database /var/cache/apt-proxy/.apt-proxy/db/packages.db
2005/05/20 17:06 CEST [-] [db] whois: 
2005/05/20 17:06 CEST [-] [db] 	/debian/pool/main/w/whois/whois_4.7.2_i386.deb
2005/05/20 17:06 CEST [-] [db] 	/debian/pool/main/w/whois/whois_4.7.4_i386.deb

[......]

2005/05/20 17:06 CEST [-] [db] libapm1: 
2005/05/20 17:06 CEST [-] [db] 	/debian/pool/main/a/apmd/libapm1_3.2.2-3_i386.deb

[.......]

2005/05/20 17:06 CEST [-] Main loop terminated.
2005/05/20 17:06 CEST [-] Server Shut Down.
2005/05/20 17:06 CEST [-] Log opened.
2005/05/20 17:06 CEST [-] twistd 1.3.0rc1 (/usr/bin/python2.3 2.3.5) starting up
2005/05/20 17:06 CEST [-] reactor class: twisted.internet.default.SelectReactor
2005/05/20 17:06 CEST [-] Loading /usr/sbin/apt-proxy...
2005/05/20 17:06 CEST [-] Updating twisted's process module.
2005/05/20 17:06 CEST [-] No updating required.
2005/05/20 17:06 CEST [-] Loaded.
2005/05/20 17:06 CEST [-] apt_proxy.apt_proxy.Factory starting on 8086
2005/05/20 17:06 CEST [-] Starting factory <apt_proxy.apt_proxy.Factory instance at 0xb7b12c0c>
2005/05/20 17:06 CEST [-] set uid/gid 104/65534
2005/05/20 17:06 CEST [-] [debug] Verifying database: /var/cache/apt-proxy/.apt-proxy/db/access.db
2005/05/20 17:06 CEST [-] [debug] Opening database /var/cache/apt-proxy/.apt-proxy/db/access.db
2005/05/20 17:16 CEST [-] [debug] Doing periodic cleaning up
2005/05/20 17:16 CEST [-] [debug] Periodic cleaning done

At this point, the cache size still isn't reduced. There are still many
packages with more than 2 versions, for instance:

karl:~# ls -l /var/cache/apt-proxy/debian/pool/main/a/apmd/
total 152
-rw-------  1 aptproxy nogroup 57104 Mar  1 20:02 apmd_3.2.2-3_i386.deb
-rw-r--r--  1 aptproxy root    24060 Feb 23  2004 libapm1_3.2.1-4_i386.deb
-rw-r--r--  1 aptproxy root    24222 May  1  2004 libapm1_3.2.2-1_i386.deb
-rw-r--r--  1 aptproxy root    24274 Jun  7  2004 libapm1_3.2.2-2_i386.deb
-rw-------  1 aptproxy nogroup 24366 Mar  1 20:02 libapm1_3.2.2-3_i386.deb

The access rights and groups variation is due to my manual restore of the
cache. However I don't know whether this is responsible for the failed
cleaning.

Christian

-- 
Christian Knoke            * * *            http://cknoke.de
* * * * * * * * *  Ceterum censeo Microsoft esse dividendum.