Re: [Pkgutil-users] pkgutil bug - cannot handle empty repos

[Peter B. <bon...@op...>]

On Tue, Aug 3, 2010 at 2:13 PM,  <ma...@pr...> wrote:
> Using v2.1 of pkgutil.
>
> I've set-up these mirror options in /etc/opt/csw/pkgutil.conf:
>
> mirror=https://swrepo.mydomain.com/sw/current
> mirror=https://swrepo.mydomain.com/sw/stable
> mirror=https://swrepo.mydomain.com/sw/unstable
>
> I would like these to be standard settings across all of my development clients (and potentially other lists of repos too).
>
> However, it is quite possible that sometimes the "unstable" repo might not actually contain any software.  So it has no catalogue file, it's just an empty directory.
>
> We're GPG signing everything ...
>
> # pkgutil -U
> <snip>
> => Fetching new catalog and descriptions (https://swrepo.mydomain.com/sw/unstable/i386/5.10) if available ...
> --2010-08-03 12:57:39--  https://swrepo.mydomain.com/sw/unstable/i386/5.10/catalog
> ...
> HTTP request sent, awaiting response... 404 Not Found
> 2010-08-03 12:57:39 ERROR 404: Not Found.
> ...
> Fetching of catalog failed.
>
> Ok, it would be nicer if it said "no catalog available", as actually there has been no failure, it has done what was expected.  But that's an aside.  Let's continue ...
>
> # pkgutil -a
> Checking catalog integrity with gpg.
> gpg: Signature made 29 July 2010 10:36:28 BST using DSA key ID 61404A7B
> gpg: Good signature from "swrepo server <root@testhost>"
> Checking catalog integrity with gpg.
> gpg: Signature made  3 August 2010 12:47:13 BST using DSA key ID 61404A7B
> gpg: Good signature from "swrepo server <root@testhost>"
> Catalog /var/opt/csw/pkgutil/catalog.swrepo.mydomain.com_sw_unstable_i386_5.10 is not signed! Check your mirror settings or disable use_gpg in pkgutil.conf.
>
> # ls -l /var/opt/csw/pkgutil/catalog.swrepo.mydomain.com_sw_unstable_i386_5.10
> -rw-r--r--   1 root     root           0 Aug  3 12:57 /var/opt/csw/pkgutil/catalog.swrepo.mydomain.com_sw_unstable_i386_5.10
>
> So, because we had an empty repo with no catalogue file, pkgutil has created a zero-length file in /var which is, of course, not GPG signed.  Our client is now no longer able to use the repo.
>
> Ok, what if we delete the file?
>
> # rm /var/opt/csw/pkgutil/catalog.swrepo.mydomain.com_sw_unstable_i386_5.10
> # pkgutil -a
> Checking catalog integrity with gpg.
> gpg: Signature made 29 July 2010 10:36:28 BST using DSA key ID 61404A7B
> gpg: Good signature from "swrepo server <root@testhost>"
> Checking catalog integrity with gpg.
> gpg: Signature made  3 August 2010 12:47:13 BST using DSA key ID 61404A7B
> gpg: Good signature from "swrepo server <root@testhost>"
> => Fetching new catalog and descriptions (https://swrepo.mydomain.com/sw/unstable/i386/5.10) if available ...
> --2010-08-03 13:05:59--  https://swrepo.mydomain.com/sw/unstable/i386/5.10/catalog
> ...
> HTTP request sent, awaiting response... 404 Not Found
> 2010-08-03 13:05:59 ERROR 404: Not Found.
> ...
> Fetching of catalog failed.
>
> Nope, that doesn't work.  The only solution is to comment out
>
> mirror=https://swrepo.mydomain.com/sw/unstable
>
> in /etc/opt/csw/pkgutil.conf.  Or, of course, create an empty catalog file in the empty repo and GPG sign it.  Ideally pkgutil should be better at handling empty repos.

I think the real solution here is to sign your empty catalogs, that is
don't handle them differently just because they're empty. You have gpg
checks set and an empty file can never pass that.

I'm prepared to discuss how to handle the actual downloading of empty
catalogs/failed downloads. One idea I have is to download to a
temporary file and checking for a non zero exit code from wget before
moving it to the real catalog file. That would avoid empty catalogs
overwriting your old working one after a failed download. In your case
with empty catalogs it wouldn't trigger an error (until the gpg
checks).

Also maybe setting gpg checks per catalog?

-- 
/peter