pkgutil-users — User list for questions etc.

[Pkgutil-users] SF.net SVN: pkgutil:[289] trunk/pkgutil

[<bon...@us...>]

Revision: 289
          http://pkgutil.svn.sourceforge.net/pkgutil/?rev=289&view=rev
Author:   bonivart
Date:     2010-08-10 15:55:22 +0000 (Tue, 10 Aug 2010)

Log Message:
-----------
pkgutil: add info about which catalog is being gpg checked (Mark Bannister)

Modified Paths:
--------------
    trunk/pkgutil


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Pkgutil-users] SF.net SVN: pkgutil:[288] tags/2.1/

[<bon...@us...>]

Revision: 288
          http://pkgutil.svn.sourceforge.net/pkgutil/?rev=288&view=rev
Author:   bonivart
Date:     2010-08-10 14:28:40 +0000 (Tue, 10 Aug 2010)

Log Message:
-----------
create tag for 2.1 release

Added Paths:
-----------
    tags/2.1/


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

Re: [Pkgutil-users] [csw-users] CSWsvn sparc has wrong architecture

[Peter B. <bon...@op...>]

On Tue, Aug 3, 2010 at 9:13 PM, Dagobert Michelsen <da...@op...> wrote:
> Am 03.08.2010 um 16:04 schrieb phi...@rz...:
>> Hi Dago,
>>
>> i did make a package stream on a i386 architekture with
>>
>> the flag -T sparc:5.10. (using pkgutil)
>>
>> well, anyway i move my packaging zone to a sparc machine.
>>
>> seems to work right now.
>
> Fwd'ing your email to pkgutil-users@ as another user reported a
> similar problem.

I can't reproduce that. Look at the link from pastebin:

http://pastebin.com/mL83XLfx

I'm on OpenSolaris on i386 and it immediately uses the 5.10 catalog
for Sparc and everything is correct.

-- 
/peter

Re: [Pkgutil-users] [csw-users] CSWsvn sparc has wrong architecture

[Dagobert M. <da...@op...>]

Hi Philip,

> Hi Philip,
>
> Am 02.08.2010 um 17:21 schrieb phi...@rz...:
>> subversion seems to have the wrong arch:
>>
>> # uname -a
>> SunOS rzbwsss01ap 5.10 Generic_142900-04 sun4u sparc SUNW,SPARC- 
>> Enterprise
>>
>> # pkgutil -c | grep svn
>> CSWsvn                    1.6.11,REV=2010.05.27      
>> 1.6.11,REV=2010.05.26
>>
>> root@nohost:/opt/csw/bin# file *|grep -vi sparc |grep -v script
>> svn:            ELF 32-bit LSB executable 80386 Version 1 [FPU],  
>> dynamically linked, stripped
>> svn-contrib:    directory
>> svn-tools:      directory
>> svnadmin:       ELF 32-bit LSB executable 80386 Version 1 [FPU],  
>> dynamically linked, stripped
>> svndumpfilter:  ELF 32-bit LSB executable 80386 Version 1 [FPU],  
>> dynamically linked, stripped
>> svnlook:        ELF 32-bit LSB executable 80386 Version 1 [FPU],  
>> dynamically linked, stripped
>> svnserve:       ELF 32-bit LSB executable 80386 Version 1 [FPU],  
>> dynamically linked, stripped
>> svnsync:        ELF 32-bit LSB executable 80386 Version 1 [FPU],  
>> dynamically linked, stripped
>> svnversion:     ELF 32-bit LSB executable 80386 Version 1 [FPU],  
>> dynamically linked, stripped
>>
>> it's the same package version in stable and unstable.
>
> I doubt that as stable hasn't been updated for a long time.
>
>> could you compile it for sparc architecture?
>
> This is from the Sparc 5.10 catalog:
>> subversion 1.6.11,REV=2010.05.26 CSWsvn  
>> subversion-1.6.11,REV=2010.05.26-SunOS5.9-sparc-CSW.pkg.gz  
>> b94377c9163a19915e4d2ac9773b7c4a 2034882 CSWcommon|CSWapache2rt| 
>> CSWbdb48|CSWexpat|CSWggettextrt|CSWiconv|CSWneon|CSWoldaprt|CSWsasl| 
>> CSWlibserf|CSWsqlite3rt|CSWzlib none none
>
> Subversion 1.6.11,REV=2010.05.27 was only released for i386, the  
> corresponding sparc package
> has the timestamp 1.6.11,REV=2010.05.26 due to the massive buildtime  
> of the software:
>
>> dam@login [login]:/export/mirror/opencsw/current > ls -l */5.10/ 
>> subversion-*
>> lrwxrwxrwx   1 web      web           64 May 29 03:30 i386/5.10/ 
>> subversion-1.6.11,REV=2010.05.27-SunOS5.9-i386-CSW.pkg.gz -> ../5.9/ 
>> subversion-1.6.11,REV=2010.05.27-SunOS5.9-i386-CSW.pkg.gz
>> lrwxrwxrwx   1 web      web           65 May 29 03:30 sparc/5.10/ 
>> subversion-1.6.11,REV=2010.05.26-SunOS5.9-sparc-CSW.pkg.gz -> ../ 
>> 5.9/subversion-1.6.11,REV=2010.05.26-SunOS5.9-sparc-CSW.pkg.gz
>
>
> It looks like you have just the wrong package installed. I suggest  
> you just "pkgrm CSWsvn"
> and simply reinstall it with "pkg-get -i subversion". As you now  
> have an inconsistent
> installation I wonder how you installed it in the first place.


Am 03.08.2010 um 16:04 schrieb phi...@rz...:
> Hi Dago,
>
> i did make a package stream on a i386 architekture with
>
> the flag -T sparc:5.10. (using pkgutil)
>
> well, anyway i move my packaging zone to a sparc machine.
>
> seems to work right now.

Fwd'ing your email to pkgutil-users@ as another user reported a  
similar problem.


Best regards

   -- Dago

[Pkgutil-users] Version 2.1 released

[Peter B. <bon...@op...>]

As you may have noticed, version 2.1 has just been released a few days
ago. See http://pkgutil.net for more info.

I have been more or less offline myself, attending a friends wedding
over the weekend, but I'm now back reading up on the posts to the
list.

With 2.1 out the door we can get on with the new stuff! :-)

-- 
/peter

[Pkgutil-users] pkgutil RFE - better handling of multiple catalogues

[<ma...@pr...>]

If I do this:

mirror=https://swrepo.mydomain.com/sw/repo1
mirror=https://swrepo.mydomain.com/sw/repo2
mirror=https://swrepo.mydomain.com/sw/repo3
mirror=https://swrepo.mydomain.com/sw/repo4
mirror=https://swrepo.mydomain.com/sw/repo5

in /etc/opt/csw/pkgutil.conf, I would expect pkgutil to be better at informing me which catalogue it was dealing with when performing repetitive tasks.  For example:

Checking catalog integrity with gpg.
Checking catalog integrity with gpg.
Checking catalog integrity with gpg.
Checking catalog integrity with gpg.
etc.

...is not very informative.

Also pkgutil -a lists what software is available, but I can't get pkgutil to tell me which catalogue contained the software.  This would be really useful.

Best regards,
Mark.

[Pkgutil-users] pkgutil bug - cannot handle empty repos

[<ma...@pr...>]

Using v2.1 of pkgutil.

I've set-up these mirror options in /etc/opt/csw/pkgutil.conf:

mirror=https://swrepo.mydomain.com/sw/current
mirror=https://swrepo.mydomain.com/sw/stable
mirror=https://swrepo.mydomain.com/sw/unstable

I would like these to be standard settings across all of my development clients (and potentially other lists of repos too).

However, it is quite possible that sometimes the "unstable" repo might not actually contain any software.  So it has no catalogue file, it's just an empty directory.

We're GPG signing everything ...

# pkgutil -U
<snip>
=> Fetching new catalog and descriptions (https://swrepo.mydomain.com/sw/unstable/i386/5.10) if available ...
--2010-08-03 12:57:39--  https://swrepo.mydomain.com/sw/unstable/i386/5.10/catalog
...
HTTP request sent, awaiting response... 404 Not Found
2010-08-03 12:57:39 ERROR 404: Not Found.
...
Fetching of catalog failed.

Ok, it would be nicer if it said "no catalog available", as actually there has been no failure, it has done what was expected.  But that's an aside.  Let's continue ...

# pkgutil -a
Checking catalog integrity with gpg.
gpg: Signature made 29 July 2010 10:36:28 BST using DSA key ID 61404A7B
gpg: Good signature from "swrepo server <root@testhost>"
Checking catalog integrity with gpg.
gpg: Signature made  3 August 2010 12:47:13 BST using DSA key ID 61404A7B
gpg: Good signature from "swrepo server <root@testhost>"
Catalog /var/opt/csw/pkgutil/catalog.swrepo.mydomain.com_sw_unstable_i386_5.10 is not signed! Check your mirror settings or disable use_gpg in pkgutil.conf.

# ls -l /var/opt/csw/pkgutil/catalog.swrepo.mydomain.com_sw_unstable_i386_5.10
-rw-r--r--   1 root     root           0 Aug  3 12:57 /var/opt/csw/pkgutil/catalog.swrepo.mydomain.com_sw_unstable_i386_5.10

So, because we had an empty repo with no catalogue file, pkgutil has created a zero-length file in /var which is, of course, not GPG signed.  Our client is now no longer able to use the repo.

Ok, what if we delete the file?

# rm /var/opt/csw/pkgutil/catalog.swrepo.mydomain.com_sw_unstable_i386_5.10
# pkgutil -a
Checking catalog integrity with gpg.
gpg: Signature made 29 July 2010 10:36:28 BST using DSA key ID 61404A7B
gpg: Good signature from "swrepo server <root@testhost>"
Checking catalog integrity with gpg.
gpg: Signature made  3 August 2010 12:47:13 BST using DSA key ID 61404A7B
gpg: Good signature from "swrepo server <root@testhost>"
=> Fetching new catalog and descriptions (https://swrepo.mydomain.com/sw/unstable/i386/5.10) if available ...
--2010-08-03 13:05:59--  https://swrepo.mydomain.com/sw/unstable/i386/5.10/catalog
...
HTTP request sent, awaiting response... 404 Not Found
2010-08-03 13:05:59 ERROR 404: Not Found.
...
Fetching of catalog failed.

Nope, that doesn't work.  The only solution is to comment out

mirror=https://swrepo.mydomain.com/sw/unstable

in /etc/opt/csw/pkgutil.conf.  Or, of course, create an empty catalog file in the empty repo and GPG sign it.  Ideally pkgutil should be better at handling empty repos.

Regards,
Mark.

[Pkgutil-users] pkgutil bug -U not honouring -T

[<ma...@pr...>]

Using pkgutil 2.1 on an i386 machine:

# pkgutil -T sparc:5.10 -W . -U
=> Fetching new catalog and descriptions (http://www.grangefields.co.uk/mirrors/csw/current/i386/5.10) if available ...
--12:32:56--  http://www.grangefields.co.uk/mirrors/csw/current/i386/5.10/catalog
           => `./catalog.www.grangefields.co.uk_mirrors_csw_current_i386_5.10'

... downloads the i386 catalogue, ignoring the -T argument.

# pkgutil -T sparc:5.10 -W . -d pca
=> Fetching new catalog and descriptions (http://www.grangefields.co.uk/mirrors/csw/current/sparc/5.10) if available ...
--12:33:29--  http://www.grangefields.co.uk/mirrors/csw/current/sparc/5.10/catalog
           => `./catalog.www.grangefields.co.uk_mirrors_csw_current_sparc_5.10'

... fetches the sparc catalogue, honouring the -T argument.

Regards,
Mark Bannister.

Re: [Pkgutil-users] RFC - New catalogue format (version 10)

[<ma...@pr...>]

Ok, mail client or mailing list or something has stripped everything I typed within chevrons.  So my original email didn't make much sense.  (Whoever thought it was a good idea to support HTML format in emails should be incarcerated for causing a public nuisance).

I've attached my last mail as a file.  Fortunately I'd copied it to the clipboard before I sent it.

Best regards,
Mark.

[Pkgutil-users] RFC - New catalogue format (version 10)

[<ma...@pr...>]

Having tried to identify other ways of improving pkgutil to address its shortcomings around the management of multiple versions of non-CSW software, I have come to the conclusion that the only proper way to do this is to define a new catalogue format (which I guess will be version 10).

I appreciate that opencsw.org will not be able to use this format, and will have to stick with version 9, because unfortunately it will not be backwards compatible.  However, the version 10 format will bring with it a number of key benefits for users who want to set-up their own internal software repositories.

pkgutil and bldcat will need to be modified to support a version 10 format, but will continue to be able to handle earlier formats too, so one toolset is used for downloading content from opencsw.org as well as from newer repositories as they wish.

Here are my proposed changes for the version 10 catalogue format:

* A line 'version:10' to appear before the first catalogue entry to identify this, non-ambiguously, as a version 10 catalogue.  Note that this cannot be guaranteed to be the first line of the file, as a PGP signature may come before it.

* The 1st field (catalogue name, or 'common' name) will no longer be a unique identifier for a package and its version.  An individual package can now only be uniquely addressed by combining the 3rd field (package name) and the 2nd field (version string).  The catalogue name now only uniquely identifies a software package, but which may encompass multiple versions of that package.  This will allow pkgutil to operate as documented, i.e. from the man page "You may specify an explicit version (e.g. amarok-1.4.8,REV=2008.02.26) if desired, otherwise the latest version found is chosen" which is misleading because at present, there can never be more than one catalogue entry for "amarok".

* The 7th field (dependency list) and 9th field (incompatible list) will support the following formats:

<pkgname>|<pkgname>|...
---> legacy format

<type>:<name>|<type>:<name>|..
 ---> where <type> can be the letter 'p' for a package called <name> or 'c' for the catalogue (common) entry <name>

Note that bldcat and pkgutil will use type 'c' in a dependency list for packages that appear in the catalogue and type 'p' for packages that do not appear in the catalogue.  The tools will disallow the use of type 'p' if the package is in the catalogue.  This binds dependencies to particular versions of software in the catalogue, without requiring the package name to be amended (as is currently the case).  Identifying which version to bind to will be automatic where possible, referring to a mapping file where there is a choice to be made, otherwise the user will be prompted by bldcat when the information is lacking.

Comments?  Any other pain points that need fixing in the catalogue format?

Best regards,
Mark.

Re: [Pkgutil-users] package map file

[<ma...@pr...>]

I realised that the map file shouldn't actually be in /etc/opt/csw/pkgutil.map, because it will primarily be used by bldcat.  I suggest going to Dago's original suggestion, calling it 'mapfile'.  I think it is better located in the repo, in the same directory as the 'catalog' and 'descriptions' files.

Best regards,
Mark.

Re: [Pkgutil-users] package map file

[<ma...@pr...>]

>>> For what you envision there are hooks which can provide the update
>>> functionality even on package name change.
>>
>> I don't understand.  What hooks do you mean?  How would I use them
>> to fix the dependency issue?  Please explain further.
>
> See here:
> http://wiki.opencsw.org/package-hooks
> The upgrade script would look inside the installed and new package and 
> see if the update would be necessary and how it should be done. It could e.g.  remove PKGa
> and install PKGb as new version instead.

Fundamentally I need to change the list of packages that pkgutil thinks it wants to install.  If I can't change the catalogue format so that dependencies are associated with the correct package versions, then I have three options:

1) Break compatibility by branching off a new version of pkgutil for anyone who doesn't care about compatibility with pkg-get or Sunfreeware.
2) Introduce a new file that lives alongside the catalogue file that defines the correct version-specific dependencies.
3) Modify the prebatchinstall hook to allow an external hook script to modify the list of packages to install.  Although the dependency list would still need to come from somewhere, so this is practically the same as 2).

None of these options sound ideal to me.

Any alternatives?

Best regards,
Mark.

Re: [Pkgutil-users] bldcat patch to auto-generate catalog name and GPG sign

[Peter B. <bon...@op...>]

On Wed, Jul 28, 2010 at 12:13 PM,  <ma...@pr...> wrote:
> So, taking the latest bldcat from SourceForge, here we go:
>
> # rm /export/swrepo/stable/i386/5.10/catalog
> # ./bldcat /export/swrepo/stable/i386/5.10
> Inspecting /export/swrepo/stable/i386/5.10/berkeleydb4-4.2.52,REV=2009.10.18-SunOS5.8-all-CSW.pkg.gz
> Inspecting /export/swrepo/stable/i386/5.10/berkeleydb42-4.2.52,REV=2009.10.18_rev=p5-SunOS5.8-i386-CSW.pkg.gz
> Inspecting /export/swrepo/stable/i386/5.10/bzip2-1.0.5,REV=2010.06.28-SunOS5.9-i386-CSW.pkg.gz
> <snip>
>
> # cat /export/swrepo/stable/i386/5.10/catalog
> berkeleydb4 4.2.52,REV=2009.10.18 CSWbdb4 export/swrepo/stable/i386/5.10/berkeleydb4-4.2.52,REV=2009.10.18-SunOS5.8-all-CSW.pkg.gz 0a25868a37d6690acd99a25df4bc3511 696 CSWbdb42|CSWcommon none none
> berkeleydb42 4.2.52,REV=2009.10.18_rev=p5 CSWbdb42 export/swrepo/stable/i386/5.10/berkeleydb42-4.2.52,REV=2009.10.18_rev=p5-SunOS5.8-i386-CSW.pkg.gz 18abbcdda31d6d66f2b90f9830321f74 3182435 CSWcommon none none
> bzip2 1.0.5,REV=2010.06.28 CSWbzip2 export/swrepo/stable/i386/5.10/bzip2-1.0.5,REV=2010.06.28-SunOS5.9-i386-CSW.pkg.gz a52eb6c2b1cf435089029f4428c4525a 405633 CSWcommon|CSWisaexec none none
> <snip>

OK, I see it now. I have fixed it in r287 of bldcat. Thanks for the report.

> By the way, I notice you've put $Id$ at the top of your scripts (bldcat, pkgutil etc.).  I think you mean $Id:$ don't you?  You'll also need to do an 'svn propset svn:keywords Id <filename>' on each file where you want that tag to be substituted, if you haven't already done so.

SF doesn't show the Id online but when you check out a file it's
expanded so this is correct in the packages but not if you download a
file directly from SF instead of using SVN.

-- 
/peter

[Pkgutil-users] Compare available packages to those installed

[<ma...@pr...>]

An uncontroversial patch, I hope, is attached.

This patch can be applied to pkgutil-2.1 (rev 286).

It adds the following:

* New option added, pkgutil -A (--compare-available), that is similar to pkgutil -c, except that it lists the packages in the catalogue and compares them with the installed version (or says "not installed").
* pkgutil -C now has a long option equivalent; --compare-different.

I'm immediately finding a use for this feature, and hope you do too.

Best regards,
Mark.

Re: [Pkgutil-users] package map file

[Dagobert M. <da...@op...>]

Hi Mark,

Am 28.07.2010 um 14:11 schrieb ma...@pr...:
> On Wed 28/07/10 12:18 PM , Dagobert Michelsen da...@op... sent:
>> Am 28.07.2010 um 12:58 schrieb ma...@pr...:
>>> a) catalogue format changed so that dependencies
>>> are listed by catalogue name, not package name
>>
>> This is virtually undoable as the format is fixed.
>
> Fixed by whom?  pkgutil could easily support both formats.

By the sites using it. OpenCSW catalogs are also processed by pkg-get  
which will
not support alternative catalog formats. Same goes for SunFreeware,  
although
they use an even older format. It must be backward-compatible.  
Additional
files (like description) would be ok, I guess.

>> For what you envision there are hooks which can provide the update  
>> functionality even on
>> package name change.
>
> I don't understand.  What hooks do you mean?  How would I use them  
> to fix the dependency issue?  Please explain further.

See here:
   http://wiki.opencsw.org/package-hooks
The upgrade script would look inside the installed and new package and  
see if the
update would be necessary and how it should be done. It could e.g.  
remove PKGa
and install PKGb as new version instead.

>>> c) some automatic logic for handling version numbers without a REV
>>> component on non-CSW packages (as per my earlier pkgutil patch)
>>> unless an entry in the pkgutil.map file explicitly provides the
>>> version string to use in the catalogue
>>
>> I would prefer to enforce having REV in the pkgmap if the package  
>> doesn't have it.
>> It would obsolete special compare-functions this way.
>
> Defeats the object of supporting third party packages without  
> tampering with them.  We can't force third parties to stick to a  
> particular pkginfo format.  Bear in mind that the REV component of  
> the VERSION string was never a standard anyway (check the pkginfo(4)  
> man page), it was only something that Sun was doing with their  
> packages and I guess Peter thought he may as well do the same thing  
> with pkgutil.  Really PSTAMP is supposed to do this job, according  
> to the official documentation.  For non-CSW packages, we really will  
> need to cope with different variations of VERSION strings.

True. That's why I suggested condensing all version information to REV  
style with
some kind of plugin to bldcat.


Best regards

   -- Dago

Re: [Pkgutil-users] package map file

[<ma...@pr...>]

On Wed 28/07/10 12:18 PM , Dagobert Michelsen da...@op... sent:
> Am 28.07.2010 um 12:58 schrieb ma...@pr...:
>> a) catalogue format changed so that dependencies
>> are listed by catalogue name, not package name
>
> This is virtually undoable as the format is fixed.

Fixed by whom?  pkgutil could easily support both formats.

> For what you envision there are hooks which can provide the update functionality even on  
> package name change.

I don't understand.  What hooks do you mean?  How would I use them to fix the dependency issue?  Please explain further.

>> c) some automatic logic for handling version numbers without a REV
>> component on non-CSW packages (as per my earlier pkgutil patch)
>> unless an entry in the pkgutil.map file explicitly provides the
>> version string to use in the catalogue
> 
> I would prefer to enforce having REV in the pkgmap if the package doesn't have it.
> It would obsolete special compare-functions this way.

Defeats the object of supporting third party packages without tampering with them.  We can't force third parties to stick to a particular pkginfo format.  Bear in mind that the REV component of the VERSION string was never a standard anyway (check the pkginfo(4) man page), it was only something that Sun was doing with their packages and I guess Peter thought he may as well do the same thing with pkgutil.  Really PSTAMP is supposed to do this job, according to the official documentation.  For non-CSW packages, we really will need to cope with different variations of VERSION strings.

Best regards,
Mark.

[Pkgutil-users] SF.net SVN: pkgutil:[287] trunk

[<bon...@us...>]

Revision: 287
          http://pkgutil.svn.sourceforge.net/pkgutil/?rev=287&view=rev
Author:   bonivart
Date:     2010-07-28 12:03:15 +0000 (Wed, 28 Jul 2010)

Log Message:
-----------
bldcat: fix bug with package dir being included in filename (Mark Bannister)

Modified Paths:
--------------
    trunk/bldcat
    trunk/readme


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

Re: [Pkgutil-users] package map file

[Dagobert M. <da...@op...>]

Hi Mark,

Am 28.07.2010 um 12:58 schrieb ma...@pr...:

> Started new thread (was 'patch to support non-standard VERSION  
> entries').
>
> Attached, again, is my (amended) suggestion for a mapping file.
>
> As suggested earlier, a file like this could be automatically  
> generated if run on a system that already had the software stack  
> installed.
>
> The intention is to have a file where the version/revision that goes  
> into the catalogue, and/or the catalogue name, and/or package  
> dependencies can be tweaked without having to tamper with the  
> package itself.  The reason we want to leave the package alone is to  
> cater for third party software where the vendor will not officially  
> support it if it has been altered.
>
> This pkgutil.map file would have to be part of a number of changes  
> to pkgutil, namely:
>
> a) catalogue format changed so that dependencies are listed by  
> catalogue name, not package name -- the rationale behind this change  
> is to stop the practice whereby to support different versions of the  
> same software you need to rename the packages (this way you just  
> need a different catalogue name and filename)

This is virtually undoable as the format is fixed. For what you envision
there are hooks which can provide the update functionality even on  
package
name change.

> b) catalogue names automatically generated by 'bldcat' on non-CSW  
> packages (as per my earlier bldcat patch) unless an entry in the  
> pkgutil.map file explicitly provides the name

Ok.

> c) some automatic logic for handling version numbers without a REV  
> component on non-CSW packages (as per my earlier pkgutil patch)  
> unless an entry in the pkgutil.map file explicitly provides the  
> version string to use in the catalogue

I would prefer to enforce having REV in the pkgmap if the package  
doesn't have it.
It would obsolete special compare-functions this way.

> d) dependencies automatically generated by 'bldcat' as today unless  
> an entry in the pkgutil.map file explicitly provides the dependency  
> list.

Ok.


Best regards

   -- Dago

[Pkgutil-users] package map file

[<ma...@pr...>]

Started new thread (was 'patch to support non-standard VERSION entries').

Attached, again, is my (amended) suggestion for a mapping file.

As suggested earlier, a file like this could be automatically generated if run on a system that already had the software stack installed.

The intention is to have a file where the version/revision that goes into the catalogue, and/or the catalogue name, and/or package dependencies can be tweaked without having to tamper with the package itself.  The reason we want to leave the package alone is to cater for third party software where the vendor will not officially support it if it has been altered.

This pkgutil.map file would have to be part of a number of changes to pkgutil, namely:

a) catalogue format changed so that dependencies are listed by catalogue name, not package name -- the rationale behind this change is to stop the practice whereby to support different versions of the same software you need to rename the packages (this way you just need a different catalogue name and filename)
b) catalogue names automatically generated by 'bldcat' on non-CSW packages (as per my earlier bldcat patch) unless an entry in the pkgutil.map file explicitly provides the name
c) some automatic logic for handling version numbers without a REV component on non-CSW packages (as per my earlier pkgutil patch) unless an entry in the pkgutil.map file explicitly provides the version string to use in the catalogue
d) dependencies automatically generated by 'bldcat' as today unless an entry in the pkgutil.map file explicitly provides the dependency list.

Thoughts?

Peter - could all this be done via a module, as per your suggestion, so that people could choose to "plug-in" these capabilities, or unplug them if they want to do it the CSW way?

Best regards,
Mark.

Re: [Pkgutil-users] Software stacks

[Dagobert M. <da...@op...>]

Hi Mark,

Am 28.07.2010 um 12:18 schrieb ma...@pr...:
> Having made myself even more familiar with the pkgutil code, let me  
> offer an apology.  I had failed to realise that you could do this:
>
> mirror=https://swrepo.mydomain.com/sw/current
> mirror=https://swrepo.mydomain.com/sw/stable
> mirror=https://swrepo.mydomain.com/sw/unstable

or
   mirror=https://swrepo.mydomain.com/sw/veritas35
   mirror=https://swrepo.mydomain.com/sw/oracle10g2
for different vendor versions. You would have one line per stack and
adjusting it would mean switch versions.

> ...all in the same pkgutil.conf file.
>
> This is what you were trying to explain to me, but I didn't latch  
> onto the fact that pkgutil could handle multiple catalogues.
>
> So this is great, and you're right, it does solve the software stack  
> problem.

Yep, that's what I meant :-)

> All this leaves is how we properly automate the generating of a  
> catalogue name without requiring the packages to be tampered with.   
> I think we can probably address this with a package map file, which  
> I'll start a new thread for.

Cool.


Best regards

   -- Dago

Re: [Pkgutil-users] patch to support non-standard VERSION entries

[<ma...@pr...>]

On Mon 26/07/10  4:11 PM , Peter Bonivart bon...@op... sent:
> I chose that example intentionally since an official 1.0 should never
> be replaced with a beta for the same version, that's a downgrade. I
> meant 1.0 beta 2 with my 1.0b2. Maybe you read it as something else,
> kind of proves my point... ;-)

Peter,

I have only just realised what you meant.  Wow, that took a few days for the penny to drop!

So, you're saying you would call something 1.0b1, then 1.0b2 before the official release 1.0 becomes available.

Hmmm, interesting numbering scheme, never thought of doing it that way.  See the problem.

Maybe package maps can help.....starting a new thread now.

Best regards,
Mark.

[Pkgutil-users] Software stacks

[<ma...@pr...>]

Hi Dago,

Having made myself even more familiar with the pkgutil code, let me offer an apology.  I had failed to realise that you could do this:

mirror=https://swrepo.mydomain.com/sw/current
mirror=https://swrepo.mydomain.com/sw/stable
mirror=https://swrepo.mydomain.com/sw/unstable

...all in the same pkgutil.conf file.

This is what you were trying to explain to me, but I didn't latch onto the fact that pkgutil could handle multiple catalogues.

So this is great, and you're right, it does solve the software stack problem.

All this leaves is how we properly automate the generating of a catalogue name without requiring the packages to be tampered with.  I think we can probably address this with a package map file, which I'll start a new thread for.

Best regards,
Mark.

Re: [Pkgutil-users] bldcat patch to auto-generate catalog name and GPG sign

[<ma...@pr...>]

On Tue 27/07/10  5:24 PM , Peter Bonivart bon...@op... sent:
> On Mon, Jul 26, 2010 at 2:32 PM,  ma...@pr... wrote:
>> I have one question.  If I run 'bldcat .' I get a
>> catalogue file that pkgutil can use.  If I run 'bldcat
>> some/path/to/some/directory' the resulting catalogue file cannot be used by
>> pkgutil because the pathname has been encoded in the catalogue.  Is this a
>> deliberate feature, or is this a bug?
>
> I can not reproduce your problem. I did the same thing without getting
> the pathname included into the filename. Can you give me the actual
> command you ran and a few lines of the generated catalog?

You have to delete and re-create the catalog to reproduce this (if it re-uses old information, you don't get the problem).  So, taking the latest bldcat from SourceForge, here we go:

# rm /export/swrepo/stable/i386/5.10/catalog
# ./bldcat /export/swrepo/stable/i386/5.10
Inspecting /export/swrepo/stable/i386/5.10/berkeleydb4-4.2.52,REV=2009.10.18-SunOS5.8-all-CSW.pkg.gz
Inspecting /export/swrepo/stable/i386/5.10/berkeleydb42-4.2.52,REV=2009.10.18_rev=p5-SunOS5.8-i386-CSW.pkg.gz
Inspecting /export/swrepo/stable/i386/5.10/bzip2-1.0.5,REV=2010.06.28-SunOS5.9-i386-CSW.pkg.gz
Inspecting /export/swrepo/stable/i386/5.10/ca_certificates-20091101,REV=2009.11.01-SunOS5.8-all-CSW.pkg.gz
Inspecting /export/swrepo/stable/i386/5.10/common-1.4.7,REV=2009.09.20-SunOS5.8-i386-CSW.pkg
<snip>

# cat /export/swrepo/stable/i386/5.10/catalog
berkeleydb4 4.2.52,REV=2009.10.18 CSWbdb4 export/swrepo/stable/i386/5.10/berkeleydb4-4.2.52,REV=2009.10.18-SunOS5.8-all-CSW.pkg.gz 0a25868a37d6690acd99a25df4bc3511 696 CSWbdb42|CSWcommon none none
berkeleydb42 4.2.52,REV=2009.10.18_rev=p5 CSWbdb42 export/swrepo/stable/i386/5.10/berkeleydb42-4.2.52,REV=2009.10.18_rev=p5-SunOS5.8-i386-CSW.pkg.gz 18abbcdda31d6d66f2b90f9830321f74 3182435 CSWcommon none none
bzip2 1.0.5,REV=2010.06.28 CSWbzip2 export/swrepo/stable/i386/5.10/bzip2-1.0.5,REV=2010.06.28-SunOS5.9-i386-CSW.pkg.gz a52eb6c2b1cf435089029f4428c4525a 405633 CSWcommon|CSWisaexec none none
ca_certificates 20091101,REV=2009.11.01 CSWcacertificates export/swrepo/stable/i386/5.10/ca_certificates-20091101,REV=2009.11.01-SunOS5.8-all-CSW.pkg.gz 2900aaab9fd765853b22ef8d7ab76831 129040 CSWcswclassutils|CSWcommon none none
common 1.4.7,REV=2009.09.20 CSWcommon export/swrepo/stable/i386/5.10/common-1.4.7,REV=2009.09.20-SunOS5.8-i386-CSW.pkg 25fc0169b494b40904f02347a2326046 25088 none none none
<snip>

If you try to download or install software now, it breaks:

# pkgutil -U
# pkgutil -d berkeleydb4
Solving needed dependencies ...
Solving dependency order ...
Package list:
        CSWbdb4-4.2.52,REV=2009.10.18
        CSWbdb42-4.2.52,REV=2009.10.18_rev=p5
        CSWcommon-1.4.7,REV=2009.09.20
Total size: 3.1 MB
3 packages to fetch. Do you want to continue? [Y,n] y
=> Fetching CSWcommon-1.4.7,REV=2009.09.20 (1/3) ...
/var/opt/csw/pkgutil/packages/export/swrepo/stable/i386/5.10/common-1.4.7,REV=2009.09.20-SunOS5.8-i386-CSW.pkg: No such file or directory

Fetching of CSWcommon-1.4.7,REV=2009.09.20 failed. Try updating your catalog with pkgutil -U.


By the way, I notice you've put $Id$ at the top of your scripts (bldcat, pkgutil etc.).  I think you mean $Id:$ don't you?  You'll also need to do an 'svn propset svn:keywords Id <filename>' on each file where you want that tag to be substituted, if you haven't already done so.

Best regards,
Mark.

[Pkgutil-users] SF.net SVN: pkgutil:[286] trunk

[<bon...@us...>]

Revision: 286
          http://pkgutil.svn.sourceforge.net/pkgutil/?rev=286&view=rev
Author:   bonivart
Date:     2010-07-28 08:54:11 +0000 (Wed, 28 Jul 2010)

Log Message:
-----------
pkgutil: 2.1 final

Modified Paths:
--------------
    trunk/pkgutil
    trunk/readme


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

Re: [Pkgutil-users] patch to support non-standard VERSION entries

[<ma...@pr...>]

 On Tue 27/07/10  3:34 PM , Peter Bonivart bon...@op... sent:
> I think it's starting to be a lot of different stuff being discussed
> in the same thread now. Maybe the package map and the software stacks
> deserve new threads started?

Good idea.

> I'm about to start using unit tests to increase the quality of the
> releases. This will likely require me to split off all but the
> simplest command line parsing from pkgutil and put basically
> everything in one or more perl modules.
> 
> This opens some possibilities regarding managing non-CSW stuff. It
> would be nice to have the compare subs in a perl module, we could have
> several complete, independent ones being selected by pkgutil.conf
> options. So the current CSW one is 100% intact and non-CSW stuff gets
> its own sub. I would like that kind of isolation instead of
> complicating the current stuff. Someone else could do a third variant
> which uses whatever method of doing the compare.
> 
> @Mark: are you comfortable with Subversion and Sourceforge? Do you
> want to be added to the project so you can start a new branch? You
> could try your stuff there, no need to send me patches.

This all sounds good, Peter.  My sourceforge username is 'cambridge'.

Modularising sounds like a great idea.  If I had a playpen I could write in support for some of the things I've been talking about, in a non-CSW module that you can test separately and then critique and/or modify as you see fit.  The problem will be that in order to do some of the things I'm suggesting, I'll need to add new command-line options, add options to pkgutil.conf, amend the catalogue format as well as provide different version comparison code.  Could this all be done in a module?

Thanks,
Mark.