pkgutil-users Mailing List for pkgutil (Page 11)
Status: Beta
Brought to you by:
bonivart
Menu
▾
▴
pkgutil-users — User list for questions etc.
You can subscribe to this list here.
| 2009 |
Jan
|
Feb
|
Mar
(9) |
Apr
(25) |
May
(19) |
Jun
(33) |
Jul
(2) |
Aug
(13) |
Sep
(28) |
Oct
(43) |
Nov
(24) |
Dec
(27) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2010 |
Jan
(5) |
Feb
(11) |
Mar
(39) |
Apr
(22) |
May
(4) |
Jun
(10) |
Jul
(52) |
Aug
(12) |
Sep
(12) |
Oct
(31) |
Nov
(13) |
Dec
(14) |
| 2011 |
Jan
(15) |
Feb
(19) |
Mar
(13) |
Apr
(1) |
May
(8) |
Jun
(1) |
Jul
(11) |
Aug
(16) |
Sep
(14) |
Oct
(4) |
Nov
(3) |
Dec
(8) |
| 2012 |
Jan
(11) |
Feb
(15) |
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
(3) |
Sep
|
Oct
|
Nov
|
Dec
|
| 2013 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
(5) |
Sep
|
Oct
|
Nov
|
Dec
|
| 2015 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
(4) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Peter B. <bon...@op...> - 2010-07-27 16:24:41
|
On Mon, Jul 26, 2010 at 2:32 PM, <ma...@pr...> wrote: > ...all packages that do not begin with uppercase CSW will have their catalogue name taken from the package name, by stripping off the leading capital letters. That looks good except dashes can be part of a package name but not part of the catalog name (they can have underscores). So we need to strip dashes to be safe. > I have also built in support for automatically GPG signing the catalogue. pkgutil can already check the catalogue to see if it is GPG signed, so it didn't make sense that bldcat couldn't build a signed catalogue: > > # bldcat --gpg . > > ...will run 'gpg --clearsign' on the catalogue file once it has been generated. Great! > I have one question. If I run 'bldcat .' I get a catalogue file that pkgutil can use. If I run 'bldcat some/path/to/some/directory' the resulting catalogue file cannot be used by pkgutil because the pathname has been encoded in the catalogue. Is this a deliberate feature, or is this a bug? I can not reproduce your problem. I did the same thing without getting the pathname included into the filename. Can you give me the actual command you ran and a few lines of the generated catalog? I did, however, find that if you specify a package dir with a trailing slash the slash is at the start of every filename. I now remove the trailing slash from the package dir if it's there. I have also added GNU stubs to bldcat and chkcat. -- /peter |
|
From: <bon...@us...> - 2010-07-27 16:08:21
|
Revision: 285
http://pkgutil.svn.sourceforge.net/pkgutil/?rev=285&view=rev
Author: bonivart
Date: 2010-07-27 16:08:15 +0000 (Tue, 27 Jul 2010)
Log Message:
-----------
bldcat: add missing -W option in man page
Modified Paths:
--------------
trunk/bldcat
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <bon...@us...> - 2010-07-27 16:03:34
|
Revision: 284
http://pkgutil.svn.sourceforge.net/pkgutil/?rev=284&view=rev
Author: bonivart
Date: 2010-07-27 16:03:28 +0000 (Tue, 27 Jul 2010)
Log Message:
-----------
bldcat: fix bug with trailing slash on package dir
Modified Paths:
--------------
trunk/bldcat
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <bon...@us...> - 2010-07-27 15:36:15
|
Revision: 283
http://pkgutil.svn.sourceforge.net/pkgutil/?rev=283&view=rev
Author: bonivart
Date: 2010-07-27 15:36:09 +0000 (Tue, 27 Jul 2010)
Log Message:
-----------
add gnu stub to bldcat and chkcat
Modified Paths:
--------------
trunk/bldcat
trunk/chkcat
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <bon...@us...> - 2010-07-27 15:20:49
|
Revision: 282
http://pkgutil.svn.sourceforge.net/pkgutil/?rev=282&view=rev
Author: bonivart
Date: 2010-07-27 15:20:43 +0000 (Tue, 27 Jul 2010)
Log Message:
-----------
pkgutil: add missing -V info about show_current
Modified Paths:
--------------
trunk/pkgutil
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: Peter B. <bon...@op...> - 2010-07-27 14:35:22
|
I think it's starting to be a lot of different stuff being discussed in the same thread now. Maybe the package map and the software stacks deserve new threads started? I'm about to start using unit tests to increase the quality of the releases. This will likely require me to split off all but the simplest command line parsing from pkgutil and put basically everything in one or more perl modules. This opens some possibilities regarding managing non-CSW stuff. It would be nice to have the compare subs in a perl module, we could have several complete, independent ones being selected by pkgutil.conf options. So the current CSW one is 100% intact and non-CSW stuff gets its own sub. I would like that kind of isolation instead of complicating the current stuff. Someone else could do a third variant which uses whatever method of doing the compare. @Mark: are you comfortable with Subversion and Sourceforge? Do you want to be added to the project so you can start a new branch? You could try your stuff there, no need to send me patches. -- /peter |
|
From: <ma...@pr...> - 2010-07-27 12:51:57
|
>> # pkgutil -i vcs -S 5.0MP3 >> >> for example, where our magic -S option (--select) tells pkgutil that >> you know there are multiple entries for the catalogue name 'vcs', >> the one you want is the one where the VERSION/REV string contains >> the text '5.0MP3'. > > This would require a dependency-mechanism not depending on a package name, but a specific version of > a package (e.g. derived from the catalogname). Only considering matching packages won't work in > specific environments - Veritas prior to 3.5 comes to my mind where you had a match > vcs 2.0 - vxvm 3.2 - vxfs 3.4 > In the catalog-approach you would bundle all of these in one catalog and everything would be fine. Not quite "everything fine". Ok, say you've got 10 software stacks to maintain on a single box. Each software stack has an average of 5 different versions. Any given box can have any combination of different versions of the software stacks. How do you expect to manage re-pointing pkgutil to 50 different repos depending on which software stack you wish to manage on a particular day? And then, if you're editing pkgutil.conf to do it, what if you're in a department of 30 sysadmins all of whom could be editing that file at any given time without your knowledge? Then add puppet into the mix. This is a recipe for trouble. pkgutil is designed to handle package dependencies. I understand this was the original purpose for developing the tool. With dependencies comes the issue of managing software stacks. By definition, pkgutil ought to have an elegant solution to this problem. > Hand-tailoring the catalog would be very tedious here. IMHO having a drop-box folder for packages > with as much automation as possible is favored as opposed to a manually crafted catalog. Agreed. Whatever the solution is, it'll need to do as much automatically as possible, only prompting the user where a choice really has to be made. Perhaps we could seed it with information collected from an existing system where the full software stack is already installed? This would be feasible. Existing CSW repos would continue to behave as they always have done, as there would be no choice to make (if you wanted to stick with different catalogue names and package names for different versions of software, as you currently do on CSW, that wouldn't change). btw did I not recently submit a patch to bldcat to auto-generate the catalogue name? Without that patch you have to hand-tailor your SysV packages which is hardly the automation approach that you are subscribing to. > A hacker is someone who abuses a tool for things it were never meant to do, > not someone who puts a spoiler to a car to go faster. To me it looks > the existing functionality in pkgutil is already capable of what you want to do, > but you want to do it differently. Disagree. I'd like to see pkgutil mature into a more powerful tool that can be used in large-scale enterprise environments. It is not quite there yet, but it's not far off. This is what I said to Peter yesterday, pkgutil is so close to being able to manage non-CSW packages extremely well, is it not better to make the necessary changes to improve the product, as long as we do so wisely and carefully (and definitely not by "hacking" anything), rather than by starting again from scratch? Earlier we spoke about a mapping file, let me suggest: # # /etc/opt/csw/pkgutil.map # # Mapping file used by pkgutil and bldcat for identifying non-standard # packages and assigning them a catalogue name and version string # in the pkgutil catalogue # #package "parameters" common-name version VRTSvcs "VERSION=5.0|PSTAMP=Veritas-5.0MP3RP2-07/07/09-11:27:00" vcs 5.0MP3RP2,REV=2009.07.07 VRTSllt "VERSION=5.0|PSTAMP=Veritas-5.0MP3RP2-07/07/09-11:27:00" llt 5.0MP3RP2,REV=2009.07.07 VRTSgab "VERSION=5.0|PSTAMP=Veritas-5.0MP3RP2-07/07/09-11:27:00" gab 5.0MP3RP2,REV=2009.07.07 VRTSperl "VERSION=5.8.8.0|PSTAMP=5.8.8.0 05012007" vperl 5.0,REV=2007.01.05 As suggested earlier, a file like this could be automatically generated if run on a system that already had the software stack installed. Thoughts? Best regards, Mark. |
|
From: Dagobert M. <da...@op...> - 2010-07-27 11:48:07
|
Hi Mark, Am 27.07.2010 um 13:36 schrieb ma...@pr...: >>> The other issue I'm going to be looking into shortly is that of >>> supporting multiple versions of the same package. >> >> Usually you use different catalogs for this and then use overlays in >> pkgutil.conf on all these, e.g. >> veritas41/ >> veritas50/ >> veritas51/ > > Yes that's feasible. Managing lots of different repos is a bit of a > headache though, don't you think, particularly the more software > stacks you have to maintain? Also, does it not make the command- > line rather long? Each time you run pkgutil you'd have to use the - > t option to specify a different repo, e.g. > > # pkgutil -t https://swrepo.mydomain.net/sw/current/veritas5.0 -i vcs > # pkgutil -t https://swrepo.mydomain.net/sw/current/veritas5.1 -i vcs I would have configured that statically in the local pkgutil.conf > vs. doing it in a single repo with different catalogue names, e.g. > > # pkgutil -i vcs50 > # pkgutil -i vcs51 > > But two problems with this. 1 - can't auto-generate the catalogue > name (re my earlier bldcat patch) if the catalogue name has to be > unique. 2 - the catalogue file uses package names in the dependency > list, which won't be unique, and should be modified to use something > that will be unique (such as the package filename). Yes. Yes. > We would need a radical fix to this, take a deep breath, here goes: > > The catalogue name can no longer be a unique identifier. The > package filename must become the unique identifier. > > Phew, radical stuff. I think I need to lie down. > > So then your instructions for sysadmins never change, the latest > version of VCS available in the production repo is always installed > by 'pkgutil -i vcs'. But in addition, you might be able to type: > > # pkgutil -i vcs -S 5.0MP3 > > for example, where our magic -S option (--select) tells pkgutil that > you know there are multiple entries for the catalogue name 'vcs', > the one you want is the one where the VERSION/REV string contains > the text '5.0MP3'. This would require a dependency-mechanism not depending on a package name, but a specific version of a package (e.g. derived from the catalogname). Only considering matching packages won't work in specific environments - Veritas prior to 3.5 comes to my mind where you had a match vcs 2.0 - vxvm 3.2 - vxfs 3.4 In the catalog-approach you would bundle all of these in one catalog and everything would be fine. Hand-tailoring the catalog would be very tedious here. IMHO having a drop-box folder for packages with as much automation as possible is favored as opposed to a manually crafted catalog. > We could be more radical still. Yes we could. We could allow > package filenames to contain directory names, so that we could > organise our repo more logically. So we could have your > > veritas41/ > veritas50/ > veritas51/ > > subdirectories in the repo if we chose, and the catalogue filename > would contain the subdirectory name as well. One catalogue, > multiple subdirectories. The end-user is oblivious to the fact that > when he says 'pkgutil -i vcs' that it is downloaded from a > subdirectory. > > How open is everyone to being radical with pkgutil? :-) A hacker is someone who abuses a tool for things it were never meant to do, not someone who puts a spoiler to a car to go faster. To me it looks the existing functionality in pkgutil is already capable of what you want to do, but you want to do it differently. Best regards -- Dago |
|
From: <ma...@pr...> - 2010-07-27 11:36:58
|
>> The other issue I'm going to be looking into shortly is that of >> supporting multiple versions of the same package. > > Usually you use different catalogs for this and then use overlays in > pkgutil.conf on all these, e.g. > veritas41/ > veritas50/ > veritas51/ Yes that's feasible. Managing lots of different repos is a bit of a headache though, don't you think, particularly the more software stacks you have to maintain? Also, does it not make the command-line rather long? Each time you run pkgutil you'd have to use the -t option to specify a different repo, e.g. # pkgutil -t https://swrepo.mydomain.net/sw/current/veritas5.0 -i vcs # pkgutil -t https://swrepo.mydomain.net/sw/current/veritas5.1 -i vcs vs. doing it in a single repo with different catalogue names, e.g. # pkgutil -i vcs50 # pkgutil -i vcs51 But two problems with this. 1 - can't auto-generate the catalogue name (re my earlier bldcat patch) if the catalogue name has to be unique. 2 - the catalogue file uses package names in the dependency list, which won't be unique, and should be modified to use something that will be unique (such as the package filename). We would need a radical fix to this, take a deep breath, here goes: The catalogue name can no longer be a unique identifier. The package filename must become the unique identifier. Phew, radical stuff. I think I need to lie down. So then your instructions for sysadmins never change, the latest version of VCS available in the production repo is always installed by 'pkgutil -i vcs'. But in addition, you might be able to type: # pkgutil -i vcs -S 5.0MP3 for example, where our magic -S option (--select) tells pkgutil that you know there are multiple entries for the catalogue name 'vcs', the one you want is the one where the VERSION/REV string contains the text '5.0MP3'. We could be more radical still. Yes we could. We could allow package filenames to contain directory names, so that we could organise our repo more logically. So we could have your veritas41/ veritas50/ veritas51/ subdirectories in the repo if we chose, and the catalogue filename would contain the subdirectory name as well. One catalogue, multiple subdirectories. The end-user is oblivious to the fact that when he says 'pkgutil -i vcs' that it is downloaded from a subdirectory. How open is everyone to being radical with pkgutil? :-) Best regards, Mark Bannister. |
|
From: <ma...@pr...> - 2010-07-27 10:48:21
|
On Mon 26/07/10 4:11 PM , Peter Bonivart bon...@op... sent: > On Mon, Jul 26, 2010 at 4:56 PM, ma...@pr... wrote: >> 1.0 vs. 1.0b2 would actually still work given the patch I presented you, >> because the b would be stripped leaving 1.0 vs. 1.02. What wouldn't have >> worked was 1.0 vs. 1.0b. > > Just a quick reply to this part, another will follow when I have more > time. > I chose that example intentionally since an official 1.0 should never > be replaced with a beta for the same version, that's a downgrade. I > meant 1.0 beta 2 with my 1.0b2. Maybe you read it as something else, > kind of proves my point... ;-) I think I understand your concern. But why would you put a beta package in a production repo? Surely you'd put it in a dev repo. In which case, yes of course if you request an upgrade of a package with pkgutil you expect to download the very latest, and if that's a beta, so be it. If you didn't really want the beta and wanted a different version, you can request that. But the intention is quite explicit in your pkgutil man page, that with -i when no specific version is requested, "the latest version found is chosen". The latest beta *is* the latest version. > I just see it as an endless stream of patches to plug new variations > that crop up but as long as it's isolated from the CSW way of doing > the compare I'm still not opposed to it and I will test your patch as > soon as possible. I've tightened my patch up even more, so that VERSION comparison will never occur on a CSW package (only the existing REV comparison). In order to get VERSION compare to work now, noncsw must be set to true, pkgcmpstyle must be set to 1, and the package name must not begin with the letters CSW. This is a surefire way that this code will never affect CSW. In order to do this, I've had to pass the package name as an extra arg to the verscmp subroutine. The attached patch needs to be applied after the big patch I sent you. Thanks, Mark. |
|
From: Dagobert M. <da...@op...> - 2010-07-27 07:12:19
|
Hi Mark, Am 26.07.2010 um 16:56 schrieb ma...@pr...: >> What we at CSW have found is that it's more or less impossible to >> compare unknown version strings and _know_ which one is newer, simply >> due to not knowing which ruleset the author of that software used. >> Not >> to mention when they all of a sudden change it. > > Understood we might not catch 100% of cases, but if we can catch 98% > of cases, the outlying 2% can be treated as exceptions. 1.0 vs. > 1.0b2 would actually still work given the patch I presented you, > because the b would be stripped leaving 1.0 vs. 1.02. What wouldn't > have worked was 1.0 vs. 1.0b. But I have yet to find an example > like this from one of our vendors (all the examples I can find will > work with the patch I sent you). Ok. But I would either rely 100% on the builtin-mechanism or 100% on the mapping file. Don't try to guess or hell breaks loose. >> How about adding a script to the pkgutilplus package that unpacks a >> package, adds a REV-field and repacks it again? ;-) > > Dago's suggestion of a separate mapfile is also a good idea, where > the comparison logic can be fine tuned, although expecting this to > be done for every non-CSW package is onerous. You would write a REV-string extractor for every vendor, for VRTS this could be taken from "PSTAMP=VERITAS-5.0:25-May-2006" > Start with good built-in logic that works for 98% of cases, then use > a mapfile to clean up the exceptions. ...and bail out on the remaining 2% if there is no mapfile. > I realise pkgutil was originally only written for CSW, but I believe > there is a gain to be had in building in full non-CSW support so a > wider audience can realise its benefits. It's a great tool that > plugs a sizeable gap in Solaris package management. > > The other issue I'm going to be looking into shortly is that of > supporting multiple versions of the same package. Usually you use different catalogs for this and then use overlays in pkgutil.conf on all these, e.g. veritas41/ veritas50/ veritas51/ > Can I check, is bldcat GPLed? It didn't say so in the comment at > the top of the script. I guess so. Best regards -- Dago |
|
From: <bon...@us...> - 2010-07-26 15:49:05
|
Revision: 281
http://pkgutil.svn.sourceforge.net/pkgutil/?rev=281&view=rev
Author: bonivart
Date: 2010-07-26 15:48:59 +0000 (Mon, 26 Jul 2010)
Log Message:
-----------
pkgutil: fix detecting end of REV list during compares (Mark Bannister)
Modified Paths:
--------------
trunk/pkgutil
trunk/readme
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: Peter B. <bon...@op...> - 2010-07-26 15:12:17
|
On Mon, Jul 26, 2010 at 4:56 PM, <ma...@pr...> wrote: > Understood we might not catch 100% of cases, but if we can catch 98% of cases, the outlying 2% can be treated as exceptions. 1.0 vs. 1.0b2 would actually still work given the patch I presented you, because the b would be stripped leaving 1.0 vs. 1.02. What wouldn't have worked was 1.0 vs. 1.0b. But I have yet to find an example like this from one of our vendors (all the examples I can find will work with the patch I sent you). Just a quick reply to this part, another will follow when I have more time. I chose that example intentionally since an official 1.0 should never be replaced with a beta for the same version, that's a downgrade. I meant 1.0 beta 2 with my 1.0b2. Maybe you read it as something else, kind of proves my point... ;-) I just see it as an endless stream of patches to plug new variations that crop up but as long as it's isolated from the CSW way of doing the compare I'm still not opposed to it and I will test your patch as soon as possible. -- /peter |
|
From: <ma...@pr...> - 2010-07-26 14:56:21
|
On Mon 26/07/10 2:01 PM , Peter Bonivart bon...@op... sent: > I was kind of expecting that your small first patch would lead to > something like this. ;-) Give an inch, I'll take a mile ... I was just warming up :-) > What we at CSW have found is that it's more or less impossible to > compare unknown version strings and _know_ which one is newer, simply > due to not knowing which ruleset the author of that software used. Not > to mention when they all of a sudden change it. Understood we might not catch 100% of cases, but if we can catch 98% of cases, the outlying 2% can be treated as exceptions. 1.0 vs. 1.0b2 would actually still work given the patch I presented you, because the b would be stripped leaving 1.0 vs. 1.02. What wouldn't have worked was 1.0 vs. 1.0b. But I have yet to find an example like this from one of our vendors (all the examples I can find will work with the patch I sent you). > How about adding a script to the pkgutilplus package that unpacks a > package, adds a REV-field and repacks it again? ;-) As Dago says with pkgedit, it's easy to do this. However, we don't want to get into vendor support entanglement. This is our issue. If we start modifying third party packages just to get them working in our repo, the vendor could - quite rightly - turn round and say "then we don't support you because you've tinkered with our software". Because pkgutil is so close to working with non-tampered packages, it's much easier to modify pkgutil to get it right than it is to start from scratch and write another version of pkgutil all over again that does almost everything the same. If, as I have done, that functionality is kept cleanly separated so that it does not interfere with CSW support. Dago's suggestion of a separate mapfile is also a good idea, where the comparison logic can be fine tuned, although expecting this to be done for every non-CSW package is onerous. Start with good built-in logic that works for 98% of cases, then use a mapfile to clean up the exceptions. I realise pkgutil was originally only written for CSW, but I believe there is a gain to be had in building in full non-CSW support so a wider audience can realise its benefits. It's a great tool that plugs a sizeable gap in Solaris package management. The other issue I'm going to be looking into shortly is that of supporting multiple versions of the same package. At the moment, and correct me if I'm wrong, pkgutil expects catalogue names to be unique. So the only way to support multiple versions in the same catalogue is to tweak the catalogue name. In fact a prime example, already in opencsw.org, is GCC, where you can request gcc2, gcc3 or gcc4. There are other examples on opencsw.org. In a large corporation, where there are always different versions of software stacks on the go at the same time, it would be easier if the software could be addressed using the same catalogue name, but where a specific version could be requested on the command-line. This is further complicated by the fact that the dependencies in the catalogue use package names - so dependencies on particular versions can only be specified if the package names differ too. Have you ever considered improving this? I'm happy to continue to send you patches for a variety of improvements to the tool, and I am as keen as you are to keep the tool easy to maintain and fully CSW compatible. I have more time I can devote to this over the next few weeks. Can I check, is bldcat GPLed? It didn't say so in the comment at the top of the script. Thanks, Mark. |
|
From: Dagobert M. <da...@op...> - 2010-07-26 14:13:45
|
Hi Peter, Am 26.07.2010 um 15:01 schrieb Peter Bonivart: > How about adding a script to the pkgutilplus package that unpacks a a > package, adds a REV-field and repacks it again? ;-) I am currently working on "pkgedit", which allows editing of every aspects of a package and repacking it as pristine as possible. However, this may not be necessary. It may be more feasible to have a simple mapfile which maps <pkg> <version> <rev> with the <rev> part be calculated by a customer-specific way (like invoking a script 'getpkgversion'). That way the packages could stay as they are but also the standard comparison could be used, it would just need to consider the mapfile on reading. Prefix-specific converters (like for VRTS) could be provided in a contrib/ directory. Best regards -- Dago |
|
From: Peter B. <bon...@op...> - 2010-07-26 13:01:50
|
On Mon, Jul 26, 2010 at 12:47 PM, <ma...@pr...> wrote:
> Investigating support for non-CSW packages further, we could do with some more changes in pkgutil. Presently, pkgutil will only ever look at the REV component of a VERSION string when determining which package is newer. This is fine for CSW packages, where there is a standard that is strictly followed for the formatting of the VERSION string in pkginfo. However, in order for us to support non-CSW packages in our internal repo, from a variety of third party vendors, we need to have a slightly looser interpretation of package version to cater for packages where the REV component is the sub-version (i.e. version takes precedence) or where the REV component is entirely missing.
I was kind of expecting that your small first patch would lead to
something like this. ;-)
What we at CSW have found is that it's more or less impossible to
compare unknown version strings and _know_ which one is newer, simply
due to not knowing which ruleset the author of that software used. Not
to mention when they all of a sudden change it.
That's why we decided to always add our own version, the REV-field, to
each package with a by us known content so we can easily and safely
decide what is newer.
> Here are some examples of what we needed:
>
> VERSION=5.0.8 is newer than VERSION=5.0.7
> VERSION=2.0,REV=16 is newer than VERSION=1.0,REV=18
Yes, those are easy because they can be numerically decided like our
REV-field. But how about 1.0 vs. 1.0b2? If the packages are from a 3rd
party and you're not willing to normalize the version you need to
handle everything possible and I guarantee someone will think of
something you didn't. :-)
> I've thought hard about how to support this in pkgutil without breaking CSW packages, and came to the conclusion that the best way was to add a new option in pkgutil.conf that would add this needed functionality, which by default would be switched off.
It would definitely be optional behavior. It must be isolated as much
as possible from the current behavior.
> While working on this, I also spotted and fixed a bug in the verscmp subroutine, where this check:
> if (! $p2list[$i]) {
> $update = 1;
> last;
> }
>
> Should have said:
> if ($i >= scalar(@p2list)) {
> $update = 1;
> last;
> }
>
> To avoid an error message about a non-initialised array element, and also to avoid the scenario where 5.0.8 would not be greater than 5.0.7 because of the zero in the second position.
Thanks, I will test this immediately. If this is good it will be in 2.1.
> By the way, I may have more suggested changes for you in the next few days. When do you plan to release 2.1?
2.1 will go out shortly, it's feature frozen already. This is way to
big a patch to be included. But keep on coming with the requests. :-)
> I hope you get the patches ok and are happy to accept them. I'm happy to discuss alternative solutions.
I will test them but I don't promise anything since I'm not really
happy about the growing non-CSW support. From the start I didn't
support it at all but more and more people asked for it. It does
complicate the code and increases the risk of bugs, especially since
the "unknown" factor is always there, what will people use it with
next?
How about adding a script to the pkgutilplus package that unpacks a a
package, adds a REV-field and repacks it again? ;-) Seriously, that's
simplified what a contributed script does today to make use of SUNW
packages possible. Earlier I got two separate huge patches to support
SUNW packages directly in pkgutil and they were very invasive and hard
to test so when this variant surfaced I included it immediately
because it doesn't change core pkgutil at all. It's all done
externally to make packages compliant before we use them in pkgutil. I
loved that. :-)
Just a thought.
--
/peter
|
|
From: <ma...@pr...> - 2010-07-26 12:32:19
|
Following on from the changes I submitted for pkgutil, again to support non-standard packages in a local (non-CSW) repository, I have found it necessary to make a modification to the behaviour of bldcat. To support a repository of third party packages, we would like to use pkgutil without making any changes to those packages. During testing, I've assessed pkgutil and bldcat and determined that the requirement to encode the catalogue name as the first word of the NAME parameter inside pkginfo is only a requirement of bldcat. If I manually add catalogue entries with a catalogue name (common name) of my choice, pkgutil works fine. So, rather than change large numbers of third party packages just so that bldcat can create a catalogue file, it seemed the more sensible approach to modify bldcat to auto-generate the catalogue name from the package name on non-CSW packages, if supplied with the -a (or --auto) command-line option. With the attached patch, if you run: # bldcat --auto . ...all packages that do not begin with uppercase CSW will have their catalogue name taken from the package name, by stripping off the leading capital letters. So, for example, VRTSvcs becomes 'vcs'. This, along with the VERSION comparison changes I submitted earlier for pkgutil, means that we are now able to fully support non-CSW packages in our repository, without making any changes to those packages. I have also built in support for automatically GPG signing the catalogue. pkgutil can already check the catalogue to see if it is GPG signed, so it didn't make sense that bldcat couldn't build a signed catalogue: # bldcat --gpg . ...will run 'gpg --clearsign' on the catalogue file once it has been generated. I also fixed a bug whereby bldcat would generate warnings when processing a previously signed catalogue file. I hope you find the patch acceptable and are able to include it in your next release. I have one question. If I run 'bldcat .' I get a catalogue file that pkgutil can use. If I run 'bldcat some/path/to/some/directory' the resulting catalogue file cannot be used by pkgutil because the pathname has been encoded in the catalogue. Is this a deliberate feature, or is this a bug? Thanks & regards, Mark Bannister. |
|
From: <ma...@pr...> - 2010-07-26 10:52:50
|
On Mon 26/07/10 11:47 AM , ma...@pr... sent: <snip> > I hope you get the patches ok and are happy to accept them. I'm happy to > discuss alternative solutions. > Thanks & regards, > Mark Bannister. Yuck, this mail client is horrible and strips all the formatting. Patches attached as files instead. Thanks again, Mark. |
|
From: <ma...@pr...> - 2010-07-26 10:47:39
|
> On Sat 24/07/10 2:40 PM , Peter Bonivart bon...@op... sent:
>> On Fri, Jul 23, 2010 at 12:24 PM, ma...@pr... wrote:
>> Can I suggest that the following patch is accepted
>> into the next release of pkgutil?
<snip>
>
> Since It's a numerical comparison it could be considered a bug not
> cleaning up the values so I have checked in your change, it will be in
> 2.1 released soon. Thank you.
>
> --
> /peter
Thanks Peter.
Investigating support for non-CSW packages further, we could do with some more changes in pkgutil. Presently, pkgutil will only ever look at the REV component of a VERSION string when determining which package is newer. This is fine for CSW packages, where there is a standard that is strictly followed for the formatting of the VERSION string in pkginfo. However, in order for us to support non-CSW packages in our internal repo, from a variety of third party vendors, we need to have a slightly looser interpretation of package version to cater for packages where the REV component is the sub-version (i.e. version takes precedence) or where the REV component is entirely missing.
Here are some examples of what we needed:
VERSION=5.0.8 is newer than VERSION=5.0.7
VERSION=2.0,REV=16 is newer than VERSION=1.0,REV=18
I've thought hard about how to support this in pkgutil without breaking CSW packages, and came to the conclusion that the best way was to add a new option in pkgutil.conf that would add this needed functionality, which by default would be switched off.
While working on this, I also spotted and fixed a bug in the verscmp subroutine, where this check:
if (! $p2list[$i]) {
$update = 1;
last;
}
Should have said:
if ($i >= scalar(@p2list)) {
$update = 1;
last;
}
To avoid an error message about a non-initialised array element, and also to avoid the scenario where 5.0.8 would not be greater than 5.0.7 because of the zero in the second position.
By the way, I may have more suggested changes for you in the next few days. When do you plan to release 2.1?
The patches below also incorporate the patch I sent on Friday, so you will probably need to back that one out first. The intention is that to enable this functionality, both of the following configuration options need to be set:
noncsw=true
pkgcmpstyle=1
Here is a patch for pkgutil.conf:
*** pkgutil.conf Mon Jul 26 11:29:34 2010
--- pkgutil.conf.new Mon Jul 26 11:10:45 2010
***************
*** 63,68 ****
--- 63,78 ----
# Default: 0
pkgliststyle=1
+ # Style of package comparison used when checking for newer packages. 0 tells
+ # pkgutil to compare only the REV component of the VERSION string, and this is
+ # the correct behaviour for CSW packages. If you want to support non-CSW
+ # packages that do not always have a REV component, setting this to 1 will
+ # request that pkgutil compares the main VERSION string first, and then it
+ # will only look at the REV component if the main VERSION string is identical.
+ # The non-CSW behaviour can only be enabled when noncsw is set to true.
+ # Default: 0
+ pkgcmpstyle=1
+
# How to handle soft errors from hooks that are called. A hook that exits
# with code 1 will cause the pkgutil to stop. If this value is true,
# exit code 2 will also cause pkgutil to stop. Exit code 2 is a non-fatal
Here is the patch for pkgutil (quite a big one, sorry about that):
*** 2.1b1/pkgutil Sat Jul 17 15:44:53 2010
--- pkgutil-2.1b1-edited Mon Jul 26 11:22:10 2010
***************
*** 39,44 ****
--- 39,45 ----
my $use_gpg = 0;
my $noncsw = 0;
my $pkgliststyle = 0;
+ my $pkgcmpstyle = 0;
my $maxpkglist = 100000;
my @mirror;
my $defaultmirror = ("http://ibiblio.org/pub/packages/solaris/opencsw/current");
***************
*** 476,481 ****
--- 477,483 ----
"use_gpg" => "",
"use_md5" => "",
"pkgliststyle" => "",
+ "pkgcmpstyle" => "",
"maxpkglist" => "",
"noncsw" => "",
"stop_on_hook_soft_error" => "",
***************
*** 539,544 ****
--- 541,547 ----
}
$pkgliststyle = $config{pkgliststyle} if defined $config{pkgliststyle};
+ $pkgcmpstyle = $config{pkgcmpstyle} if defined $config{pkgcmpstyle};
$maxpkglist = $config{maxpkglist} if $config{maxpkglist};
$use_md5 = $use_gpg = $noncsw = 0;
$use_md5 = 1 if ($config{use_md5} eq "true");
***************
*** 1432,1438 ****
print "mirror\t\t\t" . ((scalar(@{$config{mirror}})) ? join("\n\t\t", @{$config{mirror}}) : "not set") . "\n\t\t\t(default: $defaultmirror)\n";
print "noncsw\t\t\t" . (($noncsw) ? $noncsw : "false") . " (default: false)\n";
print "pkgaddopts\t\t" . (($config{pkgaddopts}) ? $config{pkgaddopts} : "not set") . " (default: none)\n";
! print "pkgliststyle\t\t$pkgliststyle (default: 0)\n";
print "root_path\t\t" . (($config{root_path}) ? $config{root_path} : "not set") . " (default: /)\n";
print "stop_on_hook_soft_error\t" . (($config{stop_on_hook_soft_error}) ? $config{stop_on_hook_soft_error} : "not set") . " (default: false)\n";
print "use_gpg\t\t\t" . (($use_gpg) ? "true" : "false") . " (default: false)\n";
--- 1435,1442 ----
print "mirror\t\t\t" . ((scalar(@{$config{mirror}})) ? join("\n\t\t", @{$config{mirror}}) : "not set") . "\n\t\t\t(default: $defaultmirror)\n";
print "noncsw\t\t\t" . (($noncsw) ? $noncsw : "false") . " (default: false)\n";
print "pkgaddopts\t\t" . (($config{pkgaddopts}) ? $config{pkgaddopts} : "not set") . " (default: none)\n";
! print "pkgliststyle\t\t" . (($config{pkgliststyle}) ? $config{pkgliststyle} : "not set") . " (default: 0)\n";
! print "pkgcmpstyle\t\t" . (($config{pkgcmpstyle}) ? $config{pkgcmpstyle} : "not set") . " (default: 0)\n";
print "root_path\t\t" . (($config{root_path}) ? $config{root_path} : "not set") . " (default: /)\n";
print "stop_on_hook_soft_error\t" . (($config{stop_on_hook_soft_error}) ? $config{stop_on_hook_soft_error} : "not set") . " (default: false)\n";
print "use_gpg\t\t\t" . (($use_gpg) ? "true" : "false") . " (default: false)\n";
***************
*** 1732,1738 ****
# Comparison of two package versions as per
# http://pkgutil.net/get-install-and-configure#toc8
# As per cmp or <=>, -1, 0, or 1 if p1 is less than, equal to or greater than p2
! # Note that if neither has a REV code, -1 is *ALWAYS* returned.
# p1rev, p2rev - versions to compare
sub verscmp {
my($p1rev,$p2rev) = @_; # crev (new), irev (old), then 1 => upgrade
--- 1736,1743 ----
# Comparison of two package versions as per
# http://pkgutil.net/get-install-and-configure#toc8
# As per cmp or <=>, -1, 0, or 1 if p1 is less than, equal to or greater than p2
! # Note that if neither has a REV code, -1 is *ALWAYS* returned, unless
! # noncsw is true and pkgcmpstyle is 1.
# p1rev, p2rev - versions to compare
sub verscmp {
my($p1rev,$p2rev) = @_; # crev (new), irev (old), then 1 => upgrade
***************
*** 1758,1766 ****
}
}
! # 3rd case: installed REV, catalog REV => if newer, upgrade
if (! $skip) {
if ($p1rev =~ /REV=/ && $p2rev =~ /REV=/) {
if ($p1rev =~ /rev=/) {
($p1tmp) = ($p1rev =~ /REV=(.+)_/);
} else {
--- 1763,1814 ----
}
}
! # 3rd case: check main VERSION string => if newer, upgrade
! # (only if noncsw is true and pkgcmpstyle is 1)
! if (! $skip && $noncsw && $pkgcmpstyle) {
! print STDERR "Comparing VERSION strings\n" if $debug;
! $p1tmp = $p1rev;
! $p1tmp =~ s/,REV=.*$//;
! $p2tmp = $p2rev;
! $p2tmp =~ s/,REV=.*$//;
!
! if ($p1tmp ne $p2tmp) {
! # VERSION component is not equal
! @p1list = split(/\./,$p1tmp);
! @p2list = split(/\./,$p2tmp);
!
! for (my $i = 0; $i < scalar(@p1list); $i++) {
! if ($i >= scalar(@p2list)) {
! $update = 1;
! last;
! }
! $p1list[$i] =~ s/[^0-9]//g;
! $p2list[$i] =~ s/[^0-9]//g;
!
! if ($p1list[$i] != $p2list[$i]) {
! if ($p1list[$i] > $p2list[$i]) {
! $update = 1;
! } else {
! $update = -1;
! }
! print STDERR "$i $p1list[$i] $p2list[$i] $update\n" if $debug;
! last;
! }
! print STDERR "$i $p1list[$i] $p2list[$i] $update\n" if $debug;
! }
! $update = -1 if (! $update && scalar(@p2list) > scalar(@p1list));
! $skip = 1;
! } elsif ($p1rev !~ /REV=/ && $p2rev !~ /REV=/) {
! # VERSION component is equal and there is not REV component
! $update = 0;
! $skip = 1;
! }
! }
!
! # 4th case: installed REV, catalog REV => if newer, upgrade
if (! $skip) {
if ($p1rev =~ /REV=/ && $p2rev =~ /REV=/) {
+ print STDERR "Comparing REV strings\n" if $debug;
if ($p1rev =~ /rev=/) {
($p1tmp) = ($p1rev =~ /REV=(.+)_/);
} else {
***************
*** 1776,1785 ****
@p2list = split(/\./,$p2tmp);
for (my $i = 0; $i < scalar(@p1list); $i++) {
! if (! $p2list[$i]) {
$update = 1;
last;
}
if ($p1list[$i] != $p2list[$i]) {
if ($p1list[$i] > $p2list[$i]) {
$update = 1;
--- 1824,1836 ----
@p2list = split(/\./,$p2tmp);
for (my $i = 0; $i < scalar(@p1list); $i++) {
! if ($i >= scalar(@p2list)) {
$update = 1;
last;
}
+ $p1list[$i] =~ s/[^0-9]//g;
+ $p2list[$i] =~ s/[^0-9]//g;
+
if ($p1list[$i] != $p2list[$i]) {
if ($p1list[$i] > $p2list[$i]) {
$update = 1;
***************
*** 1797,1803 ****
}
}
! # 4th case: installed no REV, catalog no REV => upgrade
if (! $skip) {
$update = 1 if ($p1rev !~ /REV=/ && $p2rev !~ /REV=/);
}
--- 1848,1854 ----
}
}
! # 5th case: installed no REV, catalog no REV => upgrade
if (! $skip) {
$update = 1 if ($p1rev !~ /REV=/ && $p2rev !~ /REV=/);
}
***************
*** 2042,2047 ****
--- 2093,2102 ----
Style of package list when installing/upgrading packages. 0 is the one used since the first version of pkgutil, it's space efficient but less readable. 1 is the one introduced in v1.7, it's one package per line which is easier to read.
+ C<pkgcmpstyle>
+
+ Style of package comparison used when checking for newer packages. 0 tells pkgutil to compare only the REV component of the VERSION string, and this is the correct behaviour for CSW packages. If you want to support non-CSW packages that do not always have a REV component, setting this to 1 will request that pkgutil compares the main VERSION string first, and then it will only look at the REV component if the main VERSION string is identical. The non-CSW behaviour can only be enabled when noncsw is set to true.
+
C<root_path>
Set alternate root path (-R with pkg commands). Default is /.
I hope you get the patches ok and are happy to accept them. I'm happy to discuss alternative solutions.
Thanks & regards,
Mark Bannister.
|
|
From: <bon...@us...> - 2010-07-25 18:46:57
|
Revision: 280
http://pkgutil.svn.sourceforge.net/pkgutil/?rev=280&view=rev
Author: bonivart
Date: 2010-07-25 18:46:51 +0000 (Sun, 25 Jul 2010)
Log Message:
-----------
readme: updated for beta 2
Modified Paths:
--------------
trunk/readme
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: Peter B. <bon...@op...> - 2010-07-24 13:40:59
|
On Fri, Jul 23, 2010 at 12:24 PM, <ma...@pr...> wrote:
> Can I suggest that the following patch is accepted into the next release of
> pkgutil?
>
> *** 2.1b1/pkgutil Sat Jul 17 15:44:53 2010
> --- pkgutil-2.1b1-edited Fri Jul 23 11:00:14 2010
> ***************
> *** 1776,1781 ****
> --- 1776,1783 ----
> @p2list = split(/\./,$p2tmp);
>
> for (my $i = 0; $i < scalar(@p1list); $i++) {
> + $p1list[$i] =~ s/[^0-9]//g;
> + $p2list[$i] =~ s/[^0-9]//g;
> if (! $p2list[$i]) {
> $update = 1;
> last;
>
> The reason we need this is we are using pkgutil against our own internal
> repository, and we have in-house developed packages and third party packages
> with non-standard VERSION parameters, such as:
>
> VERSION=2.0p1,REV=16_sun
> or
> VERSION=5.0,REV=5.0A55_sol
>
> pkgutil currently splits the REV component by dots and then performs a
> numerical comparison on each element in order to determine which package is
> more current. This works for us just fine, except that the components are
> not entirely numerical so we get the following errors:
>
> Argument "16_sun" isn't numeric in numeric ne (!=) at /opt/csw/bin/pkgutil
> line 1783.
> Argument "18_sun" isn't numeric in numeric ne (!=) at /opt/csw/bin/pkgutil
> line 1783.
>
> The patch above fixes this issue by stripping all non-digits from an element
> before performing the comparison.
Since It's a numerical comparison it could be considered a bug not
cleaning up the values so I have checked in your change, it will be in
2.1 released soon. Thank you.
--
/peter
|
|
From: <bon...@us...> - 2010-07-23 21:49:45
|
Revision: 279
http://pkgutil.svn.sourceforge.net/pkgutil/?rev=279&view=rev
Author: bonivart
Date: 2010-07-23 21:49:39 +0000 (Fri, 23 Jul 2010)
Log Message:
-----------
pkgutil: clean version field from non-digits when doing compares (Mark)
Modified Paths:
--------------
trunk/pkgutil
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <ma...@pr...> - 2010-07-23 10:55:46
|
BODY { font-family:Arial, Helvetica, sans-serif;font-size:12px; }Can
I suggest that the following patch is accepted into the next release
of pkgutil?
*** 2.1b1/pkgutil Sat Jul 17 15:44:53 2010
--- pkgutil-2.1b1-edited Fri Jul 23 11:00:14 2010
***************
*** 1776,1781 ****
--- 1776,1783 ----
@p2list = split(/./,$p2tmp);
for (my $i = 0; $i < scalar(@p1list); $i++) {
+ $p1list[$i] =~ s/[^0-9]//g;
+ $p2list[$i] =~ s/[^0-9]//g;
if (! $p2list[$i]) {
$update = 1;
last;
The reason we need this is we are using pkgutil against our own
internal repository, and we have in-house developed packages and
third party packages with non-standard VERSION parameters, such as:
VERSION=2.0p1,REV=16_sun
or
VERSION=5.0,REV=5.0A55_sol
pkgutil currently splits the REV component by dots and then performs
a numerical comparison on each element in order to determine which
package is more current. This works for us just fine, except that
the components are not entirely numerical so we get the following
errors:
Argument "16_sun" isn't numeric in numeric ne (!=) at
/opt/csw/bin/pkgutil line 1783.
Argument "18_sun" isn't numeric in numeric ne (!=) at
/opt/csw/bin/pkgutil line 1783.
The patch above fixes this issue by stripping all non-digits from an
element before performing the comparison.
Thanks & regards,
Mark.
|
|
From: <bon...@us...> - 2010-07-18 14:46:27
|
Revision: 278
http://pkgutil.svn.sourceforge.net/pkgutil/?rev=278&view=rev
Author: bonivart
Date: 2010-07-18 14:46:21 +0000 (Sun, 18 Jul 2010)
Log Message:
-----------
pkgutil: update readme
Modified Paths:
--------------
trunk/readme
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <bon...@us...> - 2010-07-17 13:54:01
|
Revision: 277
http://pkgutil.svn.sourceforge.net/pkgutil/?rev=277&view=rev
Author: bonivart
Date: 2010-07-17 13:53:53 +0000 (Sat, 17 Jul 2010)
Log Message:
-----------
pkgutil: better support root_path
Modified Paths:
--------------
trunk/pkgutil
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
