From: Charles W. <cwi...@us...> - 2010-04-19 23:14:57
|
I'm nearing completion of the "grand rebuild" of all the msys components in the distro. So, once I do that, then the next step I believe is to construct an appropriate distribution manifest file for mingw-get. But... How do we want to manage this? I'd prefer to upload separate "fragments" for each package/subsys/release, and then "magically" have the /real/ distribution manifest file be autogenerated from those, on sourceforge. This has both pros and cons (and one really big con): pro: decentralized management. If Cesar uploads an updated, say, coreutils-7.0, then he can also upload a specific coreutils-7.0-1-msys.xml fragment, which should get automatically added in, without clobbering the existing(?) coreutils-5.92-3-msys.xml entry. con: doesn't really work well with "meta packages" (e.g. 'package-group's), because IIUC each package group's membership list will have to be manually updated in the xml for that meta package to "pick up" the newest upload for end users automatically. So, you're still hand editing the "real" xml -- or at least some other xml fragment for the msys-dtk.package-group. However, the end user can always manually select the new coreutils. BIG CON: I understand we do not have sufficient flexibility to run a job on the sourceforge server to generate the distribution manifest at all. Plus, even if we COULD get the cron server working properly, we don't really have control over the true directory layout in the files area (the one you see on the MinGW Download page is fictional, generated by sourceforge's backend database). So...how do we even tell this imaginary cron job where to look for these fragments -- and since files NEVER disappear, how do we tell it to ignore one that is out-of-date? Ick. Moderately big con: even if all of the above weren't bad enough, to generate a distribution manifest from a collection of other files is a non-trivial undertaking. Since you might have coreutils-5.92-3-msys-1.0.13-* AND coreutils-7.0-1-msys-1.0.16-* each with different collections of tarballs (say, 7.0 has a '-lang' component but 5.92 does not, which implies that 7.0 has a different set of dependencies). This means that you have to *merge* the two xml fragments when generating the portion of the distribution manifest related to the coreutils packages. Merging xml entities -- without screwing it up -- is a non-trivial task. So...even though I'd LIKE to be able to use a distributed-fragment scheme, I don't think we really can. Which means we need to have one-xml-to-rule-them-all, somewhere, where the various people with upload access can also simultaneously update that file. CVS? wiki? ongoing revisions simply posted to the mailing list? </me runs...> After a little more thought, I don't think we really lose TOO much doing it this way. Unlike cygwin (with 57 active maintainers and 1471 separate packages), we don't really have THAT many. Maybe 5 active maintainers and ~200 packages, if you count all the various sub-components. And they don't REALLY get updated all that often. Comments? -- Chuck |
From: Keith M. <kei...@us...> - 2010-04-20 20:03:15
Attachments:
package-list.xml
msys-base.xml
|
On Tuesday 20 April 2010 00:14:24 Charles Wilson wrote: > I'm nearing completion of the "grand rebuild" of all the msys > components in the distro. Excellent! > So, once I do that, then the next step > I believe is to construct an appropriate distribution manifest > file for mingw-get. Entirely peripheral, but I've recently being playing with the PortableApps.com framework to create MSYS-on-a-Stick, for use at work; in parallel with setting it up, I've started to cobble up a (still very much incomplete) msys-base.xml manifest (attached), so I can use mingw-get to help me to piece it together. > But... > > How do we want to manage this? I'd prefer to upload separate > "fragments" for each package/subsys/release, and then "magically" > have the /real/ distribution manifest file be autogenerated from > those, on sourceforge. Probably not as fine grained as you are thinking of, but mingw-get CVS is now able to construct the distribution manifest from multiple secondary files, with one top-level master index file, interpreted recursively to incorporate the secondaries. Now might be a good time to push out mingw-get-alpha-2. > This has both pros and cons (and one really big con): > > pro: decentralized management. If Cesar uploads an updated, say, > coreutils-7.0, then he can also upload a specific > coreutils-7.0-1-msys.xml fragment, which should get automatically > added in, without clobbering the existing(?) > coreutils-5.92-3-msys.xml entry. Actually, for the time being, the existing entry will need to remain in place, alongside any new release spec, otherwise the older release will not be considered as a possible existing installed component. (This is one of the items on my "FIXME" list). > con: doesn't really work well with "meta packages" (e.g. > 'package-group's), because IIUC each package group's membership > list will have to be manually updated in the xml for that meta > package to "pick up" the newest upload for end users automatically. Nope. The meta-package can specify (e.g.): <requires ge="msys-foo-*-msys-*-bin.tar" /> and it will automatically reference the latest available release of foo-bin for msys; (see how I've started to set up "msys-base"). > So, you're still hand editing the "real" xml -- or > at least some other xml fragment for the msys-dtk.package-group. That's how I'm doing it at the moment, and I don't see it changing in the near future. Keeping the manifest in small recursively included sub-files may help to make it more manageable. > However, the end user can always manually select the new > coreutils. Ultimately, yes that capability is a "must"; not sure, at the moment, how it will work. > BIG CON: I understand we do not have sufficient flexibility to run > a job on the sourceforge server to generate the distribution > manifest at all. Plus, even if we COULD get the cron server > working properly, we don't really have control over the true > directory layout in the files area (the one you see on the MinGW > Download page is fictional, generated by sourceforge's backend > database). So...how do we even tell this imaginary cron job where > to look for these fragments -- and since files NEVER disappear, > how do we tell it to ignore one that is out-of-date? > > Ick. Ick, indeed. I don't think it would even be possible, to actually *assemble* the manifest, within the confines of the FRS. Do note that, while SF assure us they will keep files forever, it does seem to be possible to *overwrite* an existing file, with new content; delete the existing file record from the FRS database, then upload a fresh copy with identically the same name. > Moderately big con: even if all of the above weren't bad enough, > to generate a distribution manifest from a collection of other > files is a non-trivial undertaking. Since you might have > coreutils-5.92-3-msys-1.0.13-* > AND coreutils-7.0-1-msys-1.0.16-* > each with different collections of tarballs (say, 7.0 has a > '-lang' component but 5.92 does not, which implies that 7.0 has a > different set of dependencies). This means that you have to > *merge* the two xml fragments when generating the portion of the > distribution manifest related to the coreutils packages. It would look something like this:-- <package name="msys-coreutils" alias="coreutils"> <affiliate group="MSYS Base System" /> <requires ge="msysCORE-*-msys-*-bin.tar" /> <component class="bin"> <description lang="en" title="The GNU Core Utilities"> <paragraph> Some generic text, describing the coreutils package. </paragraph> </description> <release tarname="coreutils-5.97-2-msys-1.0.11-bin.tar.lzma"> <requires ge="foo-*-msys-1.0.11-bin.tar" /> </release> <release tarname="coreutils-7.0-1-msys-1.0.16-bin.tar.lzma"> <requires ge="msys-foo-*-msys-1.0.16-bin.tar" /> <requires ge="msys-bar-*-msys-1.0.16-bin.tar" /> </release> </component> </package> > Merging xml entities -- without screwing it up -- is a non-trivial > task. ...and not something I want to think about automating right now. > So...even though I'd LIKE to be able to use a distributed-fragment > scheme, I don't think we really can. Which means we need to have > one-xml-to-rule-them-all, somewhere, where the various people with > upload access can also simultaneously update that file. > > CVS? That would be my choice; however, we do need to agree how the modules layout should be structured. > wiki? Not keen on that idea. > ongoing revisions simply posted to the mailing list? </me runs...> Then someone needs to take ownership, and ensure that patches are both regularly, and routinely applied. I'd rather trust individual package maintainers to apply their own updates in CVS, then push the result to FRS. > After a little more thought, I don't think we really lose TOO much > doing it this way... No, I don't think so either. -- Regards, Keith. |
From: Charles W. <cwi...@us...> - 2010-04-21 00:13:49
|
On 4/20/2010 9:39 AM, Keith Marshall wrote: > On Tuesday 20 April 2010 00:14:24 Charles Wilson wrote: >> ... > Entirely peripheral, but I've recently being playing with the > PortableApps.com framework to create MSYS-on-a-Stick, for use at > work; in parallel with setting it up, I've started to cobble up > a (still very much incomplete) msys-base.xml manifest (attached), > so I can use mingw-get to help me to piece it together. Ah, just the sort of thing I was looking for. >> How do we want to manage this? > Probably not as fine grained as you are thinking of, but mingw-get > CVS is now able to construct the distribution manifest from multiple > secondary files, with one top-level master index file, interpreted > recursively to incorporate the secondaries. But, AFAICT, only at specific "customization points": e.g. entire <package-list> elements. That is: <package-list catalogue="msys-base" /> "#includes" msys-base.xml. But, in msys-base.xml can you do: <package catalogue="coreutils"> which "#includes" coreutils.xml? (FYI: Thunderbird appears to be US-centric. It's complaining about 'catalogue' being misspelled, but is perfectly happy with 'catalog') I realize we can't break it down further. E.g. this is right out: <release catalogue="coreutils-5.92-3"/> <release catalogue="coreutils-7.0-1"/> The reason (for those following along at home) is that "merging xml entities" problem. You WANT the "coreutils" fragment to look like this: <package name="coreutils"> <affiliate group="MSYS Base System" /> <component class="bin"> <release tarname="coreutils-5.97-2-msys-1.0.11-bin.tar.lzma" /> <release tarname="coreutils-5.97-3-msys-1.0.13-bin.tar.lzma" /> </component> <component class="ext"> <release tarname="coreutils-5.97-2-msys-1.0.11-ext.tar.lzma" /> <release tarname="coreutils-5.97-3-msys-1.0.13-ext.tar.lzma" /> </component> <component class="lic"> <release tarname="coreutils-5.97-2-msys-1.0.11-lic.tar.lzma" /> <release tarname="coreutils-5.97-3-msys-1.0.13-lic.tar.lzma" /> </component> <component class="lang"> <release tarname="coreutils-5.97-3-msys-1.0.13-lang.tar.lzma" /> </component> </package> But if you specify two different .xml files, one for each release, you'd have: <package name="coreutils"> <affiliate group="MSYS Base System" /> <component class="bin"> <release tarname="coreutils-5.97-2-msys-1.0.11-bin.tar.lzma" /> </component> <component class="ext"> <release tarname="coreutils-5.97-2-msys-1.0.11-ext.tar.lzma" /> </component> <component class="lic"> <release tarname="coreutils-5.97-2-msys-1.0.11-lic.tar.lzma" /> </component> </package> and <package name="coreutils"> <affiliate group="MSYS Base System" /> <component class="bin"> <release tarname="coreutils-5.97-3-msys-1.0.13-bin.tar.lzma" /> </component> <component class="ext"> <release tarname="coreutils-5.97-3-msys-1.0.13-ext.tar.lzma" /> </component> <component class="lic"> <release tarname="coreutils-5.97-3-msys-1.0.13-lic.tar.lzma" /> </component> <component class="lang"> <release tarname="coreutils-5.97-3-msys-1.0.13-lang.tar.lzma" /> </component> </package> Merging those automatically is...not fun. And what if a typo sneaks in to the "<affiliate />" entity? So that 5.97-2 is affiliated with "MSYS Base System" but 5.97-3 is affiliated with "MSYS Baes Ssytm" -- how do you construct the "merged" result? Worse, we already have about 50 "package groups", if you subdivide at the level of "unique -src packages" without regard to version. (e.g. all of the "gettext/libintl" components are one "group". That's already an awful lot of separate http connections for mingw-get to make. We REALLY don't want to add more for each new release of each package group. No, we definitely do not want to go there. > > Now might be a good time to push out mingw-get-alpha-2. Sure! >> pro: decentralized management. > > Actually, for the time being, the existing entry will need to remain > in place, alongside any new release spec, otherwise the older release > will not be considered as a possible existing installed component. > (This is one of the items on my "FIXME" list). Right. So, I'm thinking IF we were to go as fine-grained as "one xml file per unique source tarball, without regard to version", then you'd need to edit/commit a new (e.g.) gettext.xml each time you upload a new set of gettext-0.17-2-msys-1.0.13-bin.tar.lzma gettext-0.17-2-msys-1.0.13-dev.tar.lzma gettext-0.17-2-msys-1.0.13-doc.tar.lzma gettext-0.17-2-msys-1.0.13-ext.tar.lzma gettext-0.17-2-msys-1.0.13-lic.tar.lzma gettext-0.17-2-msys-1.0.13-src.tar.lzma libasprintf-0.17-2-msys-dll-0.tar.lzma libgettextpo-0.17-2-msys-dll-0.tar.lzma libintl-0.17-2-msys-dll-8.tar.lzma adding new entries for those 9 tarballs (actually, 8, because -src doesn't get an entry -- or does it [*]), without removing any existing elements from the xml. [*] You mentioned a possible element type, sibling to <component>: <source tarname="gettext-%-msys-%-src.tar.lzma"> but I'm not sure if that's implemented yet, whether '&' or '%' was used for the "matching version number" wildcard, or whether you can use it twice to match both package version/release and subsystem- version. >> con: doesn't really work well with "meta packages" > Nope. The meta-package can specify (e.g.): > > <requires ge="msys-foo-*-msys-*-bin.tar" /> > > and it will automatically reference the latest available release of > foo-bin for msys; (see how I've started to set up "msys-base"). Ah, nice. >> So, you're still hand editing the "real" xml -- or >> at least some other xml fragment for the msys-dtk.package-group. > > That's how I'm doing it at the moment, and I don't see it changing in > the near future. Keeping the manifest in small recursively included > sub-files may help to make it more manageable. Yep -- but it's going to be a balance between maintenance and how many separate downloads of tiny little xml files mingw-get has to make, to construct its internal distribution manifest (since we can't, and shouldn't, offload that process to a sourceforge-hosted cronjob) >> So...how do we even tell this imaginary cron job where >> to look for these fragments -- and since files NEVER disappear, >> how do we tell it to ignore one that is out-of-date? >> >> Ick. > > Ick, indeed. I don't think it would even be possible, to actually > *assemble* the manifest, within the confines of the FRS. Seems that way. > Do note that, while SF assure us they will keep files forever, it > does seem to be possible to *overwrite* an existing file, with new > content; delete the existing file record from the FRS database, then > upload a fresh copy with identically the same name. That's true. (But bloody awkward: ok, I'll upload this 0-byte file with the same name, and that'll be our little code for "ignore me, I've been deleted"...) >> Merging xml entities -- without screwing it up -- is a non-trivial >> task. > > ...and not something I want to think about automating right now. Yep. >> [where to maintain one-big-xml, or many-smaller-xml] >> >> CVS? > > That would be my choice; however, we do need to agree how the modules > layout should be structured. Seems like a single directory, with individual files for each set of packages derived from the same source lineage, would be workable. Merging closely related groups together wouldn't hurt, either. I might combine all of the automakeX.Y collections (and the am-wrapper) into one big mingw-automake.xml; ditto mingw-autoconfX.Y and the ac-wrapper. That's most of the content. The rest, like the various meta-packages such as "msys-base" or "msys-dtk" can probably live in the appropriate top-level <package-list> xml. Of course, if somebody names their demonstration msys package list catalog 'msys-base': <package-list catalogue="msys-base"> And then, wants to specify a meta package with the same name: <package name="msys-base" class="virtual"> <affiliate group="MSYS Base System" /> <description lang="en" title="MSYS Base System"> <paragraph>This meta-package...</paragraph> </description> <release tarname="msys-base-1.0.14-msys-bin.meta" /> <requires ge="msys-bash-*-msys-*-bin.tar" /> <requires ge="msys-coreutils-*-msys-*-bin.tar" /> </package> They then would NOT be able to move that meta-package specification into a separate file, because its obvious name, msys-base.xml, would clash with the <package-list catalogue="msys-base"> file. But nobody would EVER do something like that... >> wiki? > > Not keen on that idea. Nor I, but I thought I should mention it. It IS intended as a collaborative authoring tool, after all. >> ongoing revisions simply posted to the mailing list? </me runs...> > > Then someone needs to take ownership, and ensure that patches are > both regularly, and routinely applied. I'd rather trust individual > package maintainers to apply their own updates in CVS Yep, that makes the most sense to me. > then push the > result to FRS. Manually? Is this were we do that "upload a file with the same name" thing (e.g. "coreutils.xml")? -- Chuck |
From: Keith M. <kei...@us...> - 2010-04-26 19:44:19
|
On Wednesday 21 April 2010 01:13:07 Charles Wilson wrote: > >> [where to maintain one-big-xml, or many-smaller-xml] > >> > >> CVS? > > > > That would be my choice; however, we do need to agree how the > > modules layout should be structured. > > Seems like a single directory, with individual files for each set > of packages derived from the same source lineage, would be > workable. I was thinking of how we want to organise the CVS repository. I don't think that these package-specific manifests belong in the source tree for mingw-get, since they relate more to *other* packages -- those which they describe. Maybe a flat directory at cvsroot level, to store them all, and use the CVSROOT/modules database to organise them into package specific subsets? > Merging closely related groups together wouldn't hurt, > either. I might combine all of the automakeX.Y collections (and > the am-wrapper) into one big mingw-automake.xml; ditto > mingw-autoconfX.Y and the ac-wrapper. As the maintainer of those package sets, that will be your call; such a combination would make sense to me. Naturally, I will be on hand to offer advice, if you require it. -- Regards, Keith. |
From: Charles W. <cwi...@us...> - 2010-04-26 20:46:01
|
On Mon, 26 Apr 2010 13:05 +0000, "Keith Marshall" wrote: > On Wednesday 21 April 2010 01:13:07 Charles Wilson wrote: > > Seems like a single directory, with individual files for each set > > of packages derived from the same source lineage, would be > > workable. > > I was thinking of how we want to organise the CVS repository. I > don't think that these package-specific manifests belong in the > source tree for mingw-get, since they relate more to *other* > packages -- those which they describe. Agreed. > Maybe a flat directory at > cvsroot level, to store them all, and use the CVSROOT/modules > database to organise them into package specific subsets? I think a flat directory at the cvsroot level is probably right; but I don't think we need to be any more granular with regards to what you check out from CVS than, say, the subsystem level. That is: mingw-get-xml/ msys-zlib.xml msys-blah.xml ... mingw-autoconf.xml mingw-automake.xml ... where *perhaps* cvs modules might list all of the msys-* as part of the "mingw-get-xml-msys" module, and all of the mingw-* ones as part of the mingw-get-xml-mingw module. But I think even that is overkill. Just check 'em all out as a common module; modify locally only the ones you care about. I doubt anyone would use their local CVS checkout as "the" xml fragment location for actually running mingw-get. Wouldn't you instead use a *downloaded* cache directory for that, rather than a *checked out* one? (Because the *downloaded* directory would contain the .lzma compressed versions that mingw-get expects, while the *checked out* one would -- probably -- contain only the uncompressed .xml sources. > > Merging closely related groups together wouldn't hurt, > > either. I might combine all of the automakeX.Y collections (and > > the am-wrapper) into one big mingw-automake.xml; ditto > > mingw-autoconfX.Y and the ac-wrapper. > > As the maintainer of those package sets, that will be your call; > such a combination would make sense to me. Naturally, I will be on > hand to offer advice, if you require it. Thanks, -- Chuck |
From: Keith M. <kei...@us...> - 2010-04-21 23:07:27
|
On Wednesday 21 April 2010 01:13:07 Charles Wilson wrote: > > Probably not as fine grained as you are thinking of, but > > mingw-get CVS is now able to construct the distribution manifest > > from multiple secondary files, with one top-level master index > > file, interpreted recursively to incorporate the secondaries. > > But, AFAICT, only at specific "customization points": e.g. entire > <package-list> elements. That is: > > <package-list catalogue="msys-base" /> > > "#includes" msys-base.xml. But, in msys-base.xml can you do: > > <package catalogue="coreutils"> > > which "#includes" coreutils.xml? No, but you can have msys-base.xml call out <package-list catalogue="coreutils" /> and then have coreutils.xml define <software-distribution ...> <package-collection subsystem="msys"> <download-host uri="..." /> <package name="msys-coreutils"> ... </package> </package-collection> </software-distribution> with all the information pertinent to msys-coreutils, and no other package, specified therein; when searching for the msys-coreutils package, mingw-get will search through its list of package-collection elements, with appropriate subsystem attribute, until it finds one which contains the requisite package entry, returning the first hit. > (FYI: Thunderbird appears to be US-centric. It's complaining about > 'catalogue' being misspelled, but is perfectly happy with > 'catalog') Yeah. Firefox does likewise, even when configured with its en_GB dictionary :( However, being a Brit, "catalog" just looks wrong to me, (and KMail, with its en_GB dictionary, flags it as misspelled). Can't please all of the people all of the time. British developers will tend to favour GB English spellings, (which AFAIK, are actually preferred universally, in the English speaking world outside of North America -- maybe even in Canada; I don't know for sure). From a technical stand-point, of course, mingw-get doesn't care how "catalogue" is spelt, or even that the word is "catalogue" at all; the sole requirement is that the attribute is consistently identified by the same unique arbitrary sequence of characters, which can be represented in UTF-8 encoding, and legal as an XML identifier, (and I'd also like to mandate that they be restricted to the ASCII printable subset of UTF-8, if XML doesn't already require this -- I'm not certain). I chose "catalogue" in this context, because it made sense to me, but I guess <package-list name="coreutils" /> would be just as acceptable, and would avoid any potential for confusion between GB and US spellings. If I make that change, it will break alpha-1's profile.xml, but that perhaps isn't too serious if I do it now; it could be more painful to delay the change. What do you think? Should I change it, for preference? > But if you specify two different .xml files, one for each release, > you'd have: > > [two disparate package definitions] > > Merging those automatically is...not fun. Exactly. As you say: we don't want to go there! The XML root element is a "software-distribution", and within that we can have: <software-distribution ...> <!-- an arbitrary number of ... --> <package-list name="foo" /><!-- each with a unique name --> <package-group-hierarchy> <!-- just one of these, but I plan to accommodate merging of fragments from individual package list files, to construct it --> ... </package-group-hierarchy> <!-- an arbitrary number of ... --> <package-collection subsystem="foosys"> <!-- required once, in each package-collection --> <download-host uri="template" /> <!-- one or more, but only one globally, for each named package --> <package name="bar"> ... </package> ... </package-collection> </software-distribution> > Right. So, I'm thinking IF we were to go as fine-grained as "one > xml file per unique source tarball, without regard to version", We can go as fine grained as one XML file per *package*; (strictly per package-collection, but that could contain as few as one actual package definition). > then you'd need to edit/commit a new (e.g.) gettext.xml each time > you upload a new set of > > gettext-0.17-2-msys-1.0.13-bin.tar.lzma > gettext-0.17-2-msys-1.0.13-dev.tar.lzma > gettext-0.17-2-msys-1.0.13-doc.tar.lzma > gettext-0.17-2-msys-1.0.13-ext.tar.lzma > gettext-0.17-2-msys-1.0.13-lic.tar.lzma > gettext-0.17-2-msys-1.0.13-src.tar.lzma > libasprintf-0.17-2-msys-dll-0.tar.lzma > libgettextpo-0.17-2-msys-dll-0.tar.lzma > libintl-0.17-2-msys-dll-8.tar.lzma Here you would edit the *existing* gettext.xml, (which would likely define four packages within its package-collection)... > adding new entries for those 9 tarballs (actually, 8, because -src > doesn't get an entry -- or does it [*]), without removing any > existing elements from the xml. Yes, you would add new <release> tag entries in each of the embedded <component> elements, within their appropriate package definitions; for the time being, you would leave the existing <release> entries in place, and unchanged. > [*] You mentioned a possible element type, sibling to <component>: > <source tarname="gettext-%-msys-%-src.tar.lzma"> > but I'm not sure if that's implemented yet, whether '&' or '%' > was used for the "matching version number" wildcard, or whether > you can use it twice to match both package version/release and > subsystem- version. I haven't (fully) implemented source retrieval logic yet, but my idea is that, as far as practical, the source tarball name can be deduced by simply substituting "src" for the component class within the canonical tarname of any related release definition, and therefore no explicit source spec will be needed. Obviously, there will be cases, such as your example here, where this will not suffice; for such cases, a <source> tag, either within the <release> element itself, (with an explicit source tarname attribute), or as either a <release> sibling or a <component> sibling, (providing a version matched template for source tarname generation), will provide the necessary override. For the version matching wildcard, I first considered "&", but in XML that needs to be represented as "&", which just seems needlessly clumsy, so I now think "%" will be a better choice; and yes, it will be allowed to match either or both of the package version and the subsystem version fields, within a single template. I think we are going to need this capability quite soon -- the "%" wildcard for version matching may also be useful to facilitate more concise dependency specifications -- so perhaps I should turn my attention to it, after I get alpha-2 out? (I also now have the installation records segregated into separate per-package manifest files, but I still need to address the uninstall facility). > > Keeping the manifest in small recursively included > > sub-files may help to make it more manageable. > > Yep -- but it's going to be a balance between maintenance and how > many separate downloads of tiny little xml files mingw-get has to > make, to construct its internal distribution manifest (since we > can't, and shouldn't, offload that process to a sourceforge-hosted > cronjob) Sure. That will be mitigated, to some extent, by downloading only when the user does an explicit "mingw-get update", (or if needed for a first time download); at other times, locally cached copies will be used. Additionally, if I'm interpreting MSDN correctly, wininet will hold its connection on the server, while accessing multiple consecutive URLs addressing a common host. > > Do note that, while SF assure us they will keep files forever, > > it does seem to be possible to *overwrite* an existing file, > > with new content; delete the existing file record from the FRS > > database, then upload a fresh copy with identically the same > > name. > > That's true. (But bloody awkward: ok, I'll upload this 0-byte file > with the same name, and that'll be our little code for "ignore me, > I've been deleted"...) I'm not suggesting we do that, (upload 0-byte files). For the XML distribution manifests, we *want* the file name to remain unchanged, even when the content is updated. At any time, the hosted copy will be the current version; when we do an update, we overwrite the out of date copy with the new version, which then becomes current. > if somebody names their demonstration msys package list > catalog 'msys-base': > ... > And then, wants to specify a meta package with the same name: > ... > They then would NOT be able to move that meta-package > specification into a separate file, because its obvious name, > msys-base.xml, would clash with the <package-list > catalogue="msys-base"> file. > > But nobody would EVER do something like that... Well, hopefully *we* wouldn't, but somebody else just might, (which of course is what you are hinting); but then, *they* would not be hosting their package in *our* repository -- mingw-get will support alternative repositories. I haven't implemented it yet, but I am thinking of hashing the locally cached distribution manifest file names, based on full download URLs, to resolve such potential conflicts. > > I'd rather trust individual > > package maintainers to apply their own updates in CVS > > Yep, that makes the most sense to me. > > > then push the > > result to FRS. > > Manually? Is this were we do that "upload a file with the same > name" thing (e.g. "coreutils.xml")? You'd actually compress it, and upload it as coreutils.xml.lzma, but yes, that is the idea; it may seem a little awkward, but really not any different in effect, from regenerating in place, if SF were to permit that, (which of course, they don't). -- Regards, Keith. |
From: Charles W. <cwi...@us...> - 2010-04-22 02:42:38
Attachments:
msys-zlib.xml
|
On 4/21/2010 7:05 PM, Keith Marshall wrote: > On Wednesday 21 April 2010 01:13:07 Charles Wilson wrote: >> But, in msys-base.xml can you do: >> <package catalogue="coreutils"> >> which "#includes" coreutils.xml? > > No, but you can have msys-base.xml call out > > <package-list catalogue="coreutils" /> > > and then have coreutils.xml define > > <software-distribution ...> > <package-collection subsystem="msys"> > <download-host uri="..." /> > <package name="msys-coreutils"> > ... > </package> > </package-collection> > </software-distribution> > > with all the information pertinent to msys-coreutils, and no other > package, specified therein I see. That'll work; thanks. [stuff about us-vs-gb spellings] > I chose "catalogue" in this > context, because it made sense to me, but I guess > > <package-list name="coreutils" /> > > would be just as acceptable, and would avoid any potential for > confusion between GB and US spellings. If I make that change, it > will break alpha-1's profile.xml, but that perhaps isn't too serious > if I do it now; it could be more painful to delay the change. What > do you think? Should I change it, for preference? It's completely up to you. You're the primary developer...you get to pick the dialect spoken by the tool. I think most USERS won't care; they might not even ever LOOK at the xml files (how many people actually look at cygwin's setup.ini files?) This is a developer thing. I think most of us are comfortable using either spelling; I might have to draw the line if I saw: <?xml version="1.0" encoding="Big5" standalone="yes"?> 你想推荐些什么菜吗 </xml> I don't know if tinyxml supports variant encodings at all (does it even, REALLY, support UTF-8?) so you might be better off asserting that the xml files all specify: encoding="iso-8859-1" or even encoding="ascii" (but then you loose the ability to use european accented characters in description text) <time passes> Hmm...according to http://www.grinninglizard.com/tinyxmldoc/index.html, "TinyXML fully supports the UTF-8 encoding, and the first 64k character entities." I dunno how they do that, without linking against some i18n library... Actually, looking at the code, it appears that TinyXML allows only low-ascii printable, OR multibyte characters, in entity names. Thus, characters from 128-255 (and 0..31, and 127) are excluded from entity (or attribute) names, so setting entity="iso-8859-1" is a bad idea. Also, there's this (in tinyxml.h): enum TiXmlEncoding { TIXML_ENCODING_UNKNOWN, TIXML_ENCODING_UTF8, TIXML_ENCODING_LEGACY }; So...encoding="ascii" is ALSO a bad idea. Sigh. It looks like you're right, Keith -- we have only two choices: don't specify the encoding at all, and hope it works, or specify UTF-8 and politely request that folks avoid all but low-ascii printable characters in entity NAMES. >> But if you specify two different .xml files, one for each release, >> you'd have: >> >> [two disparate package definitions] >> >> Merging those automatically is...not fun. > > Exactly. As you say: we don't want to go there! The XML root > element is a "software-distribution", and within that we can have: > > <software-distribution ...> ... > >> Right. So, I'm thinking IF we were to go as fine-grained as "one >> xml file per unique source tarball, without regard to version", > > We can go as fine grained as one XML file per *package*; (strictly > per package-collection, but that could contain as few as one actual > package definition). toMAYto, toMAHto...I think we're both actually talking about the same thing. I've attached a tentative version of the xml for the "zlib package": >> then you'd need to edit/commit a new (e.g.) gettext.xml each time >> you upload a new set of >> >> gettext-0.17-2-msys-1.0.13-bin.tar.lzma >> gettext-0.17-2-msys-1.0.13-dev.tar.lzma >> gettext-0.17-2-msys-1.0.13-doc.tar.lzma >> gettext-0.17-2-msys-1.0.13-ext.tar.lzma >> gettext-0.17-2-msys-1.0.13-lic.tar.lzma >> gettext-0.17-2-msys-1.0.13-src.tar.lzma >> libasprintf-0.17-2-msys-dll-0.tar.lzma >> libgettextpo-0.17-2-msys-dll-0.tar.lzma >> libintl-0.17-2-msys-dll-8.tar.lzma > > Here you would edit the *existing* gettext.xml, (which would likely > define four packages within its package-collection)... Why four, and not just one, with eight components? >> adding new entries for those 9 tarballs (actually, 8, because -src >> doesn't get an entry -- or does it [*]), without removing any >> existing elements from the xml. > > Yes, you would add new <release> tag entries in each of the embedded > <component> elements, within their appropriate package definitions; > for the time being, you would leave the existing <release> entries > in place, and unchanged. OK. >> [*] You mentioned a possible element type, sibling to <component>: >> <source tarname="gettext-%-msys-%-src.tar.lzma"> >> but I'm not sure if that's implemented yet, whether '&' or '%' >> was used for the "matching version number" wildcard, or whether >> you can use it twice to match both package version/release and >> subsystem- version. > > I haven't (fully) implemented source retrieval logic yet, but my idea > is that, as far as practical, the source tarball name can be deduced > by simply substituting "src" for the component class within the > canonical tarname of any related release definition, and therefore > no explicit source spec will be needed. Obviously, there will be > cases, such as your example here, where this will not suffice; for > such cases, a <source> tag, either within the <release> element > itself, (with an explicit source tarname attribute), or as either a > <release> sibling or a <component> sibling, (providing a version > matched template for source tarname generation), will provide the > necessary override. OK. But, whether the back-end logic is complete, is the front-end parsing ready (in mingw-get-alpha-2)? That is, can I go ahead and include, where necessary, a <source> tag -- that will simply be ignored by mingw-get for now? > For the version matching wildcard, I first considered "&", but in XML > that needs to be represented as "&", which just seems needlessly > clumsy, so I now think "%" will be a better choice; and yes, it will > be allowed to match either or both of the package version and the > subsystem version fields, within a single template. Ack. > I think we are going to need this capability quite soon -- the "%" > wildcard for version matching may also be useful to facilitate more > concise dependency specifications -- so perhaps I should turn my > attention to it, after I get alpha-2 out? Prioritization is your call. I hope that alpha-2, with enough xml fragments on sourceforge, AND me finally finishing the rebuild project, will enable wider testing. I expect bug-hunting and other user-generated issues will rearrange whatever prioritization we try to establish up front, in very short order. But, yeah: dependency matching is an important use of the version wildcard. For instance: <component class="dev"> <release tarname="zlib-1.2.3-1-msys-1.0.11-dev.tar.gz" /> <release tarname="zlib-1.2.3-2-msys-1.0.13-dev.tar.lzma" /> <requires ge="zlib-%-msys-%-dll.tar" /> <requires ge="gcc-*-msys-*-bin" /> <<<<---- (*) <requires ge="w32api-*-msys-*-dev.tar" /> </component> (*) should -dev packages list gcc-msys as a requirement? The header files (zlib.h) #include other headers that are supplied only by gcc, but...that would pull in a lot of extra stuff if somebody just wanted to, say, install zlib-dev to simply look at the headers. Ditto w32api-msys... > (I also now have the > installation records segregated into separate per-package manifest > files, but I still need to address the uninstall facility). Nifty keen. > Sure. That will be mitigated, to some extent, by downloading only > when the user does an explicit "mingw-get update", (or if needed for > a first time download); at other times, locally cached copies will > be used. Oh, I didn't realize. Yeah, that makes things more reasonable. > Additionally, if I'm interpreting MSDN correctly, wininet > will hold its connection on the server, while accessing multiple > consecutive URLs addressing a common host. I did not know that. > I'm not suggesting we do that, (upload 0-byte files). For the XML > distribution manifests, we *want* the file name to remain unchanged, > even when the content is updated. At any time, the hosted copy will > be the current version; when we do an update, we overwrite the out > of date copy with the new version, which then becomes current. Sure. >> if somebody names their demonstration msys package list >> catalog 'msys-base': >> ... >> And then, wants to specify a meta package with the same name: >> ... >> They then would NOT be able to move that meta-package >> specification into a separate file, because its obvious name, >> msys-base.xml, would clash with the <package-list >> catalogue="msys-base"> file. >> >> But nobody would EVER do something like that... > > Well, hopefully *we* wouldn't Err...those snippets were taken from your example. So, in /your/ example, we can't move the 'msys-base' meta-package specification into a file separate from the 'msys-base' package-list file. That was kind of my point: I think that your top-level package-list.xml should instead say something like: <?xml version="1.0" encoding="UTF-8" standalone="yes" ?> <software-distribution ...> <package-list catalogue="mingw32-master" /> <package-list catalogue="msys-master" /> <<--- anything but "-base" </software-distribution> Then, the file you called 'msys-base.xml' would instead be called 'msys-master.xml', and it would be able to specify: <!-- meta package approximating old monolithic msys installer --> <package-list name="msys-base" /> Finally, then we'd have a /new/ msys-base.xml file, that looks like: <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <software-distribution ...> <package-collection subsystem="msys"> <download-host uri="..." /> <package name="msys-base" class="virtual"> <affiliate group="MSYS Base System" /> <description lang="en" title="MSYS Base System"> <paragraph>This meta-package will install the appropriate collection of related packages, to (approximately) replicate the features of a basic MSYS installation, as delivered by the old family of free-standing integrated MSYS installers.</paragraph> </description> <release tarname="msys-base-1.0.14-msys-bin.meta" /> <requires ge="msys-bash-*-msys-*-bin.tar" /> <requires ge="msys-coreutils-*-msys-*-bin.tar" /> </package> </package-collection> </software-distribution> > but somebody else just might, (which > of course is what you are hinting); but then, *they* would not be > hosting their package in *our* repository -- mingw-get will support > alternative repositories. I haven't implemented it yet, but I am > thinking of hashing the locally cached distribution manifest file > names, based on full download URLs, to resolve such potential > conflicts. Hmmm...that sounds like a job for alpha-4 or -5...or version 1.1. >> Manually? Is this were we do that "upload a file with the same >> name" thing (e.g. "coreutils.xml")? > > You'd actually compress it, and upload it as coreutils.xml.lzma, but > yes, that is the idea; it may seem a little awkward, but really not > any different in effect, from regenerating in place, if SF were to > permit that, (which of course, they don't). OK. Once we get the kinks worked out of the system, we probably should document "the steps to upload a new release of a package" on the wiki. I know I'm guaranteed to mess it up at least once if I don't have a checklist. Exit question 1: can msys-bash-bin list msys-base as one of its requirements, even though msys-base lists msys-bash-bin as one of its own requires? E.g. circular deps? Exit question 2: should ALL packages reference their own -lic package (if they have one) as an explicit requirement? -- Chuck |
From: Keith M. <kei...@us...> - 2010-04-22 18:27:41
|
On Thursday 22 April 2010 03:41:47 Charles Wilson wrote: > > I chose "catalogue" in this > > context, because it made sense to me, but I guess > > > > <package-list name="coreutils" /> > > > > would be just as acceptable, and would avoid any potential for > > confusion between GB and US spellings. If I make that change, > > it will break alpha-1's profile.xml, but that perhaps isn't too > > serious if I do it now; it could be more painful to delay the > > change. What do you think? Should I change it, for preference? > > It's completely up to you. You're the primary developer...you get > to pick the dialect spoken by the tool. I think most USERS won't > care; they might not even ever LOOK at the xml files (how many > people actually look at cygwin's setup.ini files?) Very few, I should think; I certainly never felt a need to do so, back in the days when I did, occasionally, use Cygwin. > This is a developer thing. I think most of us are comfortable > using either spelling; Well obviously, easiest and least painful is to leave it just as it is; unless anyone else objects very strongly, (and very quickly), let's agree that it will remain as the GB spelling of "catalogue". > I might have to draw the line if I saw: > > <?xml version="1.0" encoding="Big5" standalone="yes"?> > 你想推荐些什么菜吗 > </xml> Which of course, wouldn't be possible anyway, (with TinyXML), as you subsequently discovered... > Actually, looking at the code, it appears that TinyXML allows only > low-ascii printable, OR multibyte characters, in entity names. > Thus, characters from 128-255 (and 0..31, and 127) are excluded > from entity (or attribute) names, so setting entity="iso-8859-1" > is a bad idea. > > Also, there's this (in tinyxml.h): > enum TiXmlEncoding > { > TIXML_ENCODING_UNKNOWN, > TIXML_ENCODING_UTF8, > TIXML_ENCODING_LEGACY > }; > > So...encoding="ascii" is ALSO a bad idea. Sigh. > > It looks like you're right, Keith -- we have only two choices: > don't specify the encoding at all, and hope it works, or specify > UTF-8 and politely request that folks avoid all but low-ascii > printable characters in entity NAMES. That's what I thought; thanks for researching it. > > We can go as fine grained as one XML file per *package*; > > (strictly per package-collection, but that could contain as few > > as one actual package definition). > > toMAYto, toMAHto...I think we're both actually talking about the > same thing. Seems that way to me too. > I've attached a tentative version of the xml for the > "zlib package": Thanks! At a cursory glance, it looks reasonably close. I'll fold it into my development sandbox, play with it later, and get back to you. Just a few initial comments:-- * I've been writing issue attributes as YYYYMMDDNN, where NN is just a serialisation count. "mingw-get update" will simply ignore any downloaded package-list, if its issue attribute doesn't increment over any corresponding locally cached version. The comparison is performed lexically, so your use of YYYYMMDDHHMM will work just as well, but are we really likely to have more than 99 updates to any one package-list in any single day? It's not a big deal, but we should adopt one convention or the other, and then stick with it. * I don't think the -dev component needs to specify *any* "requires" dependencies; certainly not msys-gcc or msys-w32api. Does it need the -dll component for -dev deployment? I suspect not. * There are five supported selectors for "requires" specifications; ("lt", "le", "eq", "ge" and "gt"). If you think the -dll really is a prerequisite of the -dev component, then <requires eq="msys-zlib-%-msys-%-dll.tar" /> is the correct representation for an exactly equal version match, (when I get it working; it will error out right now). * When you write the "requires" specification, the match string has the form of a canonical tarname, but the package-name part *must* match the "name" attribute of the containing "package" definition, irrespective of the actual tarname of the target download. You've named this "package" as "msys-zlib", (which I think is a practice we should formally adopt -- prefix the subsystem name to the name of the package, in the "package" definition, to avoid any potential ambiguity [*]), so the form I've shown above is correct. (BTW, the ".tar" at the end is required, to satisfy pkginfo's parsing rules, but it doesn't participate in the actual dependency resolution matching operation; it could just as well be specified as ".foo", and it would still work). [*] We should probably do likewise, in the actual XML file name, so the package-list spec to include this zlib package would state <package-list catalogue="msys-zlib" /> resulting in "msys-zlib.xml", (which would be hosted for download as "msys-zlib.xml.lzma"). > > Here you would edit the *existing* gettext.xml, (which would > > likely define four packages within its package-collection)... > > Why four, and not just one, with eight components? I'm thinking that users may wish to do: $ mingw-get install msys-libintl-dll (for example), without necessarily being concerned that it is really a subsidiary of msys-gettext; to make that work, you either define a specific package with name="msys-libintl", to furnish the "dll" component, or you define one "msys-gettext" package, with additional "alias=..." attributes, (actually one attribute with value as a space separated list), to identify the subsidiaries. In addition to this, you also face a problem with identification of individual dll component packages with disparate names, if you embed them all within a single package -- lookups are based on the name and alias attributes of the "package" element, and the class attribute of the contained "component" element, so mingw-get will have a harder job determining which particular "component" is intended, when faced with this disparity, (both in the case of user specified lookups, and for dependency resolution). Sure, that wouldn't be an insoluble problem, but it ain't the way it works at the moment. > OK. But, whether the back-end logic is complete, is the front-end > parsing ready (in mingw-get-alpha-2)? Strictly no, but... > That is, can I go ahead and > include, where necessary, a <source> tag -- that will simply be > ignored by mingw-get for now? ...yes, you can. There is no formal XML schema (or DOM definition) to restrict which elements are allowed, and in which contexts, (and in any case, TinyXML doesn't support validation against a DOM); the only strict requirement is that the XML /structure/ is valid. You can add any arbitrary element, or attribute you wish, in any [**] context, and mingw-get will silently ignore it, if it isn't specifically looking for it. [**] There are a few exceptions, of course: I have explicitly coded mingw-get to check that the root element in each and every package list is a "software-distribution", and in some contexts I do check that certain required elements are present, but in general, if you add anything extra, it is silently ignored. > Prioritization is your call. I hope that alpha-2, with enough xml > fragments on sourceforge, AND me finally finishing the rebuild > project, will enable wider testing. I expect bug-hunting and other > user-generated issues will rearrange whatever prioritization we > try to establish up front, in very short order. For sure. > (*) should -dev packages list gcc-msys as a requirement? As a general rule, I think not. > The > header files (zlib.h) #include other headers that are supplied > only by gcc, but...that would pull in a lot of extra stuff if > somebody just wanted to, say, install zlib-dev to simply look at > the headers. Ditto w32api-msys... Certainly, there may be an implied requirement for headers from the GCC package, and from the w32api package, but these don't become explicit until actually trying to *compile* the package, and the installation of the compiler suite is an obvious prerequisite to doing that, irrespective of any implied dependencies in an auxiliary library package. Thus, I think that msys-gcc-bin should specify its dependency on msys-w32api-dev, but not vice-versa, and there is no need for msys-zlib to specify any dependency on either. > >> But nobody would EVER do something like that... > > > > Well, hopefully *we* wouldn't > > Err...those snippets were taken from your example. So, in /your/ > example, we can't move the 'msys-base' meta-package specification > into a file separate from the 'msys-base' package-list file. Oops! I see your point, now. One possibility would be to have the msys-base.xml file specify just the msys-base package specification, together with a bunch of package-list elements to pull in the rest. Alternatively... > That was kind of my point: I think that your top-level > package-list.xml should instead say something like: > > <?xml version="1.0" encoding="UTF-8" standalone="yes" ?> > <software-distribution ...> > <package-list catalogue="mingw32-master" /> > <package-list catalogue="msys-master" /> <<--- anything but > "-base" </software-distribution> > > Then, the file you called 'msys-base.xml' would instead be called > 'msys-master.xml', ...this would be equally acceptable -- maybe better. Ultimately, I think it should be you and Cesar who decide on your preferred layout, but clearly, I shouldn't abdicate all responsibility in helping you to formulate your choice. > > I haven't implemented it yet, but I am > > thinking of hashing the locally cached distribution manifest > > file names, based on full download URLs, to resolve such > > potential conflicts. > > Hmmm...that sounds like a job for alpha-4 or -5...or version 1.1. Well, most of the code to handle it is already written, (and used to segregate the installation manifests), but it will need some book keeping, to track the name changes as the downloads are moved into the local cache directory. However, you are right; it likely isn't especially urgent. > OK. Once we get the kinks worked out of the system, we probably > should document "the steps to upload a new release of a package" > on the wiki. Agreed. > I know I'm guaranteed to mess it up at least once if > I don't have a checklist. Surely not? :) > Exit question 1: can msys-bash-bin list msys-base as one of its > requirements, even though msys-base lists msys-bash-bin as one of > its own requires? E.g. circular deps? I *think* I got the logic right, to break circular dependency specs, but why would you want to do this? Surely, there is only one actual prerequisite for msys-bash-bin, and that is msysCORE-bin. Don't specify a dependency unless it is absolutely necessary, otherwise you preclude the possibility of installing an even more minimal subset of MSYS than msys-base specifies; (it is conceivable, no matter how unlikely, that I might want just a shell, without incurring the overhead of other packages typically included within msys-base). > Exit question 2: should ALL packages reference their own -lic > package (if they have one) as an explicit requirement? Until you asked, I hadn't given it a great deal of thought. On balance, I'm inclined to say "yes, I think so", which then begs the question: should mingw-get implicitly add a -lic dependency, for every package component which has a -lic sibling? Or maybe, we should just add an explicit: <requires eq="pkgname-%-subsys-%-lic.tar" /> as a single sibling prerequisite of the *component* elements within the package definition? (Subject, of course, to me getting the "%" wildcard matching working fairly sharpish). -- Regards, Keith. |
From: Charles W. <cwi...@us...> - 2010-04-22 23:43:10
|
On Thu, 22 Apr 2010 15:42 +0000, "Keith Marshall" wrote: > On Thursday 22 April 2010 03:41:47 Charles Wilson wrote: > > This is a developer thing. I think most of us are comfortable > > using either spelling; > > Well obviously, easiest and least painful is to leave it just as it > is; unless anyone else objects very strongly, (and very quickly), > let's agree that it will remain as the GB spelling of "catalogue". Fine by me. > > I might have to draw the line if I saw: > Which of course, wouldn't be possible anyway, (with TinyXML), as you > subsequently discovered... > > It looks like you're right, Keith -- we have only two choices: > > don't specify the encoding at all, and hope it works, or specify > > UTF-8 and politely request that folks avoid all but low-ascii > > printable characters in entity NAMES. > > That's what I thought; thanks for researching it. Sure. > > I've attached a tentative version of the xml for the > > "zlib package": > > Thanks! At a cursory glance, it looks reasonably close. I'll fold > it into my development sandbox, play with it later, and get back to > you. Just a few initial comments:-- OK. I'll hold off on trying to construct any other packages -- except maybe the gettext collection, since it has already come up and presents additional questions -- until we get this one nailed down, vis: > * I've been writing issue attributes as YYYYMMDDNN, where NN is just > a serialisation count. "mingw-get update" will simply ignore any > downloaded package-list, if its issue attribute doesn't increment > over any corresponding locally cached version. The comparison is > performed lexically, so your use of YYYYMMDDHHMM will work just as > well, but are we really likely to have more than 99 updates to any > one package-list in any single day? It's not a big deal, but we > should adopt one convention or the other, and then stick with it. I'll probably automate the 'commit' step -- obviously, adding elements to reflect a new release is a manual process. But once that's done, a simple a script that (a) updates the attribute, (b) checks in to CVS, (c) compresses using lzma, and (d) uploads the .lzma to <where-ever> on sourceforge -- would be my typical approach. And... TIMESTAMP=`date +%Y%m%d%H%M` cat foo | sed -e "s/@@TIMESTAMP@@/$TIMESTAMP/" > newfoo && mv newfoo foo is a lot more straightforward -- and robust -- than OLDTIMESTAMP=$(cat foo | sed -n -e '/pattern/s/^.*\(the good stuff\).*$/\1/p') OLDSEQ=${OLDTIMESTAMP:8} NEWSEQ=$(( $OLDSEQ + 1 )) NEWTIMESTAMP=`date +Y%m%d`${NEWSEQ} since I don't have an xml parser coded in bash handy. > * I don't think the -dev component needs to specify *any* "requires" > dependencies; Disagree...see below > certainly not msys-gcc or msys-w32api. I could go along with *that* part... > Does it need > the -dll component for -dev deployment? I suspect not. But this assertion contradicts the behavior of all unix systems. In every system I can recall, the -devel package always depends on the lib package (this makes sense, because for *native* builds, you wouldn't be able to run 'make check' on the package the depends on the -devel package, unless you also installed the matching lib package for that dependency). Also, devel packages all tend to have dependencies on the other devel packages used to build them -- for instance, libpng-devel depends on libz-devel (ne' zlib-devel, depending on your distro). The question is, how far back down the chain to go: I can see the argument for cutting the "dependency chain" just ahead of the actual compiler stuff, since on a cross-build system you wouldn't need *mingw/msys's* version of the compiler anyway. But, you WOULD need msysCORE-dev (and if you were using a port of mingw-get to populate your sysroot, you probably would like for it to handle all of the -devel deps, EXCEPT for those directly related to the compile toolchain). (*) side note, for later discussion: "sysroot" installations would need a post-install step to "fixup" the ${root}/lib/*.la files, and any ${root}/bin/{pkg}-config scripts, ditto ${root}/lib/pkg-config/*.pc, ... I dunno if this sort of thing is even worth thinking about. > * There are five supported selectors for "requires" specifications; > ("lt", "le", "eq", "ge" and "gt"). If you think the -dll really is > a prerequisite of the -dev component, then > > <requires eq="msys-zlib-%-msys-%-dll.tar" /> > > is the correct representation for an exactly equal version match, > (when I get it working; it will error out right now). OK. > * When you write the "requires" specification, the match string has > the form of a canonical tarname, but the package-name part *must* > match the "name" attribute of the containing "package" definition, > irrespective of the actual tarname of the target download. Err...I'm confused. > You've > named this "package" as "msys-zlib", (which I think is a practice > we should formally adopt -- prefix the subsystem name to the name > of the package, in the "package" definition, to avoid any potential > ambiguity [*]), ... > [*] We should probably do likewise, in the actual XML file name, so > the package-list spec to include this zlib package would state > > <package-list catalogue="msys-zlib" /> > > resulting in "msys-zlib.xml", (which would be hosted for download as > "msys-zlib.xml.lzma"). OK, that's all good. > so the form I've shown above is correct. (BTW, the > ".tar" at the end is required, to satisfy pkginfo's parsing rules, > but it doesn't participate in the actual dependency resolution > matching operation; it could just as well be specified as ".foo", > and it would still work). But my confusion is this: if the "package name" is "msys-zlib", does that mean that **all** the <release> tags need to specify themselves as: <release tarname="msys-zlib-1.2.3-2-msys-1.0.13-dev.tar.lzma"> <download tarname="zlib-1.2.3-2-msys-1.0.13-dev.tar.lzma" /> </release> > > > Here you would edit the *existing* gettext.xml, (which would > > > likely define four packages within its package-collection)... > > > > Why four, and not just one, with eight components? > > I'm thinking that users may wish to do: > > $ mingw-get install msys-libintl-dll > > (for example), without necessarily being concerned that it is really > a subsidiary of msys-gettext; to make that work, you either define a > specific package with name="msys-libintl", to furnish the "dll" > component, or you define one "msys-gettext" package, with additional > "alias=..." attributes, (actually one attribute with value as a space > separated list), to identify the subsidiaries. > > In addition to this, you also face a problem with identification of > individual dll component packages with disparate names, if you embed > them all within a single package -- lookups are based on the name > and alias attributes of the "package" element, and the class > attribute of the contained "component" element, so mingw-get will > have a harder job determining which particular "component" is > intended, when faced with this disparity, (both in the case of user > specified lookups, and for dependency resolution). Sure, that > wouldn't be an insoluble problem, but it ain't the way it works at > the moment. I see; that makes sense. Now, zlib isn't a good example here, but going back to gettext, I'd need to specify, for each of the four packages, a <source ...> element, as well as a "lic" component (see far below): otherwise, mingw-get won't "know" that the license tarball and source tarball for pkg "msys-libintl-dll" is provided by tarname="{msys}-gettext-%-msys-%-lic.tar" and tarname="{msys}-gettext-%-msys-%-src.tar", respectively. And no heuristic would ever be able to figure it out. > > OK. But, whether the back-end logic is complete, is the front-end > > parsing ready (in mingw-get-alpha-2)? > > Strictly no, but... > > > That is, can I go ahead and > > include, where necessary, a <source> tag -- that will simply be > > ignored by mingw-get for now? > > ...yes, you can. There is no formal XML schema (or DOM definition) > to restrict which elements are allowed, and in which contexts, (and > in any case, TinyXML doesn't support validation against a DOM); the > only strict requirement is that the XML /structure/ is valid. You > can add any arbitrary element, or attribute you wish, in any [**] > context, and mingw-get will silently ignore it, if it isn't > specifically looking for it. OK -- I can live with that. (I'm used to validating parsers with DOM or schema support...) > > (*) should -dev packages list gcc-msys as a requirement? > > As a general rule, I think not. OK. > > The > > header files (zlib.h) #include other headers that are supplied > > only by gcc, but...that would pull in a lot of extra stuff if > > somebody just wanted to, say, install zlib-dev to simply look at > > the headers. Ditto w32api-msys... > > Certainly, there may be an implied requirement for headers from the > GCC package, and from the w32api package, but these don't become > explicit until actually trying to *compile* the package, and the > installation of the compiler suite is an obvious prerequisite to > doing that, irrespective of any implied dependencies in an auxiliary > library package. Thus, I think that msys-gcc-bin should specify its > dependency on msys-w32api-dev, but not vice-versa, OK. > and there is no > need for msys-zlib to specify any dependency on either. In this case, I can go along with that (e.g. w32api-dev is pretty close to the "bottom" of the stack, that it could fall under my "leave out the compiler/core-system-related dev files" rule). But, I'd think that libintl-dev, for instance, should require libiconv-dev (even though building gettext itself requires lots more stuff, including expat-dev, termcap-dev, etc -- but the *library*, libintl, doesn't depend on those things). IOW, if project "foo" says "you need to install libintl-dev" to compile this package -- but foo doesn't directly depend on libiconv-dev, I want libiconv-dev to get installed automatically when I dutifully traipse off to install libintl-dev; if it doesn't, then I'll get a compile-time error that I was not expecting, since the libiconv headers (and libiconv import lib) are missing -- and I won't know why. But I do *not* need libexpat-dev just to compile something that requires libintl; libexpat itself is used only by the gettext apps, not the gettext libs. > Oops! I see your point, now. One possibility would be to have the > msys-base.xml file specify just the msys-base package specification, > together with a bunch of package-list elements to pull in the rest. > Alternatively... Whichever. > > Then, the file you called 'msys-base.xml' would instead be called > > 'msys-master.xml', > > ...this would be equally acceptable -- maybe better. Ultimately, I > think it should be you and Cesar who decide on your preferred layout, > but clearly, I shouldn't abdicate all responsibility in helping you > to formulate your choice. Well, the top-level layout is more a question of, what does/will mingw-get support. I'd be perfectly happy to just provide content in the form of appropriate msys-*.xml and mingw-*.xml fragments for specific packages, and let somebody with a better understanding of the mingw-get innards handle marshalling them together. Hint... Of course, all of that -- including said .xml fragments -- will be subject to commentary and revision driven by discussion on this list. > Well, most of the code to handle it is already written, (and used > to segregate the installation manifests), but it will need some book > keeping, to track the name changes as the downloads are moved into > the local cache directory. However, you are right; it likely isn't > especially urgent. I can't remember; were you planning to use a database backend of some kind, or just flat files, to manage the *installation manifest(s)* [as opposed to the *distribution manifest(s)]? > > Exit question 1: can msys-bash-bin list msys-base as one of its > > requirements, even though msys-base lists msys-bash-bin as one of > > its own requires? E.g. circular deps? > > I *think* I got the logic right, to break circular dependency specs, > but why would you want to do this? Surely, there is only one actual > prerequisite for msys-bash-bin, and that is msysCORE-bin. Don't > specify a dependency unless it is absolutely necessary, otherwise > you preclude the possibility of installing an even more minimal > subset of MSYS than msys-base specifies; (it is conceivable, no > matter how unlikely, that I might want just a shell, without > incurring the overhead of other packages typically included within > msys-base). Hmmm...good point. I was kind of thinking that "everything should require msys-base" (just like "everything requires cygwin" in the cygwin distro) -- to the point that we could add that implicitly within mingw-get's logic. But that got me thinking about the circular deps inherent in such a scheme. But...my assumptions were incorrect; nothing in *MinGW* requires msys at all. And, the whole point of msys is to be minimal; if somebody explicitly wants to experiment with a "crippled" sub-minimum installation, who are we to say no? > > Exit question 2: should ALL packages reference their own -lic > > package (if they have one) as an explicit requirement? > > Until you asked, I hadn't given it a great deal of thought. On > balance, I'm inclined to say "yes, I think so", which then begs the > question: should mingw-get implicitly add a -lic dependency, for > every package component which has a -lic sibling? Or maybe, we > should just add an explicit: > > <requires eq="pkgname-%-subsys-%-lic.tar" /> > > as a single sibling prerequisite of the *component* elements within > the package definition? (Subject, of course, to me getting the "%" > wildcard matching working fairly sharpish). Actually, I think a better approach would be something similar to the <source ...> element: an optional <lic ...> element if the -lic tarname can't be "guessed" -- e.g. the "lic" component for msysCORE-bin, msysCORE-dev, and msysCORE-dbg is actually msysCORE-bin. Granted, both msysCORE-dev and msysCORE-dbg *already* require the -bin package for other reasons, but the principle remains. Then, we could make a policy decision as to whether mingw-get would implicitly always install the matching lic component, or not. I think this to-install-lic-or-not decision should NOT be left to 'did the packager remember to add <requires eq="*-lic">'...maybe mingw-get could default to always installing lic packages, but with a top-level flag or cmdline argument --i-dont-care-about-licensing. -- Chuck |
From: Earnie <ea...@us...> - 2010-04-23 11:47:24
|
Charles Wilson wrote: > On Thu, 22 Apr 2010 15:42 +0000, "Keith Marshall" wrote: > > On Thursday 22 April 2010 03:41:47 Charles Wilson wrote: > >> This is a developer thing. I think most of us are comfortable > >> using either spelling; > > > > Well obviously, easiest and least painful is to leave it just as it > > is; unless anyone else objects very strongly, (and very quickly), > > let's agree that it will remain as the GB spelling of > > "catalogue". > > Fine by me. > My thought earlier in the thread was that we US Americans are a bit lazy but in reality it was most likely changed for technological reasons in that it is two characters shorter and back when computers weren't memory cheap and punch cards where being used those two characters mattered greatly. In reading it I prefer catalogue but in typing it, well I'm lazy. :) Either is fine by me. LOL, I just clicked the suggested spell corrector on Thunderbird and the very first suggestion is "uncatalogued". Go figure that out. Earnie |
From: Keith M. <kei...@us...> - 2010-04-23 21:51:55
|
On Friday 23 April 2010 00:43:03 Charles Wilson wrote: > On Thu, 22 Apr 2010 15:42 +0000, "Keith Marshall" wrote: > > On Thursday 22 April 2010 03:41:47 Charles Wilson wrote: > additional questions -- until we get this one nailed down, vis: > > > * I've been writing issue attributes as YYYYMMDDNN, where NN is > > just a serialisation count. "mingw-get update" will simply > > ignore any downloaded package-list, if its issue attribute > > doesn't increment over any corresponding locally cached version. > > The comparison is performed lexically, so your use of > > YYYYMMDDHHMM will work just as well, but are we really likely to > > have more than 99 updates to any one package-list in any single > > day? It's not a big deal, but we should adopt one convention or > > the other, and then stick with it. > > I'll probably automate the 'commit' step Makes sense, I guess, (but see my caveat below). > -- obviously, adding > elements to reflect a new release is a manual process. But once > that's done, a simple a script that (a) updates the attribute, (b) > checks in to CVS, (c) compresses using lzma, and (d) uploads the > .lzma to <where-ever> on sourceforge -- would be my typical > approach. And... > > TIMESTAMP=`date +%Y%m%d%H%M` > cat foo | sed -e "s/@@TIMESTAMP@@/$TIMESTAMP/" > newfoo && > mv newfoo foo > > is a lot more straightforward -- and robust But not without its own pitfalls, and Murphy's Law applies! When you base a serialisation counter on wall clock time, there is always going to be a window within which a regression might occur -- especially when developers in different time zones are involved. We can mitigate that by always using UTC, (date -u +%Y%m%d%H%M), but even that leaves a short (1 minute) window, (possibly extended by clock skew if we don't all have our systems tightly synchronised to the same atomic time reference), in which a regression may occur; if it can happen, (and it can), then Murphy tells us that it WILL, no matter how mathematically improbable it may appear to be. > -- than > > OLDTIMESTAMP=$(cat foo | sed -n -e '/pattern/s/^.*\(the good > stuff\).*$/\1/p') > OLDSEQ=${OLDTIMESTAMP:8} > NEWSEQ=$(( $OLDSEQ + 1 )) > NEWTIMESTAMP=`date +Y%m%d`${NEWSEQ} Sure, that would be fragile. Simply incrementing a serial counter is good, where it is updated manually, but its Achilles' heel is that it relies on human discipline. Automation removes that weakness, but it substitutes Paul Ehrlich's paradigm: "To err is human; to really foul things up, you need a computer". > since I don't have an xml parser coded in bash handy. We could create a tool to facilitate this particular task, but nah, I doubt it's worth the effort. > > * I don't think the -dev component needs to specify *any* > > "requires" dependencies; > > Disagree...see below > > ...snip detail; summarising... Okay. If I understand you correctly, you are looking at this from the perspective of a developer, and identifying as dependencies for libfoo-dev everything that you need to have to in place, to get you from foo-src to libfoo-dev itself. That's fine, but it isn't my perspective, and IMO it isn't what mingw-get dependency rules should be about -- at least, not right now. My perspective is more user oriented: if I install the pre-built libfoo-dev, what additional packages do I need to install, to be able to deploy it effectively in my own application development? THESE are the REAL dependencies which mingw-get needs to be concerned about. Of course, deployment of libfoo-dev must require a compiler and its supporting tool chain, but that's an implied dependency of any -dev package, not specific to libfoo-dev in particular, and therefore I don't think it is really appropriate to specify this as an explicit dependency for any -dev package. Similarly, any other -dev package which is an intrinsic dependency of the compiler suite, is not a direct prerequisite of libfoo-dev, so should not be specified as such. It does seem that we are able to agree on this. OTOH, if libfoo-dev cannot be successfully be deployed without also installing libbar-dev, which *isn't* intrinsic to the tool chain, then `<requires ge="libbar-*-subsys-*-dev.tar" />' is a legitimate assertion in the case of libfoo-dev; I agree that it should be specified -- my "-dev component needs no dependency declaration" was too sweepingly dismissive, in a broader context than the particular example to which it related. Your example of libintl-dev requiring libiconv-dev IS a case of a dependency which SHOULD be specified. However, in the particular case... > > Does it need > > the -dll component for -dev deployment? I suspect not. in the context of "does libfoo-dev really depend on libfoo-dll?" to enable use of libfoo-dev in an applications development work flow, is less cut and dried. If that work flow includes the running and testing of the developed application, then the answer would seem to be "yes". But wait. Is this dependency a property of libfoo-dev itself, or is it a dependency of the application being developed? My perspective -- and I'll freely admit that it's tempered by my normal use of a cross-hosted build environment, with testing taking place in an independent environment -- is that libfoo-dev does *not* depend on libfoo-dll, because I *don't* need libfoo-dll in my build environment; it is my developed application which depends on libfoo-dll, which I need to install in my test environment. > (*) side note, for later discussion: "sysroot" installations would > need a post-install step to "fixup" the ${root}/lib/*.la files, > and any ${root}/bin/{pkg}-config scripts, ditto > ${root}/lib/pkg-config/*.pc, ... I dunno if this sort of thing is > even worth thinking about. It has been mentioned before; I think, as I did then, that we should aim to incorporate some form of embedded script interpreter, to deal with post-install activities, but that's something for a later phase of development. > > * When you write the "requires" specification, the match string > > has the form of a canonical tarname, but the package-name part > > *must* match the "name" attribute of the containing "package" > > definition, irrespective of the actual tarname of the target > > download. > > Err...I'm confused. So, I need to explain it better. Let's try to illustrate it with an arbitrary example; our repository provides: libbar-1.0-2-msys-1.0.12-dev.tar.lzma libfoo-1.6-1-msys-1.0.14-dev.tar.lzma We write minimal package definitions for these, (embedded within a package collection element with subsystem="msys", which we don't show here): <package name="msys-bar"> <component class="dev"> <release tarname="libbar-1.0-1-msys-1.0.12-dev.tar.lzma" /> <release tarname="libbar-1.0-2-msys-1.0.12-dev.tar.lzma" /> </component> </package> <package name="msys-foo"> <component class="dev"> <release tarname="libfoo-1.6-1-msys-1.0.14-dev.tar.lzma" /> </component> </package> Now, (following on from the earlier discussion), we want to indicate that libfoo-dev depends on libbar-dev, (but we can accept any MSYS version which happens to be available, as long as it is at least libbar version 1.0 or later); let's try: <package name="msys-foo"> <component class="dev"> <release tarname="libfoo-1.6-1-msys-1.0.14-dev.tar.lzma" /> <requires ge="libbar-1.0-msys-*-dev.tar" /> </component> </package> If we now try `mingw-get install msys-foo-dev', it will error out, reporting "libbar: unknown package". Huh? Why didn't it find the "libbar-1.0-2-msys-1.0.12-dev.tar.lzma" <release> definition? It's because, when mingw-get parses that <requires> spec, it first calls pkginfo() to decompose the requirement spec into: * package name; (here it is "libbar") * package version; (here >= 1.0 with no upper bound) * subsystem name; (here it is "msys", matched case neutrally) * subsystem version; (in this case, anything goes) * component class; (here it is "dev") * archive type; (here it is "tar", but it won't be checked) Next, it will perform a top-down search of the XML manifest, looking in each <package-collection> element with a "subsystem" attribute which matches the target, "msys", for an embedded <package> element with a "name" attribute matching "libbar", containing a <component> element with "class" attribute matching "dev", and stopping as soon as it finds a complete match, or it runs out of places to search. In this case, there is no <package> to match "libbar", (we called it "msys-bar"), so the search fails. To fix it, we need to write our <requires> spec as: <requires ge="msys-bar-1.0-msys-*-dev.tar" /> With this change, the XML database search will find the appropriate <component> element within the msys-foo <package> element, so it then continues to inspect each of its contained <release> elements. In this phase of the search, it takes each <release> element in turn, passing its "tarname" attribute through pkginfo() to decompose it, and then considering *only* its package version and subsystem version fields, comparing them to the corresponding fields in the <requires> spec, until all <release> specs have been examined, and a reference to the "best fit" has been added to the "action list" for further processing. (In this context "best fit" is the highest numbered version which remains consistent with the selection criteria; at some point in the not too far distant future I also intend to add checks on development status tags, in case the user doesn't want to deploy less mature releases). > But my confusion is this: if the "package name" is "msys-zlib", > does that mean that **all** the <release> tags need to specify > themselves as: > > <release tarname="msys-zlib-1.2.3-2-msys-1.0.13-dev.tar.lzma"> > <download tarname="zlib-1.2.3-2-msys-1.0.13-dev.tar.lzma" /> > </release> No. The <release> tags should simply specify the canonical tarname of the actual package archive; the <download> override should be needed only when the archive name doesn't conform to the canonical naming rules. I hope that the preceding explanation clarifies how this will work. > going back to gettext, I'd need to specify, for each of the four > packages, a <source ...> element, as well as a "lic" component > (see far below): otherwise, mingw-get won't "know" that the > license tarball and source tarball for pkg "msys-libintl-dll" is > provided by > tarname="{msys}-gettext-%-msys-%-lic.tar" and > tarname="{msys}-gettext-%-msys-%-src.tar", > respectively. And no heuristic would ever be able to figure it > out. I'd like to think about this a bit before I reply. Perhaps I'll respond with a tentative specification for gettext. > > Ultimately, I think it should be you and Cesar who decide on > > your preferred layout, but clearly, I shouldn't abdicate all > > responsibility in helping you to formulate your choice. > > Well, the top-level layout is more a question of, what does/will > mingw-get support. I'd be perfectly happy to just provide content > in the form of appropriate msys-*.xml and mingw-*.xml fragments > for specific packages, and let somebody with a better > understanding of the mingw-get innards handle marshalling them > together. Hint... Of course, all of that -- including said .xml > fragments -- will be subject to commentary and revision driven by > discussion on this list. Absolutely. > > Well, most of the code to handle it is already written, (and > > used to segregate the installation manifests), but it will need > > some book keeping, to track the name changes as the downloads > > are moved into the local cache directory. However, you are > > right; it likely isn't especially urgent. > > I can't remember; were you planning to use a database backend of > some kind, or just flat files, to manage the *installation > manifest(s)* [as opposed to the *distribution manifest(s)]? Just flat files, again formatted as XML. The idea of using a backend database introduces a dependency on some DB server, which is an overhead I would prefer not to impose on mingw-get, in spite of the possible performance gain it might offer. > ...my assumptions were incorrect; nothing in *MinGW* requires > msys at all. And, the whole point of msys is to be minimal; if > somebody explicitly wants to experiment with a "crippled" > sub-minimum installation, who are we to say no? We shouldn't. Indeed, my own principal reason for using MSYS is to run pdfroff -- a Bourne shell script with accompanying troff macro package which I myself contributed to, and continue to maintain for the GNU Troff (groff) project. There is more in a standard MSYS Base System installation than I actually need, to fulfil this requirement. Also, for my MSYS-on-a-Stick application, I don't need everything, and here I have omitted some components. > > > Exit question 2: should ALL packages reference their own -lic > > > package (if they have one) as an explicit requirement? > > > > ... > > Actually, I think a better approach would be something similar to > the <source ...> element: an optional <lic ...> element if the > -lic tarname can't be "guessed" Good idea, and so obvious that I wonder why I didn't think of it myself. Thanks. I think, on balance, that I'd prefer the element name to be spelt out in full, but that introduces another GB vs US spelling conflict; naturally, I'd prefer the GB form: <licence>. > I think this to-install-lic-or-not decision should NOT be left to > 'did the packager remember to add <requires eq="*-lic">'...maybe > mingw-get could default to always installing lic packages, but > with a top-level flag or cmdline argument > --i-dont-care-about-licensing. I'm inclined to agree, but I'll need to think about how to force the requirement; it might take a while to figure it out. -- Regards, Keith. |
From: Keith M. <kei...@us...> - 2010-04-26 19:44:07
Attachments:
Makefile.in
|
On Friday 23 April 2010 22:51:25 Keith Marshall wrote: > > a simple a script that (a) updates the attribute, (b) > > checks in to CVS, (c) compresses using lzma, and (d) uploads the > > .lzma to <where-ever> on sourceforge -- would be my typical > > approach. And... > > > > TIMESTAMP=`date +%Y%m%d%H%M` > > cat foo | sed -e "s/@@TIMESTAMP@@/$TIMESTAMP/" > newfoo && > > mv newfoo foo > > > > is a lot more straightforward -- and robust > > But not without its own pitfalls, and Murphy's Law applies! > > When you base a serialisation counter on wall clock time, there is > always going to be a window within which a regression might occur > -- especially when developers in different time zones are > involved. We can mitigate that by always using UTC, (date -u > +%Y%m%d%H%M), but even that leaves a short (1 minute) window, > (possibly extended by clock skew if we don't all have our systems > tightly synchronised to the same atomic time reference), in which > a regression may occur; if it can happen, (and it can), then > Murphy tells us that it WILL, no matter how mathematically > improbable it may appear to be. In fact, the more I think about it, the *less* convinced I become, that a serialised issue number based on `date +%Y%m%d%H%M` actually is more robust than one based on `date +%Y%m%d`nn; in fact, with a little trivial bookkeeping, I think the reverse may be true. The attached Makefile.in illustrates the concept. As it is, this doesn't automate the (required) preliminary CVS update of the issue number registry (issue.log), nor the following commit, nor the upload of the generated foo.xml.lzma to SF, but these could easily be added. -- Regards, Keith. |
From: Charles W. <cwi...@us...> - 2010-04-27 02:00:46
|
On 4/26/2010 8:11 AM, Keith Marshall wrote: > On Friday 23 April 2010 22:51:25 Keith Marshall wrote: >>> a simple a script that (a) updates the attribute, (b) >>> checks in to CVS, (c) compresses using lzma, and (d) uploads the >>> .lzma to <where-ever> on sourceforge -- would be my typical >>> approach. And... >>> >>> TIMESTAMP=`date +%Y%m%d%H%M` >>> cat foo | sed -e "s/@@TIMESTAMP@@/$TIMESTAMP/" > newfoo && >>> mv newfoo foo >>> >>> is a lot more straightforward -- and robust >> >> But not without its own pitfalls, and Murphy's Law applies! >> >> When you base a serialisation counter on wall clock time, there is >> always going to be a window within which a regression might occur >> -- especially when developers in different time zones are >> involved. We can mitigate that by always using UTC, (date -u >> +%Y%m%d%H%M), but even that leaves a short (1 minute) window, >> (possibly extended by clock skew if we don't all have our systems >> tightly synchronised to the same atomic time reference), in which >> a regression may occur; if it can happen, (and it can), then >> Murphy tells us that it WILL, no matter how mathematically >> improbable it may appear to be. > > In fact, the more I think about it, the *less* convinced I become, > that a serialised issue number based on `date +%Y%m%d%H%M` actually > is more robust than one based on `date +%Y%m%d`nn; in fact, with a > little trivial bookkeeping, I think the reverse may be true. The > attached Makefile.in illustrates the concept. Well, that's just...cheating. <g> If you want to use an external registry file to establish (global or per-package) sequence numbers, then you can easily use the CVS timestamp -- imposed at the *server* by checkin, so no worries about clock skew -- of a similar external 'stamp' file as the timestamp for an updated .xml fragment: %.xml.lzma: %.xml DT=`date +%Y%m%dT%H%M%S%Z` echo "Last updated: $$DT $@" > stamp.new (cd $(srcdir) && cvs update stamp) if cmp stamp.new $(srcdir)/stamp ; then \ mv stamp.new $(srcdir)/stamp ; \ (cd $(srcdir) && cvs commit -m "update timestamp" stamp) ;\ fi # now, use 'cvs log stamp' to get the server timestamp of the # stamp file, and use that to update the xml. But...that has obvious drawbacks, so I'm not seriously proposing it. > As it is, this doesn't automate the (required) preliminary CVS update > of the issue number registry (issue.log), nor the following commit, > nor the upload of the generated foo.xml.lzma to SF, but these could > easily be added. Sure. I tend to think in terms of shell scripting, instead of makefiles, but there are benefits here of treating this like a 'build' process. (Provided the dependencies don't explode on you by mistake, causing erroneous "updates" of the issue registry). I don't have strong feelings about it either way; whichever way is easiest (for me...) and causes (me...) the least hassle, I'm for it. So, if somebody were already planning write a Makefile.in that just Does The Right Thing for whatever issue number schema is adopted, I'd be happy to use it. <g> -- Chuck |
From: Charles W. <cwi...@us...> - 2010-04-27 03:32:32
Attachments:
msys-zlib.xml
|
On 4/23/2010 5:51 PM, Keith Marshall wrote: > On Friday 23 April 2010 00:43:03 Charles Wilson wrote: >> since I don't have an xml parser coded in bash handy. > > We could create a tool to facilitate this particular task, but nah, I > doubt it's worth the effort. Agreed. >>> * I don't think the -dev component needs to specify *any* >>> "requires" dependencies; >> >> Disagree...see below >> >> ...snip detail; summarising... > > Okay. If I understand you correctly, you are looking at this from > the perspective of a developer, and identifying as dependencies for > libfoo-dev everything that you need to have to in place, to get you > from foo-src to libfoo-dev itself. Err...not exactly. Those would be the build requires for libfoo-dev (which right now, we just list in the RELEASE_NOTES.txt file). No, the "runtime" requires (e.g. the ones listed in the xml fragment) are those necessary for an end user to USE -- that is, build some other package bar -- that lists libfoo-dev in bar's build-requires list. Classic -- and common -- example: bar uses libintl, but does NOT directly use libiconv. To link bar.exe I need libiconv.[a, .dll.a]; a (static) link will fail otherwise, because I need libiconv.a to satisfy undefined symbols in libintl.a. (Now, sometimes a dynamic link will succeed, because there are no undefined symbols in libintl.dll.a -- but when I'm packaging libintl-dev I don't know that Bob plans to build bar dynamically or statically, so I have to assume worst case: the link will fail if libiconv.a is not available. So, according to this chain of reasoning, libintl-dev should list libiconv-dev as one of its requirements. (However, this is a bad choice to verify on linux, since iconv() is provided by glibc not GNU libiconv...). So, picking another example: rpm -q --requires libpng-devel ... zlib-devel devel(libz) ... Now, TECHNICALLY, building foo would ALSO fail if any of the following msys packages were missing: msys-gcc (duh!) msys-binutils msys-msysCORE-dev msys-w32api (...maybe.) But, I think we're pretty much in agreement that -- in order to allow SOME ability of end user to install -dev packages for mere inspection purposes, without pulling in all of msysDVLPR, if for no other reason -- these "bottom level" -dev package can be omitted. Even tho you really CAN'T build bar.exe (which depends on libfoo, which -- really -- depends on header files and libs provided by those bottom level -dev pkgs) without them. Besides, the linux distros don't insert gcc in the requires of all -dev packages, either. > That's fine, but it isn't my > perspective, and IMO it isn't what mingw-get dependency rules should > be about -- at least, not right now. If I had been talking about the build-requires to transform foo-src to libfoo-dev, I would agree: mingw-get dependency rules should not be about build-requires. But... They should be about usability requirements. On an msys system, it does me no good to install libintl-dev unless I also install libiconv-dev. I can't USE libintl-dev without it. > My perspective is more user oriented: if I install the pre-built > libfoo-dev, what additional packages do I need to install, to be > able to deploy it effectively in my own application development? > THESE are the REAL dependencies which mingw-get needs to be > concerned about. Err...right...<thinking: gee, sounds like we're talking about the same thing. Getting confused as to what the argument is...> > Of course, deployment of libfoo-dev must require a compiler and its > supporting tool chain, but that's an implied dependency of any -dev > package, not specific to libfoo-dev in particular, and therefore I > don't think it is really appropriate to specify this as an explicit > dependency for any -dev package. Alright, so far so good... > Similarly, any other -dev package > which is an intrinsic dependency of the compiler suite, is not a > direct prerequisite of libfoo-dev, so should not be specified as > such. It does seem that we are able to agree on this. e.g. mingw-runtime (for 'mingw'-ish -dev packages), msysCORE-dev for msys-ish ones, right, right, okay,... > OTOH, if libfoo-dev cannot be successfully be deployed without also > installing libbar-dev, which *isn't* intrinsic to the tool chain, > then `<requires ge="libbar-*-subsys-*-dev.tar" />' is a legitimate > assertion in the case of libfoo-dev; I agree that it should be > specified -- my "-dev component needs no dependency declaration" was > too sweepingly dismissive, in a broader context than the particular > example to which it related. Your example of libintl-dev requiring > libiconv-dev IS a case of a dependency which SHOULD be specified. Ah-HA! We DO agree...at least this far. > However, in the particular case... > >>> Does it need >>> the -dll component for -dev deployment? I suspect not. > > in the context of "does libfoo-dev really depend on libfoo-dll?" to > enable use of libfoo-dev in an applications development work flow, > is less cut and dried. If that work flow includes the running and > testing of the developed application, then the answer would seem to > be "yes". Right. And I'll go back to linux: $ rpm -q --requires libpng-devel libpng3 = 2:1.2.40-1.1mdv2010.0 zlib-devel ... $ rpm -q --requires libopenssl0.9.8-devel libopenssl0.9.8 = 0.9.8k-5.1mdv2010.0 devel(libcrypto) devel(libdl) ... > But wait. Is this dependency a property of libfoo-dev > itself, or is it a dependency of the application being developed? > My perspective -- and I'll freely admit that it's tempered by my > normal use of a cross-hosted build environment, with testing taking > place in an independent environment -- is that libfoo-dev does *not* > depend on libfoo-dll, because I *don't* need libfoo-dll in my build > environment; it is my developed application which depends on > libfoo-dll, which I need to install in my test environment. Ah, but the same argument would be true on linux: if I'm building for $host=solaris the package bar -- which needs (solaris)libfoo-dev to build -- I don't need to install (solaris)libfooN on my linux box. But...Red Hat is not in the business, usually, of providing a bunch of explicit packages for cross-hosted sysroots. They have no "solaris-libfoo-dev". Granted, Fedora DOES actually provide an entire *mingw* sysroot package set, but that's a special case. WE'RE talking about managing the installation of *native* msys and mingw packages; not a set of mingw-ish/msys-ish sysroot packages (*). So, just as Mandriva's "libpng-devel" -- a devel package for native linux development -- requires the .so package libpng3 (as well as the zlib-devel package), our mingw libfoo-dev package should require the .dll package libfoo-dll-N. So...except for this last issue, I think we're in violent agreement. >> (*) side note, for later discussion: "sysroot" installations would >> need a post-install step to "fixup" the ${root}/lib/*.la files, >> and any ${root}/bin/{pkg}-config scripts, ditto >> ${root}/lib/pkg-config/*.pc, ... I dunno if this sort of thing is >> even worth thinking about. > > It has been mentioned before; I think, as I did then, that we should > aim to incorporate some form of embedded script interpreter, to deal > with post-install activities, but that's something for a later phase > of development. Right...the whole sysroot thing is a can of worms. (*) I could see explicit logic in (unix-hosted)mingw-get to filter the requirements list: "-dev packages that list -dll packages in their requirements should have those -dll requirements filtered out". >> Err...I'm confused. > > So, I need to explain it better. Let's try to illustrate it with an > arbitrary example; our repository provides: [snip] > I hope that the preceding explanation clarifies how > this will work. I think I see now. This also make it more clear why you'd want four different packages for the gettext collection. Otherwise, your requirements get...weird: <package "msys-foo"> <release tarname="libfoo-1.6-1-msys-1.0.14-dev.tar.lzma" <requires ge="msys-gettext-*-msys-*-dll-8.tar" /> </release> err...what? foo uses "libintl", what's this "msys-gettext" thing? So, yeah, you probably want to put libintl-dll into its own "msys-libintl" package. Then there's the whole dll soname issue which Cesar brought up, but we can defer that discussion to his thread. >> going back to gettext, I'd need to specify, for each of the four >> packages, a <source ...> element, as well as a "lic" component >> (see far below): otherwise, mingw-get won't "know" that the >> license tarball and source tarball for pkg "msys-libintl-dll" is >> provided by >> tarname="{msys}-gettext-%-msys-%-lic.tar" and >> tarname="{msys}-gettext-%-msys-%-src.tar", >> respectively. And no heuristic would ever be able to figure it >> out. > > I'd like to think about this a bit before I reply. Perhaps I'll > respond with a tentative specification for gettext. Meh. I think this is ok. If I want to separate the DLL subpackages into their own "packages", then some duplication of the provenance data is going to be necessary: <source tarname="....">, <licence tarname="..."> is fine, especially if the version number matching (%) is robust enough. I gotta somehow tell mingw-get that "msys-libintl" 'comes from' "msys-gettext" -- it has no other way to know that. >> I can't remember; were you planning to use a database backend of >> some kind, or just flat files, to manage the *installation >> manifest(s)* [as opposed to the *distribution manifest(s)]? > > Just flat files, again formatted as XML. The idea of using a backend > database introduces a dependency on some DB server, which is an > overhead I would prefer not to impose on mingw-get, in spite of the > possible performance gain it might offer. Ack. IF you were to use a DB -- and I'm not saying you should -- I don't think it should be a heavyweight one that actually requires a separate db server process. Both mysqlite and gdbm operate "in process" -- as do the plain old BSD ndbm implementations. [snip discussion for sub-minimum msys installations] >>>> Exit question 2: should ALL packages reference their own -lic >>>> package (if they have one) as an explicit requirement? >>> >>> ... >> >> Actually, I think a better approach would be something similar to >> the <source ...> element: an optional <lic ...> element if the >> -lic tarname can't be "guessed" > > Good idea, and so obvious that I wonder why I didn't think of it > myself. Thanks. I think, on balance, that I'd prefer the element > name to be spelt out in full, but that introduces another GB vs US > spelling conflict; naturally, I'd prefer the GB form: <licence>. Fine by me. De ones dat writes da code gets ta pick da names. >> I think this to-install-lic-or-not decision should NOT be left to >> 'did the packager remember to add <requires eq="*-lic">'...maybe >> mingw-get could default to always installing lic packages, but >> with a top-level flag or cmdline argument >> --i-dont-care-about-licensing. > > I'm inclined to agree, but I'll need to think about how to force the > requirement; it might take a while to figure it out. Ack. Here's my revised msys-zlib.xml, incorporating all the changes suggested so far (I'm only guessing about msysCORE...) -- Chuck |
From: Charles W. <cwi...@us...> - 2010-04-27 02:31:32
|
On 4/26/2010 4:45 PM, Charles Wilson wrote: > mingw-get-xml/ > msys-zlib.xml > msys-blah.xml > ... > mingw-autoconf.xml > mingw-automake.xml > ... > > where *perhaps* cvs modules might list all of the msys-* as part of the > "mingw-get-xml-msys" module, and all of the mingw-* ones as part of the > mingw-get-xml-mingw module. But I think even that is overkill. Just > check 'em all out as a common module; modify locally only the ones you > care about. Oh, one other thought: if we go with "msys packages must use an explicit 'msys' prefix in their package name, but mingw packages need no explicit prefix" then it would make sense to have a mingw-get-xml-mingw module, just so that all of the variously named: autoconf.xml automake.xml gcc.xml g++.xml etc fragments can be positively (*) identified as part of the 'set of all xml fragments associated with mingw packages'. (*) as opposed to "negatively" identified: "ls * | grep -Ev '^msys-'" == mingw -- Chuck |
From: Charles W. <cwi...@us...> - 2010-05-04 03:15:57
|
On 4/26/2010 10:30 PM, Charles Wilson wrote: > On 4/26/2010 4:45 PM, Charles Wilson wrote: >> mingw-get-xml/ See below. >> msys-zlib.xml >> msys-blah.xml >> ... >> mingw-autoconf.xml >> mingw-automake.xml >> ... >> >> where *perhaps* cvs modules might list all of the msys-* as part of the >> "mingw-get-xml-msys" module, and all of the mingw-* ones as part of the >> mingw-get-xml-mingw module. But I think even that is overkill. Just >> check 'em all out as a common module; modify locally only the ones you >> care about. On this: > Oh, one other thought: if we go with "msys packages must use an explicit > 'msys' prefix in their package name, but mingw packages need no explicit > prefix" then it would make sense to have a mingw-get-xml-mingw module, > just so that all of the variously named: > > autoconf.xml > automake.xml > gcc.xml > g++.xml > > etc fragments can be positively (*) identified as part of the 'set of > all xml fragments associated with mingw packages'. > > (*) as opposed to "negatively" identified: "ls * | grep -Ev '^msys-'" == > mingw ...I punt, for now. I propose to create a new top-level directory in CVS named mingw-dist (*) as a repository for the various *.xml files/fragments to be used by mingw-get. "mingw-dist" because "mingw-get-xml" is just too wordy, and I do want the new dir to show up right next to mingw-get/). (*) suggestions? I'm not planning (yet) to create any entries in the CVSROOT/MODULES file. In mingw-dist/, I'll start checking in the various msys-*.xml files, but NOT any of the ones which would correspond to -mingw32- packages. This will give us more time to work thru the "to-prefix-or-not-to-prefix" question, and the "create MODULES entry or not" question. I'm not planning to create a download location in the FRS, nor to lzma/copy these .xmls over there. This will give us time to work out THOSE details, too. I'm not yet planning to put an upload script, nor Makefile, in the mingw-dist directory. Let's get some .xml files up there, first. Just xml. Nothin' but xml. For now. Furthermore, I don't plan any directory structure under mingw-dist/; just 50 or so files names msys-*.xml. When you add in another 20 or so for the various mingw-ish packages, that's really not too many. I'm probably not going to do this very fast. I plan to upload one or two updated msys-* package to FRS, and checkin their corresponding .xml files, per day. There are 53 different msys packages: autoconf/ dash/ gmp/ libxml2/ patch/ unzip/ automake/ diffutils/ grep/ lndir/ perl/ vim/ bash/ expat/ groff/ m4/ popt/ wget/ bison/ file/ gzip/ make/ rebase/ xz/ bzip2/ findutils/ inetutils/ man/ regex/ zlib/ coreutils/ flex/ less/ minires/ sed/ crypt/ gawk/ libarchive/ mktemp/ tar/ cvs/ gdbm/ libiconv/ openssh/ termcap/ cygutils/ gettext/ libtool/ openssl/ texinfo/ plus guile/, autogen/ and rxvt/. You do the math; I'll do them in dependency order, so even if you manually update to the "new" package every day nothing should break (heck, that's what *I* did while working my way through the list of packages, anyway...) FWIW, I've finished updating all except those last three (guile, autogen, rxvt), but plan on completing those this week. Note: dash, rebase, libxml2, expat, unzip, and wget are new: Rebase seems to now be necessary in *some* cases for the same reasons it is distributed with cygwin: the dreaded "*** unable to remap <some dll> to same address as parent(0xDF0000) != 0xE00000" error. Dash is provided specifically so that one can have a POSIX shell which does NOT rely on any DLLs other than msys itself, from which to run rebase. (Plus, the script "rebaseall" checks to see that the shell *is* dash, besides using the #!/bin/dash shbang line.) As an aside, this version of dash supports $LINENO, so it *ought* to be sufficient to run configure scripts; it'd be interesting to see if, on msys, there is a significant speedup over bash. Expat is present to support new features in xgettext.exe from gettext-0.17. libxml2 is present to support an upcoming version rev of perl (IIRC; it's been a couple of months since I did this while fighting to version-rev perl from 5.6.1 to 5.8.8/5.10.0. I temporarily gave up on that, so as of today, even the "new" perl is still based on 5.6.1. So...libxml2 remains, but is not yet used). Unzip is provided to support my upcoming (but incomplete, and not listed above) console2 package -- whose functionality will be "run this script to download, unpack, and configure console2 for use with MSYS/MinGW". (Because the upstream console2 fellas ship their binaries in .zip files.) Anyway, for symmetry, I *may* also port Zip to msys, but not soon. msys-wget is provided also to support the upcoming console2 package. Unlike the mingw port of wget we provide, this one has SSL support (thanks to msys-openssl-100.dll) -- which is both good and bad. I find that I need to use --no-check-certificate since I'm too lazy to install certificates into my msys tree. Technically, lpr-enhanced is also an msys- package, but since it doesn't have any binary components, I didn't include it in the "rebuild" ordeal. Similarly, the msys "version" of w32api should also be included somewhere, but... -- Chuck |
From: Keith M. <kei...@us...> - 2010-05-04 18:00:09
|
On Tuesday 04 May 2010 04:14:54 Charles Wilson wrote: > I propose to create a new top-level directory in CVS named > mingw-dist (*) I've been planning to do this myself; `/pkglist' or `/package-list' were my initial thoughts, but I like `/mingw-dist' better; thanks! > as a repository for the various *.xml files/fragments to be used > by mingw-get. "mingw-dist" because "mingw-get-xml" is just too > wordy, and I do want the new dir to show up right next to > mingw-get/). > > (*) suggestions? Do you have sufficient "karma" to create a top level directory? I can't remember, but in any case, I'll create the empty structure for you, tonight. I'll make it like: $CVSROOT mingw-dist mingw32 msys > I'm not planning (yet) to create any entries in the > CVSROOT/MODULES file. I doubt if you have "karma" for that anyway, but we can defer any decision on whether or not to proceed with it, until we get some content in place. (We can also address the "karma" issue later, if it becomes necessary). > In mingw-dist/, I'll start checking in the various msys-*.xml > files, Please do so, in $CVSROOT/mingw-dist/msys/ rather than directly into $CVSROOT/mingw-dist/ > but NOT any of the ones which would correspond to -mingw32- > packages. This will give us more time to work thru the > "to-prefix-or-not-to-prefix" question, and the "create MODULES > entry or not" question. Sure. I will, in any case set up the infrastructure so that Cesar, (and anyone else who cares to participate), can start to populate $CVSROOT/mingw-dist/mingw32 when he is ready. > I'm not planning to create a download location in the FRS, nor to > lzma/copy these .xmls over there. This will give us time to work > out THOSE details, too. Sure. These probably want to go in a pseudo-subdirectory of the mingw-get tree itself. > I'm not yet planning to put an upload script, nor Makefile, in the > mingw-dist directory. I may do so, based on the tentative idea I proposed last week. If you specify the `issue' attribute, (in the root element of each of your XML files), as: issue="@YYYYMMDDNN@" then that can be automatically managed, as the .xml.lzma files are generated prior to upload. Once it's there, we can refine it as we firm up the detailed requirements. > Let's get some .xml files up there, first. Sure. > Just xml. Nothin' but xml. For now. Furthermore, I don't plan any > directory structure under mingw-dist/; just 50 or so files names > msys-*.xml. When you add in another 20 or so for the various > mingw-ish packages, that's really not too many. Nonetheless, I'd still prefer to segregate mingw32 and msys from the outset, as I've specified above; I don't think any deeper level of directory branching should be necessary. -- Regards, Keith. |
From: Charles W. <cwi...@us...> - 2010-05-05 00:46:58
|
On 5/4/2010 5:16 AM, Keith Marshall wrote: > On Tuesday 04 May 2010 04:14:54 Charles Wilson wrote: >> I propose to create a new top-level directory in CVS named >> mingw-dist (*) > > I've been planning to do this myself; `/pkglist' or `/package-list' > were my initial thoughts, but I like `/mingw-dist' better; thanks! NP. > Do you have sufficient "karma" to create a top level directory? I used to, back when I needed it to reorganize the mingw-utils stuff. But it was supposed to be removed; I'm not sure if it ever was. > I > can't remember, but in any case, I'll create the empty structure for > you, tonight. I'll make it like: > > $CVSROOT > mingw-dist > mingw32 > msys > >> I'm not planning (yet) to create any entries in the >> CVSROOT/MODULES file. > > I doubt if you have "karma" for that anyway, but we can defer any > decision on whether or not to proceed with it, until we get some > content in place. (We can also address the "karma" issue later, if > it becomes necessary). Ack. (Actually, this is the same as above; I used to -- and still may, but am no longer supposed to -- have this much karma). >> In mingw-dist/, I'll start checking in the various msys-*.xml >> files, > > Please do so, in $CVSROOT/mingw-dist/msys/ rather than directly into > $CVSROOT/mingw-dist/ OK. >> but NOT any of the ones which would correspond to -mingw32- >> packages. This will give us more time to work thru the >> "to-prefix-or-not-to-prefix" question, and the "create MODULES >> entry or not" question. > > Sure. I will, in any case set up the infrastructure so that Cesar, > (and anyone else who cares to participate), can start to populate > $CVSROOT/mingw-dist/mingw32 when he is ready. Ack. >> I'm not planning to create a download location in the FRS, nor to >> lzma/copy these .xmls over there. This will give us time to work >> out THOSE details, too. > > Sure. These probably want to go in a pseudo-subdirectory of the > mingw-get tree itself. Sounds ok. >> I'm not yet planning to put an upload script, nor Makefile, in the >> mingw-dist directory. > > I may do so, based on the tentative idea I proposed last week. If > you specify the `issue' attribute, (in the root element of each of > your XML files), as: > > issue="@YYYYMMDDNN@" > > then that can be automatically managed, as the .xml.lzma files are > generated prior to upload. Once it's there, we can refine it as we > firm up the detailed requirements. Sounds good to me. > Nonetheless, I'd still prefer to segregate mingw32 and msys from the > outset, as I've specified above; I don't think any deeper level of > directory branching should be necessary. Fine by me. -- Chuck |
From: Cesar S. <ces...@gm...> - 2010-05-07 02:34:55
|
Charles Wilson wrote: > Or, instead, move all *scripts* and items with additional, external > dependencies out of msysCORE-bin and into msysCORE-ext. OK by me. I'll take the opportunity and refresh the mount/umount scripts from CVS. I do not have any DLL changes so far, so I can release it as 1.0.14-2. How about this layout: msysCORE-1.0.14-2-msys-1.0.14-bin.tar.lzma: bin/msysmnt.exe bin/ps.exe bin/msys-1.0.dll msysCORE-1.0.14-2-msys-1.0.14-ext.tar.lzma: m.ico msys.bat msys.ico bin/cls bin/clsb bin/cmd bin/ftp bin/lnkcnv bin/mount bin/msysinfo bin/start bin/umount bin/which etc/fstab.sample etc/inputrc.default etc/profile share/doc/MSYS/ share/doc/MSYS/COPYING share/doc/MSYS/COPYING.LIB share/doc/MSYS/CYGWIN_LICENSE share/doc/MSYS/msysCORE-1.0.14-1-msys-RELEASE_NOTES.txt share/doc/MSYS/MSYS_LICENSE.rtf share/doc/MSYS/MSYS_MISSION share/doc/MSYS/MSYS_VS_CYGWIN share/doc/MSYS/MSYS_WELCOME.rtf share/doc/MSYS/README.rtf postinstall/pi.bat postinstall/pi.sh Regards, Cesar |
From: Cesar S. <ces...@gm...> - 2010-05-07 02:42:12
|
Cesar Strauss wrote: > How about this layout: Sorry, I mean: msysCORE-1.0.14-2-msys-1.0.14-bin.tar.lzma: bin/msys-1.0.dll bin/msysmnt.exe bin/ps.exe share/doc/MSYS/COPYING share/doc/MSYS/COPYING.LIB share/doc/MSYS/CYGWIN_LICENSE share/doc/MSYS/msysCORE-1.0.14-1-msys-RELEASE_NOTES.txt share/doc/MSYS/MSYS_LICENSE.rtf share/doc/MSYS/MSYS_MISSION share/doc/MSYS/MSYS_VS_CYGWIN share/doc/MSYS/MSYS_WELCOME.rtf share/doc/MSYS/README.rtf msysCORE-1.0.14-2-msys-1.0.14-ext.tar.lzma: m.ico msys.bat msys.ico bin/cls bin/clsb bin/cmd bin/ftp bin/lnkcnv bin/mount bin/msysinfo bin/start bin/umount bin/which etc/fstab.sample etc/inputrc.default etc/profile postinstall/pi.bat postinstall/pi.sh |
From: Charles W. <cwi...@us...> - 2010-05-07 03:27:47
|
On 5/6/2010 10:42 PM, Cesar Strauss wrote: > Cesar Strauss wrote: >> How about this layout: > msysCORE-1.0.14-2-msys-1.0.14-bin.tar.lzma: > > bin/msys-1.0.dll > bin/msysmnt.exe > bin/ps.exe > share/doc/MSYS/COPYING > share/doc/MSYS/COPYING.LIB > share/doc/MSYS/CYGWIN_LICENSE > share/doc/MSYS/msysCORE-1.0.14-1-msys-RELEASE_NOTES.txt ^^^^^ 2 > share/doc/MSYS/MSYS_LICENSE.rtf > share/doc/MSYS/MSYS_MISSION > share/doc/MSYS/MSYS_VS_CYGWIN > share/doc/MSYS/MSYS_WELCOME.rtf > share/doc/MSYS/README.rtf + m.ico + msys.ico It's not like these have any actual dependency on anything else; they're just data. + etc/fstab.sample + etc/inputrc.default These are basically documentation. + etc/profile This is tricky. We're trying to set things up so that msys-bash can <require> msysCORE-bin, and msysCORE-ext can <require> msysCORE-bin AND msys-bash-bin. So, if /etc/profile is in msysCORE-ext, then you'd be very tempted to have msys-bash <require> msysCORE-ext, and bang! circular dependency. So, I think /etc/profile belongs in msysCORE-bin, even though it is a "shell script". > msysCORE-1.0.14-2-msys-1.0.14-ext.tar.lzma: > > msys.bat > bin/cls > bin/clsb > bin/cmd > bin/ftp > bin/lnkcnv > bin/mount > bin/msysinfo > bin/start > bin/umount > bin/which > postinstall/pi.bat > postinstall/pi.sh But hold off on taking /any/ action until Keith signs off on the plan; he's the grand master of mingw-get mysteries... -- Chuck |
From: Charles W. <cwi...@us...> - 2010-05-07 05:07:59
|
On 5/6/2010 10:34 PM, Cesar Strauss wrote: > OK by me. I'll take the opportunity and refresh the mount/umount scripts > from CVS. I do not have any DLL changes so far, so I can release it as > 1.0.14-2. About that... It just occurred to me: I've been having some trouble rebuilding guile with the new msys-gcc-3 and cygwin-like --enable-runtime-pseudo-relocs (the old version of msys-guile was more mingw-ish, using explicit declspec decorations instead of relying on auto-import/pseudo-relocs. However, doing it the "mingw" way makes it impossible to update msys-autogen past version 5.9.2). And this morning, the following pops up on the cygwin-patches list: Re: CFA: pseudo-reloc v2 http://cygwin.com/ml/cygwin-patches/2010-q2/msg00004.html http://cygwin.com/ml/cygwin-patches/2010-q2/msg00016.html The gist of it is, Dave Korn discovered a flaw in cygwin (and msys) pseudo-relocs and has proposed a fix. They're still trying to determine the best way to implement the fix for cygwin; after they're done we might want to adopt something similar for msysCORE-1.0.15. This would only affect msys DLLs and EXEs that are compiled with --enable-runtime-pseudo-relocs, which is exactly zero at this time. The only candidates for exploiting pseudo-reloc support which are part of the official set of packages are: guile and autogen. That's it. So it's not like there's any rush, but I thought I should mention it. -- Chuck |
From: Charles W. <cwi...@us...> - 2010-05-07 23:01:56
|
On 5/7/2010 1:07 AM, Charles Wilson wrote: > On 5/6/2010 10:34 PM, Cesar Strauss wrote: >> OK by me. I'll take the opportunity and refresh the mount/umount scripts >> from CVS. I do not have any DLL changes so far, so I can release it as >> 1.0.14-2. > > About that... Well, there are also the modifications to sys/unistd.h and netdb.h -- which technically do not affect the DLL itself, but they ARE part of the distribution... I'm currently testing Dave Korn's pseudo-reloc + fork fix (*not* cgf's version that eventually went into cygwin), and DK's implementation *technically* also does not affect the DLL either -- since it involves startup code that is statically linked into each EXE and DLL compiled under msys. But still, I think those changes -- assuming the pseudo-reloc fix passes muster -- are sufficient for a 1.0.15 bump. * Add declarations of fchdir and getdomainname to sys/unistd.h * Add declarations of rcmd, rexec, and rresvport to netdb.h * Correct bug involving double evaluations of pseudo-relocations after fork(). Affected MSYS applications must be recompiled to take advantage of this fix. * Add --replace option to mount and umount scripts * Split -bin component into -bin and -ext. -- Chuck |
From: Cesar S. <ces...@gm...> - 2010-05-08 23:55:07
|
On 05/07/2010 08:01 PM, Charles Wilson wrote: > But still, I think those changes -- assuming the pseudo-reloc fix passes > muster -- are sufficient for a 1.0.15 bump. > > > * Add declarations of fchdir and getdomainname to sys/unistd.h > * Add declarations of rcmd, rexec, and rresvport to netdb.h > * Correct bug involving double evaluations of pseudo-relocations > after fork(). Affected MSYS applications must be recompiled to > take advantage of this fix. > * Add --replace option to mount and umount scripts > * Split -bin component into -bin and -ext. > Certainly. Feel free to go ahead with this. Thanks, Cesar |
From: Charles W. <cwi...@us...> - 2010-05-09 01:25:00
Attachments:
build-machinery-1.0.15.patch
|
On 5/8/2010 7:54 PM, Cesar Strauss wrote: > On 05/07/2010 08:01 PM, Charles Wilson wrote: >> But still, I think those changes -- assuming the pseudo-reloc fix passes >> muster -- are sufficient for a 1.0.15 bump. >> >> >> * Add declarations of fchdir and getdomainname to sys/unistd.h >> * Add declarations of rcmd, rexec, and rresvport to netdb.h >> * Correct bug involving double evaluations of pseudo-relocations >> after fork(). Affected MSYS applications must be recompiled to >> take advantage of this fix. >> * Add --replace option to mount and umount scripts >> * Split -bin component into -bin and -ext. >> > > Certainly. Feel free to go ahead with this. Well, (a) the patches must still be reviewed -- I'm not gonna commit new code to CVS without review. (b) I'd prefer it if you handled msysCORE releases. I was just raising the flag that I had /just/ discovered a pending issue affecting the DLL right as we were discussing a reorganization release -- which, given this pending issue, it probably should be a true version-bump release. FWIW, this is the patch to the build machinery, assuming: that my patches to msys itself are approved and committed, and you update your local build tree with that version of the CVS source, and with the latest version of the base-files -- e.g. mount/umount. -- Chuck |