From: Keith M. <kei...@us...> - 2010-04-21 23:07:27
|
On Wednesday 21 April 2010 01:13:07 Charles Wilson wrote: > > Probably not as fine grained as you are thinking of, but > > mingw-get CVS is now able to construct the distribution manifest > > from multiple secondary files, with one top-level master index > > file, interpreted recursively to incorporate the secondaries. > > But, AFAICT, only at specific "customization points": e.g. entire > <package-list> elements. That is: > > <package-list catalogue="msys-base" /> > > "#includes" msys-base.xml. But, in msys-base.xml can you do: > > <package catalogue="coreutils"> > > which "#includes" coreutils.xml? No, but you can have msys-base.xml call out <package-list catalogue="coreutils" /> and then have coreutils.xml define <software-distribution ...> <package-collection subsystem="msys"> <download-host uri="..." /> <package name="msys-coreutils"> ... </package> </package-collection> </software-distribution> with all the information pertinent to msys-coreutils, and no other package, specified therein; when searching for the msys-coreutils package, mingw-get will search through its list of package-collection elements, with appropriate subsystem attribute, until it finds one which contains the requisite package entry, returning the first hit. > (FYI: Thunderbird appears to be US-centric. It's complaining about > 'catalogue' being misspelled, but is perfectly happy with > 'catalog') Yeah. Firefox does likewise, even when configured with its en_GB dictionary :( However, being a Brit, "catalog" just looks wrong to me, (and KMail, with its en_GB dictionary, flags it as misspelled). Can't please all of the people all of the time. British developers will tend to favour GB English spellings, (which AFAIK, are actually preferred universally, in the English speaking world outside of North America -- maybe even in Canada; I don't know for sure). From a technical stand-point, of course, mingw-get doesn't care how "catalogue" is spelt, or even that the word is "catalogue" at all; the sole requirement is that the attribute is consistently identified by the same unique arbitrary sequence of characters, which can be represented in UTF-8 encoding, and legal as an XML identifier, (and I'd also like to mandate that they be restricted to the ASCII printable subset of UTF-8, if XML doesn't already require this -- I'm not certain). I chose "catalogue" in this context, because it made sense to me, but I guess <package-list name="coreutils" /> would be just as acceptable, and would avoid any potential for confusion between GB and US spellings. If I make that change, it will break alpha-1's profile.xml, but that perhaps isn't too serious if I do it now; it could be more painful to delay the change. What do you think? Should I change it, for preference? > But if you specify two different .xml files, one for each release, > you'd have: > > [two disparate package definitions] > > Merging those automatically is...not fun. Exactly. As you say: we don't want to go there! The XML root element is a "software-distribution", and within that we can have: <software-distribution ...> <!-- an arbitrary number of ... --> <package-list name="foo" /><!-- each with a unique name --> <package-group-hierarchy> <!-- just one of these, but I plan to accommodate merging of fragments from individual package list files, to construct it --> ... </package-group-hierarchy> <!-- an arbitrary number of ... --> <package-collection subsystem="foosys"> <!-- required once, in each package-collection --> <download-host uri="template" /> <!-- one or more, but only one globally, for each named package --> <package name="bar"> ... </package> ... </package-collection> </software-distribution> > Right. So, I'm thinking IF we were to go as fine-grained as "one > xml file per unique source tarball, without regard to version", We can go as fine grained as one XML file per *package*; (strictly per package-collection, but that could contain as few as one actual package definition). > then you'd need to edit/commit a new (e.g.) gettext.xml each time > you upload a new set of > > gettext-0.17-2-msys-1.0.13-bin.tar.lzma > gettext-0.17-2-msys-1.0.13-dev.tar.lzma > gettext-0.17-2-msys-1.0.13-doc.tar.lzma > gettext-0.17-2-msys-1.0.13-ext.tar.lzma > gettext-0.17-2-msys-1.0.13-lic.tar.lzma > gettext-0.17-2-msys-1.0.13-src.tar.lzma > libasprintf-0.17-2-msys-dll-0.tar.lzma > libgettextpo-0.17-2-msys-dll-0.tar.lzma > libintl-0.17-2-msys-dll-8.tar.lzma Here you would edit the *existing* gettext.xml, (which would likely define four packages within its package-collection)... > adding new entries for those 9 tarballs (actually, 8, because -src > doesn't get an entry -- or does it [*]), without removing any > existing elements from the xml. Yes, you would add new <release> tag entries in each of the embedded <component> elements, within their appropriate package definitions; for the time being, you would leave the existing <release> entries in place, and unchanged. > [*] You mentioned a possible element type, sibling to <component>: > <source tarname="gettext-%-msys-%-src.tar.lzma"> > but I'm not sure if that's implemented yet, whether '&' or '%' > was used for the "matching version number" wildcard, or whether > you can use it twice to match both package version/release and > subsystem- version. I haven't (fully) implemented source retrieval logic yet, but my idea is that, as far as practical, the source tarball name can be deduced by simply substituting "src" for the component class within the canonical tarname of any related release definition, and therefore no explicit source spec will be needed. Obviously, there will be cases, such as your example here, where this will not suffice; for such cases, a <source> tag, either within the <release> element itself, (with an explicit source tarname attribute), or as either a <release> sibling or a <component> sibling, (providing a version matched template for source tarname generation), will provide the necessary override. For the version matching wildcard, I first considered "&", but in XML that needs to be represented as "&", which just seems needlessly clumsy, so I now think "%" will be a better choice; and yes, it will be allowed to match either or both of the package version and the subsystem version fields, within a single template. I think we are going to need this capability quite soon -- the "%" wildcard for version matching may also be useful to facilitate more concise dependency specifications -- so perhaps I should turn my attention to it, after I get alpha-2 out? (I also now have the installation records segregated into separate per-package manifest files, but I still need to address the uninstall facility). > > Keeping the manifest in small recursively included > > sub-files may help to make it more manageable. > > Yep -- but it's going to be a balance between maintenance and how > many separate downloads of tiny little xml files mingw-get has to > make, to construct its internal distribution manifest (since we > can't, and shouldn't, offload that process to a sourceforge-hosted > cronjob) Sure. That will be mitigated, to some extent, by downloading only when the user does an explicit "mingw-get update", (or if needed for a first time download); at other times, locally cached copies will be used. Additionally, if I'm interpreting MSDN correctly, wininet will hold its connection on the server, while accessing multiple consecutive URLs addressing a common host. > > Do note that, while SF assure us they will keep files forever, > > it does seem to be possible to *overwrite* an existing file, > > with new content; delete the existing file record from the FRS > > database, then upload a fresh copy with identically the same > > name. > > That's true. (But bloody awkward: ok, I'll upload this 0-byte file > with the same name, and that'll be our little code for "ignore me, > I've been deleted"...) I'm not suggesting we do that, (upload 0-byte files). For the XML distribution manifests, we *want* the file name to remain unchanged, even when the content is updated. At any time, the hosted copy will be the current version; when we do an update, we overwrite the out of date copy with the new version, which then becomes current. > if somebody names their demonstration msys package list > catalog 'msys-base': > ... > And then, wants to specify a meta package with the same name: > ... > They then would NOT be able to move that meta-package > specification into a separate file, because its obvious name, > msys-base.xml, would clash with the <package-list > catalogue="msys-base"> file. > > But nobody would EVER do something like that... Well, hopefully *we* wouldn't, but somebody else just might, (which of course is what you are hinting); but then, *they* would not be hosting their package in *our* repository -- mingw-get will support alternative repositories. I haven't implemented it yet, but I am thinking of hashing the locally cached distribution manifest file names, based on full download URLs, to resolve such potential conflicts. > > I'd rather trust individual > > package maintainers to apply their own updates in CVS > > Yep, that makes the most sense to me. > > > then push the > > result to FRS. > > Manually? Is this were we do that "upload a file with the same > name" thing (e.g. "coreutils.xml")? You'd actually compress it, and upload it as coreutils.xml.lzma, but yes, that is the idea; it may seem a little awkward, but really not any different in effect, from regenerating in place, if SF were to permit that, (which of course, they don't). -- Regards, Keith. |