From: Max T. W. <ma...@mt...> - 2005-07-21 15:18:34
|
Background: I became interested in MinGW and MSys as tools for building my own projects. They came to my attention by way of Dev-C++. I found that I had a lot to learn about the current state of the art and have been studying hard. I wanted to take some time to learn what was going on before I really stuck my foot in my mouth. I'm about to open my mouth and I hope I don't bite my toes! Observation I: MinGW and MSys form a very good but incomplete development environment. It is better than the Cygwin environment largely because it avoids to overhead associated with X11. It is correctably worse than Cygwin because it is missing a number of useful elements found in Cygwin like a documentation maintenance suite and the man-pages suite. Observation II: It is not possible to reconstruct the existing MSys environment from the available sources. There are a number of pieces that have become broken over time as well as a number of pieces that have been improved. A good deal of this can be traced to the fact that there is (at least as far as I have seen) no formal build management system. Pieces get added to the system by being built manually and having the files generated added to the list of files to be included in the distribution. The details of the build process have been lost over time and may no longer be reproducible due to changes (usually improvements) to the over all system. Observation III: Other systems have formal build procedures to avoid this kind of problem. Red Hat uses the RedHat Package Manager (rpm) to avoid this problem. (The published rpm pieces are somewhat less than their complete tool set.) Similarly Cygwin has a number of standards and tools designed to help with this kind of problem. In particular, the auto-tools seem to have been put together to help with this process. However, there is a lot of material in Cygwin that does not use their tools or standards. There are apparently a number of other systems as well including one in Debian (or however you spell it). In other words this is a significant problem and lots of people have worked on it. Reaction: I put together a script to allow me to record the details of how to build various packages. I posted various early versions of that script to this and (I think) the MSys mailing lists. As an experimental prototype it was hooky but usable. As Margaret Mitchell said 'it done growed' like Topsy. I have just finished a rewrite that should clean up some of the problems that crept into the prototype, but it still needs quite a bit of work. Result: The script builds packages. The build process is controlled by build detail files similar in purpose to the .spec files used by rpm. The detail files are line oriented with each line starting with a tag indicating what is being specified. Among other things, dependencies on the existence of certain files or commands can be specified to prevent building a package that is known to require those files or commands. Groups of detail files are collected into package groups. The entire group can be built or selected packages in the group can be built. Question: Is anybody else interested in this? Specifically - Earnie - would you use this if I finish cleaning it up and documenting it? I have a number of really stupid problems - are any of you willing to teach me to solve them? mt...@us... |
From: Christopher F. <me...@cg...> - 2005-07-21 17:14:11
|
On Thu, Jul 21, 2005 at 10:20:53AM -0400, Max T. Woodbury wrote: >Background: > >I became interested in MinGW and MSys as tools for building my own >projects. They came to my attention by way of Dev-C++. I found that I >had a lot to learn about the current state of the art and have been >studying hard. I wanted to take some time to learn what was going on >before I really stuck my foot in my mouth. I'm about to open my mouth >and I hope I don't bite my toes! > >Observation I: > >MinGW and MSys form a very good but incomplete development environment. >It is better than the Cygwin environment largely because it avoids to >overhead associated with X11. It is correctably worse than Cygwin >because it is missing a number of useful elements found in Cygwin like >a documentation maintenance suite and the man-pages suite. "overhead associated with X11"? You don't need to install X11 if you don't want to. Installing X11 isn't the default for Cygwin. >Observation III: > >Other systems have formal build procedures to avoid this kind of >problem. Red Hat uses the RedHat Package Manager (rpm) to avoid this >problem. (The published rpm pieces are somewhat less than their >complete tool set.) Similarly Cygwin has a number of standards and >tools designed to help with this kind of problem. In particular, the >auto-tools seem to have been put together to help with this process. If by auto-tools you mean autoconf and automake then those are supported for mingw, too. >However, there is a lot of material in Cygwin that does not use their >tools or standards. There are apparently a number of other systems as >well including one in Debian (or however you spell it). In other words >this is a significant problem and lots of people have worked on it. Cygwin doesn't have any formal tools. It just has conventions which people may to choose to use or not use. That's by design. We don't force people to do things any specific way. RPMs also differ very widely in terms of conventions used in the spec file. The only constant is that "rpmbuild -ba" works. After that it's the wild west as far as the innards of a package go. cgf |
From: Max T. W. <ma...@mt...> - 2005-07-22 03:04:46
|
Christopher Faylor wrote: > > On Thu, Jul 21, 2005 at 10:20:53AM -0400, Max T. Woodbury wrote: > >Background: > > > >Observation I: > > > >MinGW and MSys form a very good but incomplete development environment. > >It is better than the Cygwin environment largely because it avoids to > >overhead associated with X11. It is correctably worse than Cygwin > >because it is missing a number of useful elements found in Cygwin like > >a documentation maintenance suite and the man-pages suite. > > "overhead associated with X11"? You don't need to install X11 if you > don't want to. Installing X11 isn't the default for Cygwin. This is probably just my personal opinion coloring my thinking. I have worked with X for a long time and find it unpleasant to deal with. There's enough stuff that requires X if you want graphics that it's not all that optional. I'll admit that those pieces are not present in MSys or MinGW so for an equivalent environment it really doesn't make that much difference, but it is all too easy to get carried away with Cygwin. > >Observation III: > > > >Other systems have formal build procedures to avoid this kind of > >problem. Red Hat uses the RedHat Package Manager (rpm) to avoid this > >problem. (The published rpm pieces are somewhat less than their > >complete tool set.) Similarly Cygwin has a number of standards and > >tools designed to help with this kind of problem. In particular, the > >auto-tools seem to have been put together to help with this process. > > If by auto-tools you mean autoconf and automake then those are supported > for mingw, too. Actually, there seems to be a lot of stuff in their makefiles that is intended to facilitate package management that is non-functional in MinGW and MSys. > >However, there is a lot of material in Cygwin that does not use their > >tools or standards. There are apparently a number of other systems as > >well including one in Debian (or however you spell it). In other words > >this is a significant problem and lots of people have worked on it. > > Cygwin doesn't have any formal tools. It just has conventions which > people may to choose to use or not use. That's by design. We don't > force people to do things any specific way. I have to disagree. They have a number of tools and conventions that appear to be aimed at packaging. There is just so much stuff in Cygwin that you can't find anything easily. Of course you don't have to use it. You don't have to use 'mv' either. That's beside the point. I just found it all incredibly messy with a tendency to go from your index finger to your thumb by way of your elbow simply because you need to do that in an old-fashioned multi-user environment. > RPMs also differ very widely in terms of conventions used in the spec > file. The only constant is that "rpmbuild -ba" works. After that it's > the wild west as far as the innards of a package go. And sometimes even "rpmbuild -ba" doesn't work. It's another big, hairy piece of mechanism that gets in your way as much as it helps. The whole point is that information on how the MinGW and MSys systems are put together is getting lost. I've been trying to reproduce it from source and I found it to be much more than just a connect the dots process. More in my response to Lief W. max |
From: Christopher F. <me...@cg...> - 2005-07-22 03:43:59
|
On Thu, Jul 21, 2005 at 11:04:29PM -0400, Max T. Woodbury wrote: >Christopher Faylor wrote: >> >> On Thu, Jul 21, 2005 at 10:20:53AM -0400, Max T. Woodbury wrote: >> >Background: >> > >> >Observation I: >> > >> >MinGW and MSys form a very good but incomplete development environment. >> >It is better than the Cygwin environment largely because it avoids to >> >overhead associated with X11. It is correctably worse than Cygwin >> >because it is missing a number of useful elements found in Cygwin like >> >a documentation maintenance suite and the man-pages suite. >> >> "overhead associated with X11"? You don't need to install X11 if you >> don't want to. Installing X11 isn't the default for Cygwin. > >This is probably just my personal opinion coloring my thinking. I have >worked with X for a long time and find it unpleasant to deal with. There's >enough stuff that requires X if you want graphics that it's not all that >optional. I'll admit that those pieces are not present in MSys or MinGW >so for an equivalent environment it really doesn't make that much >difference, but it is all too easy to get carried away with Cygwin. There isn't much point in arguing here but I think that this is not a very logical observation, IMO. I use X only to debug cygwin problems and I've never been tempted to be "carried away". If you don't want it don't install it. >> >Observation III: >> > >> >Other systems have formal build procedures to avoid this kind of >> >problem. Red Hat uses the RedHat Package Manager (rpm) to avoid this >> >problem. (The published rpm pieces are somewhat less than their >> >complete tool set.) Similarly Cygwin has a number of standards and >> >tools designed to help with this kind of problem. In particular, the >> >auto-tools seem to have been put together to help with this process. >> >> If by auto-tools you mean autoconf and automake then those are supported >> for mingw, too. > >Actually, there seems to be a lot of stuff in their makefiles that is >intended to facilitate package management that is non-functional in >MinGW and MSys. Whose makefiles? I don't see why whatever works for Cygwin wouldn't work for MSYS. They are both emulations of a POSIX environment. >> >However, there is a lot of material in Cygwin that does not use their >> >tools or standards. There are apparently a number of other systems as >> >well including one in Debian (or however you spell it). In other words >> >this is a significant problem and lots of people have worked on it. >> >> Cygwin doesn't have any formal tools. It just has conventions which >> people may to choose to use or not use. That's by design. We don't >> force people to do things any specific way. > >I have to disagree. You might disagree but I'm stating fact, not opinion. I manage the Cygwin project. I know what is going on. I manage twenty packages myself using my own set of packaging tools. Some people use my tools, some (maybe a majority but I don't know for sure) use another set of tools, some use no specific tools at all. We get by with having many cooks maintaining many packages. Most of the packages use autoconf and, so, work on cygwin with minor tweaking. Again, I would suspect that to be true of MSYS, too. >They have a number of tools and conventions that appear to be aimed at >packaging. There is just so much stuff in Cygwin that you can't find >anything easily. Of course you don't have to use it. You don't have >to use 'mv' either. That's beside the point. I just found it all >incredibly messy with a tendency to go from your index finger to your >thumb by way of your elbow simply because you need to do that in an >old-fashioned multi-user environment. I don't really have much of a say here but I'm not particularly swayed by people who use euphemisms rather than give concrete examples. >>RPMs also differ very widely in terms of conventions used in the spec >>file. The only constant is that "rpmbuild -ba" works. After that it's >>the wild west as far as the innards of a package go. > >And sometimes even "rpmbuild -ba" doesn't work. It's another big, >hairy piece of mechanism that gets in your way as much as it helps. And, I assume, that there will be times when whatever tool you're proposing won't work either since it will be software and software always breaks. >The whole point is that information on how the MinGW and MSys systems >are put together is getting lost. I've been trying to reproduce it >from source and I found it to be much more than just a connect the dots >process. I'm sure that is a laudable goal but your message isn't particularly enhanced, IMO, by your examples. I don't know why you actually needed to mention other systems if the only thing you're trying to do is improve the mings/msys build process. I think whether this would be generally useful boils down to a few things: - if you have a nifty new build system would any of the principals use it? (I suspect that they have their own way of doing things that they are perfectly happy with) - would making this available mean more developers on the project (maybe) - would you (or someone) be willing to keep it up to date as new packages are released which require tweaking of the builder software? cgf |
From: Leif W <war...@us...> - 2005-07-21 19:12:14
|
> "Max T. Woodbury" <ma...@mt...> > 2005 July 21 Thursday 10:20 > Observation I: > > MinGW and MSys form a very good but incomplete development > environment. It is better than You'd have to specifically detail what you mean by "complete development environment". The min in MinGW and the M in MSYS indicate that it's intended to be a minimal set of tools, not a full blown development environment. Maybe some people want it to be that and nothing more, so that they have less to worry about for downloads, installation, maintenance and compatibility. That is a valid point and may be hard to persuade people otherwise. Personally I don't see a design reason why it couldn't be a more complete development environment (some extra programs ported, nice package management system, option to fetch docs). (Before you throw the rotten tomatos...) The main logistical reason is probably that there aren't enough qualified man-hours volunteering to do the initial work and also making the long-term committment to maintenance. It's hard enough to maintain what there is now. > the Cygwin environment largely because it avoids to overhead > associated with X11. It is Yeah as someone else said, I don't get the connection. Disc space overhead or GUI performance overhead? Unless you're referring to using an X-specific editor in an emulated X windows environment, I don't get it. I've never built X programs in Windows, but thre might be cases when that is handy. For example if you have a windows-based X client or server that you're building. If I don't really know what to do with something, I won't install it, and if nothing breaks and I can do what I wanted, then I don't need it. > correctably worse than Cygwin because it is missing a number of useful > elements found in > Cygwin like a documentation maintenance suite and the man-pages suite. Something like the man page docs require a few extra programs (man, groff, some db program which I forget, maybe other things), all of which would need to be ported to MinGW or MSYS, while the man pages themselves are not modified. So it would largely blow out SourceForge bandwidth to transfer non-unique or unmodified data if included, so they wouldn't be. The original sources and man pages would be obtained from the upstream providers, only the patches from MinGW project. That might be trickier to automate the retrieval, but the build process should be unaffected. > Observation II: > > It is not possible to reconstruct the existing MSys environment from > the available sources. I was wondering about this a bit too. I see a few programs from MSYS and MSYS-DTK that are not available for download, either in the FRS or the full prdownloads listing. They're available only in the MSYS and MSYS-DTK installers. It's probably documented somewhere and I just never saw where. > There are a number of pieces that have become broken over time as well > as a number of > pieces that have been improved. A good deal of this can be traced to > the fact that there > is (at least as far as I have seen) no formal build management system. > Pieces get added > to the system by being built manually and having the files generated > added to the list of > files to be included in the distribution. The details of the build > process have been lost > over time and may no longer be reproducible due to changes (usually > improvements) to the > over all system. First, let's distinguish between package management and package build systems. I think of these as two separarte tasks, though for convenience's sake the RedHat Linux distribution added both functionalities to rpm, so the line is blurred if you follow that example. In Debian, I don't know if there's a build environment per-se. I think it's all packages. You download a source package and maybe an optional auxilliary build package with some scripts to automate the build using Debian defaults. IMO, some level of package maintenance is the precursor to automating the installation process. But I'm "oldschool", like many here, and sometimes I like simple tarballs and the option to "just get it and build it myself". While it may seem like an inconvenience at first, it also allows a freedom that many would not relinquish. This would explain people's opposition. Though after I've done the installs a few times, it can get tedious to figure out which of all the files is the most recent, so personally if there were an auxilliary tool that kept track of all file dependencies in a separate, self-contained manner, such that no one is required to use it, then I personally would use it, and I think there's be less or no resistance from others regarding it's development and existence, so long as you're the one to maintain it. Each time a new version of something is released, you have to check to make sure there are no new dependencies. For example, on the users list, a user had a problem building something, and it was related to a new requirement. If you compare this to a full-blown integrated package management system like RPM, well, I have a lot of unpleasent memories from my days with RPM, due to the dependency problems, bugs in the spec files, configuring the code before being thrown into a build, and so on. Being forced to use a package management system is not fun. Well, as far as generating an installer for MinGW, MSYS, or MSYS-DTK, it would be nifty if one could run a command, have it retrieve the MinGW files and list what you need to get externally (not necessarily a URL, but at the the common name of the application or library and possibly the last known URL). The MinGW net installer is in it's initial stages and there's probably going to be some flux. Probably building off such an existing tool would be the best way to spend energy. It's really a moving target right now. That could be interpreted as an opportunity to try to make what you want of it. ;-) It currently uses Inno Setup, but there was some discussion about the download functions, and NSIS (NullSoft Installation System) was thrown out as a possible alternative. Earnie pinged the status a few days ago but I saw no pong. > Observation III: > > Other systems have formal build procedures to avoid this kind of > problem. Red Hat uses > the RedHat Package Manager (rpm) to avoid this problem. (The > published rpm pieces are > somewhat less than their complete tool set.) Similarly Cygwin has a > number of standards > and tools designed to help with this kind of problem. In particular, > the auto-tools > seem to have been put together to help with this process. However, > there is a lot of > material in Cygwin that does not use their tools or standards. There > are apparently a > number of other systems as well including one in Debian (or however > you spell it). > In other words this is a significant problem and lots of people have > worked on it. It would be handy to also have some script which allowed for automatic building of the build environment. At the very least, I'd like to be able to build MinGW, MSYS and MSYS-DTK themselves from sources in an automated way, possibly even to wrap in my own all-in-one offline installer. I would think that the build part is reasonable, but others may not. The installer part is probably left as an exercise for the motivated. > Reaction: > > I put together a script to allow me to record the details of how to > build various > packages. I posted various early versions of that script to this and > (I think) the > MSys mailing lists. As an experimental prototype it was hooky but > usable. As Hmm, I think SourceForge doesn't archive file attachments to mailing lists, and I probably missed that first posting or simply had no idea what it was about at that point in my MinGW/MSYS experience. > Result: > > The script builds packages. The build process is controlled by build > detail files > similar in purpose to the .spec files used by rpm. The detail files > are line oriented > with each line starting with a tag indicating what is being specified. > Among other > things, dependencies on the existence of certain files or commands can > be specified > to prevent building a package that is known to require those files or > commands. Groups > of detail files are collected into package groups. The entire group > can be built or > selected packages in the group can be built. > > Question: > > Is anybody else interested in this? I think it sounds interesting. > Specifically - Earnie - would you use this if I finish cleaning it up > and documenting it? I'd at least like to get hold of it and see if I could make any sense of it without the docs. The docs are good in case I can't figure it out by using it. :p > I have a number of really stupid problems - are any of you willing to > teach me to solve > them? I'm probably not experienced enough to answer specific questions. Teaching oneself to solve unknown problems is a valuable skill to develop. ;-) Anyways, sometimes it's quicker to solve by asking. But it's funny to see the "Can I ask a question?" sort of question. The wiseguy answers "You just did." Most say "Just ask, don't ask to ask.". Well so long as it's a few discreet questions at a time. Leif |
From: Keith M. <kei...@nt...> - 2005-07-23 21:25:52
|
On Thursday 21 July 2005 8:11 pm, Leif W wrote: > > correctably worse than Cygwin because it is missing a number of useful > > elements found in > > Cygwin like a documentation maintenance suite and the man-pages suite. > > Something like the man page docs require a few extra programs (man, > groff, some db program which I forget, maybe other things), all of which > would need to be ported to MinGW or MSYS ... Manpages require only the `man' program, `nroff' (ideally in the guise of `groff'), and of course, the `nroff' sources of the manpages themselves. I've been a contributor to `groff' for about two years now, and have helped to ensure that v1.19.x build OOTB with MinGW, with no need to patch the official GNU sources. (If you want HTML or PDF output, you do need `ghostscript', and for HTML only, you also need `netpbm', `psutils' and some DLLs that these depend on -- I use AFPL Ghostscript, and the rest from the GnuWin32 project; none of these are required for simply viewing manpages, or even for basic typesetting and printing tasks, however). The `man' program is another matter, however; the package bundled in most, if not all GNU/Linux distributions is NOT officially supported as a GNU package, and until very recently, when it got a new maintainer, the guy who was supporting it had no interest in providing ANY level of Win32 compatibility. I have been working toward a MinGW port, but it isn't yet ready for public consumption; can't say when it will be, as I have limited time available to work on it :( At least the new maintainer seems more willing than the old, to consider merging Win32 specific patches into the mainline package, and my ultimate goal is to help him to have that also build OOTB, with MinGW. > ... while the man pages themselves > are not modified. So it would largely blow out SourceForge bandwidth to > transfer non-unique or unmodified data if included, so they wouldn't be. > The original sources and man pages would be obtained from the upstream > providers, only the patches from MinGW project. That might be trickier > to automate the retrieval, but the build process should be unaffected. The manpage sources for many of the MinGW and MSYS packages SHOULD be available, within the respective SOURCE packages provided on the SourceForge download site, and mirrors. Best regards, Keith. |
From: Max T. W. <ma...@mt...> - 2005-07-22 05:41:36
|
Leif W wrote: > > > "Max T. Woodbury" <ma...@mt...> > > 2005 July 21 Thursday 10:20 > > > Observation I: > > > > MinGW and MSys form a very good but incomplete development > > environment. It is better than > > You'd have to specifically detail what you mean by "complete development > environment". The min in MinGW and the M in MSYS indicate that it's > intended to be a minimal set of tools, not a full blown development > environment. Maybe some people want it to be that and nothing more, so > that they have less to worry about for downloads, installation, > maintenance and compatibility. That is a valid point and may be hard to > persuade people otherwise. Personally I don't see a design reason why > it couldn't be a more complete development environment (some extra > programs ported, nice package management system, option to fetch docs). > (Before you throw the rotten tomatos...) The main logistical reason is > probably that there aren't enough qualified man-hours volunteering to do > the initial work and also making the long-term committment to > maintenance. It's hard enough to maintain what there is now. Yep. Tools that can make maintenance easier would help. As for 'minimal' - I'm inclined to go for a functional definition - can I get hold of the original pieces and rebuild it myself. In that sense, MinGW and MSys are already past the 'minimal' bound. > > the Cygwin environment largely because it avoids to overhead > > associated with X11. It is > > Yeah as someone else said, I don't get the connection. Disc space > overhead or GUI performance overhead? Unless you're referring to using > an X-specific editor in an emulated X windows environment, I don't get > it. I've never built X programs in Windows, but thre might be cases > when that is handy. For example if you have a windows-based X client or > server that you're building. If I don't really know what to do with > something, I won't install it, and if nothing breaks and I can do what I > wanted, then I don't need it. The point is that X is a mess to deal with and Cygwin has gotten so big it is impossible to tell what you need and what you don't. > > correctably worse than Cygwin because it is missing a number of useful > > elements found in Cygwin like a documentation maintenance suite and > > the man-pages suite. > > Something like the man page docs require a few extra programs (man, > groff, some db program which I forget, maybe other things), all of which > would need to be ported to MinGW or MSYS, while the man pages themselves > are not modified. So it would largely blow out SourceForge bandwidth to > transfer non-unique or unmodified data if included, so they wouldn't be. > The original sources and man pages would be obtained from the upstream > providers, only the patches from MinGW project. That might be trickier > to automate the retrieval, but the build process should be unaffected. Yep. And that's not what is happening. Things are actually getting duplicated rather than deltad. Figuring out how to do that is one of the things I think is needed most, but nobody does it now. If it can be made to work, it could save a fair amount of resources, including maintainers' time. I'd say more here, but I think it will fit better further down... Oh, and I took a quick look at MiKTeX a couple hours ago. Like most TeX suites, it seems to be huge. It looks like a lot of work actually putting all the pieces together. This is one of those 'tar-baby' things if I'm not mistaken. > > Observation II: > > > > It is not possible to reconstruct the existing MSys environment from > > the available sources. > > I was wondering about this a bit too. I see a few programs from MSYS > and MSYS-DTK that are not available for download, either in the FRS or > the full prdownloads listing. They're available only in the MSYS and > MSYS-DTK installers. It's probably documented somewhere and I just > never saw where. Maybe. Maybe not. It should be obvious where everything came from and it isn't now. 'mingwPORT' seems to be the right way to go, but I'm still trying to understand it. There's some stuff in the GNU bootstrap process as well. A fair amount of this stuff seems to be out of Cygwin. I mentioned above that this should be deltad not duplicated. At the moment I'm not doing this very well myself. While the first part of the process I set up used CVS, some of it still works on copies of .tar files. The key seems to be 'wget' which I only got to build a day or two ago and I have not yet learned what I can do with it. I need to work out the best way to use it. That probably includes a local cache and automatically applying and creating deltas (like the more compliant Cygwin packages and mingwPORT seems to do). > > There are a number of pieces that have become broken over time as well > > as a number of pieces that have been improved. A good deal of this > > can be traced to the fact that there is (at least as far as I have > > seen) no formal build management system. Pieces get added to the > > system by being built manually and having the files generated > > added to the list of files to be included in the distribution. The > > details of the build process have been lost over time and may no > > longer be reproducible due to changes (usually improvements) to the > > over all system. > > First, let's distinguish between package management and package build > systems. I think of these as two separarte tasks, though for > convenience's sake the RedHat Linux distribution added both > functionalities to rpm, so the line is blurred if you follow that > example. In Debian, I don't know if there's a build environment per-se. > I think it's all packages. You download a source package and maybe an > optional auxilliary build package with some scripts to automate the > build using Debian defaults. IMO, some level of package maintenance is > the precursor to automating the installation process. I think there are actually three pieces, not just two. Earnie's working on the package management piece. RedHat uses 'anaconda'. They only give away the back-end piece. There's apparently some internal tools that they don't talk about for managing the database anaconda uses. Some of it is in 'yum'. I know just enough about that to say that it is a non-trivial problem. It should, if it's done right, mesh with the other two pieces. The second piece is the package installation piece. That's one of the main functions of 'rpm'. INNO also seems to fit into this category. Again, Earnie seems to have this staked out. I also trust that he will ask for help if he needs it and that I should stay out of his way. The third piece is the build piece. As Christopher mentioned, it's also part of 'rpm'. It has a dependency problem much like the package installation dependency problem. That makes it seem to be part of the installation problem, but the build dependencies can be quite a bit different from the installation dependencies. > But I'm "oldschool", like many here, and sometimes I like simple > tarballs and the option to "just get it and build it myself". While it > may seem like an inconvenience at first, it also allows a freedom that > many would not relinquish. This would explain people's opposition. > Though after I've done the installs a few times, it can get tedious to > figure out which of all the files is the most recent, so personally if > there were an auxilliary tool that kept track of all file dependencies > in a separate, self-contained manner, such that no one is required to > use it, then I personally would use it, and I think there's be less or > no resistance from others regarding it's development and existence, so > long as you're the one to maintain it. Each time a new version of > something is released, you have to check to make sure there are no new > dependencies. For example, on the users list, a user had a problem > building something, and it was related to a new requirement. Well, I'm looking at the problem at least. There may be more than a few pieces missing from the MinGW and MSys kit in this regard. Some of the auto-tools seem to be pointed at this issue. They apparently do a less than adequate job from the cussing Christopher has directed at them in the past. I can say with some certainty that there will be a cost associated with the tool - at the minimum you'll have to run it and it will almost certainly produce less than perfect results - which you'll have to fix, either by changing your project structure to make the tool work better, or by adding what is needed after it's done what it can. > If you compare this to a full-blown integrated package management system > like RPM, well, I have a lot of unpleasent memories from my days with > RPM, due to the dependency problems, bugs in the spec files, configuring > the code before being thrown into a build, and so on. Being forced to > use a package management system is not fun. You got the not fun part right but RPM is not a complete system. One of the things it assumes, which it shouldn't, is that there's a tarball at the beginning of the process. It's also a bit NIH about the dependency problem. At least part of its problem, as I see it, is that the 'spec' file is designed around the build procedure. It ought to have a more database like quality to it. In any case a fresh start is indicated. > Well, as far as generating an installer for MinGW, MSYS, or MSYS-DTK, it > would be nifty if one could run a command, have it retrieve the MinGW > files and list what you need to get externally (not necessarily a URL, > but at the the common name of the application or library and possibly > the last known URL). The MinGW net installer is in it's initial stages > and there's probably going to be some flux. Probably building off such > an existing tool would be the best way to spend energy. It's really a > moving target right now. That could be interpreted as an opportunity to > try to make what you want of it. ;-) It currently uses Inno Setup, but > there was some discussion about the download functions, and NSIS > (NullSoft Installation System) was thrown out as a possible alternative. > Earnie pinged the status a few days ago but I saw no pong. I picked up the INNO kit a while ago but haven't installed it yet. From what little I've seen of it, its oriented on the installation rather than the build process. Since I'm working the build end still, I simply have not needed it. > > Observation III: > > > > Other systems have formal build procedures to avoid this kind of > > problem. Red Hat uses the RedHat Package Manager (rpm) to avoid this > > problem. (The published rpm pieces are somewhat less than their > > complete tool set.) Similarly Cygwin has a number of standards > > and tools designed to help with this kind of problem. In particular, > > the auto-tools seem to have been put together to help with this process. > > However, there is a lot of material in Cygwin that does not use their > > tools or standards. There are apparently a number of other systems as > > well including one in Debian (or however you spell it). > > In other words this is a significant problem and lots of people have > > worked on it. > > It would be handy to also have some script which allowed for automatic > building of the build environment. At the very least, I'd like to be > able to build MinGW, MSYS and MSYS-DTK themselves from sources in an > automated way, possibly even to wrap in my own all-in-one offline > installer. I would think that the build part is reasonable, but others > may not. The installer part is probably left as an exercise for the > motivated. I've got it working to the point where I can turn it loose on MSYS or msysDTK, go to bed and find the logs and other results ready the next morning. I haven't done MinGW or the mingwPORTs yet, at least partially because I needed to build 'wget'. I should also be able to build all the GCC stuff if it's working properly, but I have yet to write that set of control files. > > Reaction: > > > > I put together a script to allow me to record the details of how to > > build various packages. I posted various early versions of that > > script to this and (I think) the MSys mailing lists. As an > > experimental prototype it was hooky but > > usable. As > > Hmm, I think SourceForge doesn't archive file attachments to mailing > lists, and I probably missed that first posting or simply had no idea > what it was about at that point in my MinGW/MSYS experience. > > > Result: > > > > The script builds packages. The build process is controlled by build > > detail files similar in purpose to the .spec files used by rpm. The > > detail files are line oriented with each line starting with a tag > > indicating what is being specified. Among other things, dependencies > > on the existence of certain files or commands can be specified > > to prevent building a package that is known to require those files or > > commands. Groups of detail files are collected into package groups. > > The entire group can be built or selected packages in the group can be > > built. > > > > Question: > > > > Is anybody else interested in this? > > I think it sounds interesting. OK. It's close to alpha state. I need to put a couple things in the header before sending it out... From your response I take it that I can send you a copy when it's ready? > > Specifically - Earnie - would you use this if I finish cleaning it up > > and documenting it? > > I'd at least like to get hold of it and see if I could make any sense of > it without the docs. The docs are good in case I can't figure it out by > using it. :p At present it's a single file project - everything, including the docs are in the one file, but a copy of a set of control files would probably make the whole thing more understandable... > > I have a number of really stupid problems - are any of you willing to > > teach me to solve them? > > I'm probably not experienced enough to answer specific questions. > Teaching oneself to solve unknown problems is a valuable skill to > develop. ;-) Anyways, sometimes it's quicker to solve by asking. But > it's funny to see the "Can I ask a question?" sort of question. The > wiseguy answers "You just did." Most say "Just ask, don't ask to ask.". > Well so long as it's a few discreet questions at a time. Actually I'm quite good at solving unknown problems, but I took a rather large hit to my self-esteem not too long ago and I'm still trying to get back on my feet so-to-speak, but that's not quite what I intended by that question. I know there are people out there who know more about some things than I do, so I will be asking questions. The problem is that people will not answer them if I don't produce something useful with the answers. The implicit question is really 'if I ask stupid questions, is what I'm doing useful enough that you're likely to answer them in spite of their stupidity?' but that's way too indirect and verbose. I do run off at the mouth when I'm anxious... max |
From: Max T. W. <ma...@mt...> - 2005-07-22 07:02:44
|
Christopher Faylor wrote: >On Thu, Jul 21, 2005 at 11:04:29PM -0400, Max T. Woodbury wrote: >>Christopher Faylor wrote: >>>On Thu, Jul 21, 2005 at 10:20:53AM -0400, Max T. Woodbury wrote: I'm trimming text but leaving the back links for context... >>This is probably just my personal opinion coloring my thinking. I have >>worked with X for a long time and find it unpleasant to deal with. There's >>enough stuff that requires X if you want graphics that it's not all that >>optional. I'll admit that those pieces are not present in MSys or MinGW >>so for an equivalent environment it really doesn't make that much >>difference, but it is all too easy to get carried away with Cygwin. > > There isn't much point in arguing here but I think that this is not a > very logical observation, IMO. I use X only to debug cygwin problems > and I've never been tempted to be "carried away". If you don't want it > don't install it. I know what you're saying, but I know from personal experience that there are a lot of things I 'want' but that I will not use with any frequency. I've seen other people do the same thing. And, no it's an observation, not logic. >> Actually, there seems to be a lot of stuff in their makefiles that is >> intended to facilitate package management that is non-functional in >> MinGW and MSys. > > Whose makefiles? I don't see why whatever works for Cygwin wouldn't > work for MSYS. They are both emulations of a POSIX environment. Have you looked at the makefile in 'winsup/cygwin'? There are targets in there for packages that are not in MSys. There's also something about a 'source vault' and building distributions, none of which seemed to be functional in the MSys environment. At any rate, they are not documented well enough for me to want to try them. >>>> However, there is a lot of material in Cygwin that does not use their >>>> tools or standards. There are apparently a number of other systems as >>>> well including one in Debian (or however you spell it). In other words >>>> this is a significant problem and lots of people have worked on it. >>> >>> Cygwin doesn't have any formal tools. It just has conventions which >>> people may to choose to use or not use. That's by design. We don't >>> force people to do things any specific way. >> >> I have to disagree. > > You might disagree but I'm stating fact, not opinion. I manage the > Cygwin project. I know what is going on. I manage twenty packages > myself using my own set of packaging tools. Some people use my tools, > some (maybe a majority but I don't know for sure) use another set of > tools, some use no specific tools at all. I think you misunderstand. Your tools form a set that work with each other. I think that I know which packages use your tools and they are consistently of good quality. It's the ones that have just been thrown together that bother me. > We get by with having many cooks maintaining many packages. Most of > the packages use autoconf and, so, work on cygwin with minor tweaking. > Again, I would suspect that to be true of MSYS, too. I have found that many of the Cygwin kits build under MSys with only a change to config.sub. There are also a significant number that take a significant amount of fiddling to get them to build. >> They have a number of tools and conventions that appear to be aimed at >> packaging. There is just so much stuff in Cygwin that you can't find >> anything easily. Of course you don't have to use it. You don't have >> to use 'mv' either. That's beside the point. I just found it all >> incredibly messy with a tendency to go from your index finger to your >> thumb by way of your elbow simply because you need to do that in an >> old-fashioned multi-user environment. > > I don't really have much of a say here but I'm not particularly swayed > by people who use euphemisms rather than give concrete examples. flex seems to a fairly bad example. There were a couple of others that I tried and had to get from other sources. Understand that I consider Cygwin a fairly good system but I found it much more difficult to use than MSys. (Hey, at least you understand that while I'm upset about the mess, I'm trying to be polite.) >>> RPMs also differ very widely in terms of conventions used in the spec >>> file. The only constant is that "rpmbuild -ba" works. After that it's >>> the wild west as far as the innards of a package go. >> >> And sometimes even "rpmbuild -ba" doesn't work. It's another big, >> hairy piece of mechanism that gets in your way as much as it helps. > > And, I assume, that there will be times when whatever tool you're > proposing won't work either since it will be software and software > always breaks. I'm not talking about breakage. It's possible to use just the packaging piece of RPM without doing the whole build shtick. I've been through some of the RPM code. IIRC I've even submitted a bug report on a specific problem I found. >> The whole point is that information on how the MinGW and MSys systems >> are put together is getting lost. I've been trying to reproduce it >> from source and I found it to be much more than just a connect the dots >> process. > > I'm sure that is a laudable goal but your message isn't particularly > enhanced, IMO, by your examples. I don't know why you actually needed > to mention other systems if the only thing you're trying to do is > improve the mings/msys build process. There are methods and concepts embodied in other systems that should be evaluated and possibly applied to the build problem. Further, if it isn't too badly done, it should be adaptable to other environments. > I think whether this would be generally useful boils down to a few things: > > - if you have a nifty new build system would any of the principals > use it? (I suspect that they have their own way of doing things that > they are perfectly happy with) Strike the 'perfectly' and you're very likely to be correct. More importantly, will they contribute ideas? > - would making this available mean more developers on the project > (maybe) > > - would you (or someone) be willing to keep it up to date as new packages > are released which require tweaking of the builder software? Why do you thing I started this discussion? I will be using it myself so I'll be maintaining it as long as I use it. I just spent four solid days doing a rewrite to make it maintainable. I've about a page and a fraction of work list items and about ten pages of documentation to write. I'd like to know if I should make this a front burner item or if it should get move to a secondary status behind actually getting it to build stuff? max |
From: Earnie B. <ea...@us...> - 2005-07-22 13:56:59
|
On 7:02:39 am 2005-07-22 "Max T. Woodbury" <ma...@mt...> wrote: > Christopher Faylor wrote: > > > > - would you (or someone) be willing to keep it up to date as new > > packages are released which require tweaking of the builder > software? > > Why do you thing I started this discussion? I will be using it myself > so I'll be maintaining it as long as I use it. I just spent four > solid days doing a rewrite to make it maintainable. I've about a > page and a fraction of work list items and about ten pages of > documentation to write. I'd like to know if I should make this a > front burner item or if it should get move to a secondary status > behind actually getting it to build stuff? > I've begun the mingwPORT project in order to help: 1) provide a common means to install software bundles on win32 2) allow others to share their port without the formality need. Max, I would be interested in looking at your script but only for the purpose of how some or all of what you have may be incorporated into the portmaker source tree (view CVS). Beyond that, development of a RPM or apt-get style installer I don't believe fits us. I may be interested in something similar to the freeBSD and openBSD ports project install methods. Note that the context of my statements above are related to MinGW and not MSYS. The MSYS build should never have a public interface beyond what is generated by configure and executed by make. The extra targets you see are a result of what is known as a Cygnus build environment and are historical for Cygnus (now Red Hat) builds. It was designed to be a top layer system so that if you wanted to build say binutils and GCC in one session you drop their source in to the top directory, issue configure in your build directory and it would traverse into the mulitple packages executing configure. There has been no reason for me to change that and I don't plan to spend my time reviewing changes by others to do the same either. As for as the packages used by MSYS, I am trying, as I find time, to ensure that you can ./configure and make each of them and provide documentation if that isn't able to be accomplished. As for the MSYS installer, that too is automated but only in the realms how I have my environment set. I will give instruction on how to set your environment like mine so that you too can create the install deliverable. Earnie |
From: Max T. W. <ma...@mt...> - 2005-07-22 17:39:23
|
Earnie Boyd wrote: > On 7:02:39 am 2005-07-22 "Max T. Woodbury" <ma...@mt...> wrote: > > Christopher Faylor wrote: > > > > > > - would you (or someone) be willing to keep it up to date as new > > > packages are released which require tweaking of the builder > > > software? > > > > Why do you thing I started this discussion? I will be using it myself > > so I'll be maintaining it as long as I use it. I just spent four > > solid days doing a rewrite to make it maintainable. I've about a > > page and a fraction of work list items and about ten pages of > > documentation to write. I'd like to know if I should make this a > > front burner item or if it should get move to a secondary status > > behind actually getting it to build stuff? > > I've begun the mingwPORT project in order to help: > > 1) provide a common means to install software bundles on win32 > 2) allow others to share their port without the formality need. > > Max, I would be interested in looking at your script but only for the > purpose of how some or all of what you have may be incorporated into the > portmaker source tree (view CVS). Beyond that, development of a RPM or > apt-get style installer I don't believe fits us. I may be interested in > something similar to the freeBSD and openBSD ports project install methods. There is a bit of confusion here. It is NOT an installer. It is not intended to replace the entire RPM function. It is intended to be a way to record such information as build dependencies and configure options as much as it is a way to perform a whole series of builds without having to start each one by hand. > Note that the context of my statements above are related to MinGW and not > MSYS. The MSYS build should never have a public interface beyond what is > generated by configure and executed by make. The extra targets you see are > a result of what is known as a Cygnus build environment and are historical > for Cygnus (now Red Hat) builds. It was designed to be a top layer system > so that if you wanted to build say binutils and GCC in one session you drop > their source in to the top directory, issue configure in your build > directory and it would traverse into the mulitple packages executing > configure. There has been no reason for me to change that and I don't plan > to spend my time reviewing changes by others to do the same either. Would you entertain patches that removed the now superfluous targets? > As for as the packages used by MSYS, I am trying, as I find time, to ensure > that you can ./configure and make each of them and provide documentation if > that isn't able to be accomplished. I believe I understand and I think I have helped a little. The script and the detail files should be sufficient documentation on what is needed to build any MSYS or MINGW package. Also './configure' implies a writeable source directory tree. I've been trying to set things up so the sources could be on a CD and still have the script succeed. > As for the MSYS installer, that too is automated but only in the realms how > I have my environment set. I will give instruction on how to set your > environment like mine so that you too can create the install deliverable. Again, the script is intended to be a builder, NOT an installer. In fact it tries very hard NOT to really install what it builds so that it can be used to build testable versions of the package without messing up the working environment. On the other hand it is fairly easy to actually install the result - 'cd <pseudo-install-dir>; cp -pr * /' or 'cd <pseudo-install-dir>; cp -pr * /mingw/' is all that is needed as a follow up to running the script to do the actual installation. (If you feel like living dangerously, you can even put that step into the detail file as part of the cleanup specification.) max |