Courtesy post of a message sent to the Debian NCO maintainer:
Hi Brian,
I've taken a first step in Debian'izing the NCO upstream source.
Your files from the nco-2.2.0 deb package are now in the upstream
source. This includes all files in nco/debian as well as
nco/bld/linux1 and nco/bld/linux2.
There have been many significant changes to the Makefile since the NCO
library was restructured, so I have not tried to port any improvements
you made to the Debian bld/Makefile except to insert the LINUXARM
architecture into the LINUX* rules.
I wanted to warn you that the upstream source (in CVS, not released
yet) now has a debian directory.
I hope this does not create difficulties for you!
My plan is to try to make the upstream source Debian
compatible by about version 2.8.0 (summer?).
At that point, I envision two distinct build mechanisms:
1. nco/bld/Makefile
This is what I will continue to use for NCO development.
It works by default on all platforms I have access to and is
very useful for porting to new platforms and ones on which
I do not have root priveleges and so which may lack the GNU
toolchain. It is currently the only way to build certain
NCO configurations (e.g., AIX, OSF1, NEC, CRAY, DODS...).
I do not plan to merge your Debian-specific Makefile changes
into the generic portion of bld/Makefile, but am happy to
merge any generic stuff that is useful.
Also, if you wanted to make Debian-specific changes activate
with a keyword, e.g., DBN, then we could certainly include
Debian-specific flags/rule/target modification under that
mechanism. If you are interested in modifying the upstream
source yourself I will add you to the developers list on
sourceforge. Let me know. Otherwise I will modify your debian/* files
until they are synchronized with the current NCO and the packages
build fine on my home debian machine.
2. Generic ./configure mechanism
Rorik Peterson added a generic ./configure mechanism to NCO.
It works great on Linux but has problems on most other machines.
I could not use this in maintainer mode until I switched to
Debian (NCO is the main reason I switched from RedHat to Debian).
I think the default Debian build mechanism should be migrated into
the ./configure mechanism. This will allow Debian users to benefit
from the shared libraries etc. that autoconf/libtool/automake
support. However, I want the ./configure mechanism to continue to
work for non-Debian systems. Since Debian builds have to adhere
to the Debian policy guidelines, I can imagine that modifying
the current ./configure mechanism to fit those guidelines would
result in a ./configure mechanism that does not work for non-Debian
users (e.g., users who want to install into /usr/local by default).
I really do not know enought about autotools or Debian guidelines
to know if this will be a problem, but I'm assuming that any
problems can be addressed by adding a --enable-debian to configure.in
if necessary.
Anyway, that's the plan. The ulimate goal is to be able to upload
tagged versions of NCO (e.g., 2.8.1) to debian unstable with
a single command. Please let me know if you have any suggestions on
how to make this transition go smoothly.
Thanks,
Charlie
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I made a few somewhat major changes to the debian/ directory. What are your opinions on these?
* several new files were added that various debhelper scripts use, and make maintainance/modification fairly painless. Most are described below.
* 'rules' has been changes substantially. It builds using configure since that seems to work fine on Linux machines. I used the default skeleton file and modified it somewhat, so many of the debhelper programs are used now. For example, the dependencies are determined automatically by dh_shlibdeps and dh_makeshlibs (and turn out to only be libnetcdf3g and libc6), and info files are installed/ uninstalled by default.
* I made a second package called nco-doc, which includes the documentation in html and ps. We could add/change that to pdf, dvi, or whatever. The html is a single page, it could be broken down into chapters (see doc/Makefile.am). All the documentation could also simply be included in the main nco package (installed in /usr/share/doc/nco/doc/ instead of /usr/share/nco-doc/). However, the info and man pages already get installed with the main nco package, and the info pages contain the same information, so I'm ambivalent
* the documentation will be registered with doc-base.
* There are no dependencies for nco-doc, but we could recommend a postscript viewer. I think dpkg requires these and apt-get ignores them, but I'm not sure.
* The doc/NEWS file is terribly out of date, so I didn't include it in the nco package although it was there in the last release (2.2.0). I put doc/ChangeLog in there. See debian/docs for modifications and additions to this list. doc/nco.shtml was in the last debian distribution.
* We could add the test script bld/nco_tst.sh to a nco-doc/examples directory (?)
* when making the .deb file, ther version number is determined automatically from the main directory name (i.e. nco-2.7.2/) This seems like the easiest way to make it, particularly since 'make dist' creates nco-2.7.2.tar.gz automatically.
* since documentation and info files are handled automatically with the various debhelper scripts, I think 'prerm' and 'postinst' are not needed anymore.
I added '--disable-nco_cplusplus" to the configure options so the c++ library can be conditionally built and installed since there is no reason to inlcude it in the .deb file for now.
* the UDUnits will not be included since there is not a debian version of the library and the dependency could not be met. I'm not sure if Brian would want to maintain that aditional package on or not. I'm not a Debian maintainer.
I created nco_2.7.2-1_i386.deb on my machine and it installed fine.
rorik
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks! All your changes sound like improvements
but I still want to examine the specific modifications.
I will try to do so until this weekend.
I also had a working .deb building before your
commits so I may merge some changes I made.
Specific comments inserted below.
Charlie
SourceForge.net wrote:
> * several new files were added that various debhelper scripts use, and make
> maintainance/modification fairly painless. Most are described below.
Good, we want NCO to use the relevant debhelper scripts.
> * 'rules' has been changes substantially. It builds using configure since that
> seems to work fine on Linux machines. I used the default skeleton file and
> modified it somewhat, so many of the debhelper programs are used now. For example,
> the dependencies are determined automatically by dh_shlibdeps and dh_makeshlibs
> (and turn out to only be libnetcdf3g and libc6), and info files are installed/
> uninstalled by default.
Good, I don't expect to build debs on machines where the configure
mechanism does not work for NCO.
> * I made a second package called nco-doc, which includes the documentation in
> html and ps. We could add/change that to pdf, dvi, or whatever. The html is
> a single page, it could be broken down into chapters (see doc/Makefile.am).
> All the documentation could also simply be included in the main nco package
> (installed in /usr/share/doc/nco/doc/ instead of /usr/share/nco-doc/). However,
> the info and man pages already get installed with the main nco package, and
> the info pages contain the same information, so I'm ambivalent
Do you mean that you can install nco but not nco-doc?
I suppose this is a good idea, but what do other packages do?
Make the docs recommended?
I just want the PDF, DVI, Postscript, .txt and/or XML available in a deb somehow.
The man pages, by the way, are completely out of date since they are
based on NCO 2.2 and there is no automated way (that I know of) of
generating them from the TeXInfo source.
> * the documentation will be registered with doc-base.
Great.
> * There are no dependencies for nco-doc, but we could recommend a postscript
> viewer. I think dpkg requires these and apt-get ignores them, but I'm not sure.
I would leave viewers as recommended rather than dependencies.
The .txt format does not require a viewer.
> * The doc/NEWS file is terribly out of date, so I didn't include it in the nco
> package although it was there in the last release (2.2.0). I put doc/ChangeLog
> in there. See debian/docs for modifications and additions to this list.
> doc/nco.shtml was in the last debian distribution.
> * We could add the test script bld/nco_tst.sh to a nco-doc/examples directory
> (?)
It is a good example but it's also part of the build mechanism,
so does Debian policy dictate where to put it?
> * when making the .deb file, ther version number is determined automatically
> from the main directory name (i.e. nco-2.7.2/) This seems like the easiest
> way to make it, particularly since 'make dist' creates nco-2.7.2.tar.gz
> automatically.
I would prefer it if version numbers came from doc/VERSION.
> * since documentation and info files are handled automatically with the various
> debhelper scripts, I think 'prerm' and 'postinst' are not needed anymore.
I have no idea, and will try to think about prem and postinst this weekend.
> I added '--disable-nco_cplusplus" to the configure options so the c++ library
> can be conditionally built and installed since there is no reason to inlcude
> it in the .deb file for now.
Fine.
> * the UDUnits will not be included since there is not a debian version of the
> library and the dependency could not be met. I'm not sure if Brian would want
> to maintain that aditional package on or not. I'm not a Debian maintainer.
We should debianize UDUnits.
Let me take a crack at that this weekend because I want to become
a Debian maintainer in order to keep NCO snapshots up-to-date on
Sid. Udunits is a perfect first project for me to run by Brian Mays.
> I created nco_2.7.2-1_i386.deb on my machine and it installed fine.
I will try to build a 2.7.4 deb this weekend and if it works we can
release that.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
One more thing....the library versioning decision
rekindles a problem we never solved originally, namely, how/where do we specify the NCO version?
I would like to specify it only in one place for all the
source code. I prefer to use doc/VERSION. Perhaps
there is a better way. Is there a way to make macros etc. so that whatever is in doc/VERSION gets used for the library name? Or must we somehow manually and separately specify the
library version number (which is bound to fail
due to human error eventually)?
Thanks,
Charlie
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Concerning man pages, Debian policy (13.1) requires man pages. If they are not provided upstream, the maintainer either provides them or keeps an open bug report about it. If we don't want to maintain the man pages, we could either:
(1) simply add a warning that they might be out of date and the info file should be referenced for accurate documentation, or
(2) simply them all so they don't get out of date and again reference the info pages.
I don't think there is a reasonable way to convert texinfo to man pages with the structure of NCO, for example, there are multiple executeables.
Concerning nco-doc, yes you could install nco without nco-doc, or viseversa, although why someone would want the documenatation and not the program, I'm not sure. It seems to me that most large programs with lots of documentation, break it into documentation packages, see octave for example. I think it might also help with doc-base, but I'm not positive.
Concerning version numbers, if the source directory is not named program-version, dpkg-buildpackage will retrieve it from the last entry in debian/changelog. I'm not sure if you can force dpkg-buildpackage to directly use doc/VERSION. Now, 'make dist' does directly use doc/VERSION. However, if you're building out of a CVS snapshot, and not something generated from 'make dist', everything will be in the package-version.orig.tag.gz file. For NCO, that means the c++ stuff will be there although it is not part of the debian package, and any other files laying around in CVS. I'm not saying one is right or wrong, the maintainer should just decide what they want to do. I did add the bld/ directory to 'make dist'.
Concerning nco_tst.sh, you said this is part of the build mechanism. I don't see how, it is not _required_ for either bld/make or configure;make to work. Therefore, I don't think there is a debian policy concerning it. It really seems like a test to me.
Concerning UDUnits, we could statically link it in the nco binaries until such time as it is available in debian; it increases the library size less than 1%, and is licensed GPL.
rorik
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Charlie,
What is the status of updating the version of NCO in Debian? Did you ever hear from Brian? The NetCDF library really needs to be updated on Debian as well, since it won't link with gcc/g++ versions < 3.0. I can try and help if you want.
rorik
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks for reminding me about Debianization.
I did not realize that the current netCDF .deb was so problematic.
I have not done anything on Debianization in a few
months, and I have lost interest in doing it myself
in the near future. I'm sure Brian Mays will be happy
to transfer maintainership of NCO to one of us.
I hope you will consider taking on this role since
you have done almost all of the Debianization until now.
Brian will probably transfer netCDF as well, but you would have to ask
him. He maintains _a lot_ of packages so he'll probably be happy if
you want to take some over. I said I was going to Debianize
UDUnits as well, but I've lost motivation for that too. So the field
is wide open for anyone.
I'm debugging a new operator, ncbnr, which supercedes ncdiff, and, in
addition, does addition (and division and multiplication) of whole
files. I'm happier putting my energies into features and gearing up
to write an NCO-HDF funding proposal for NASA.
Thanks,
Charlie
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Here are some questions on the Debian side of things:
> chmod +x debian/rules
Should we change this in the CVS repository so that it always comes in
executable mode on new checkouts?
I built a .deb of 2.8.0. It produced some warnings, but looks very
clean and complete. I think we're ready to start uploading to Sid.
> libtool: install: warning: remember to run `libtool --finish /usr/lib'
Should I worry about this? Maybe it's referring to the fact that
I do not have the NCO .deb installed on this machine, even though
I'm building the deb on this machine.
> dh_shlibdeps -l debian/nco/usr/lib
> dpkg-shlibdeps: warning: format of libnco.0 not recognized
> dpkg-shlibdeps: warning: format of libnco.0 not recognized
> dpkg-shlibdeps: warning: format of libnco.0 not recognized
> dpkg-shlibdeps: warning: format of libnco.0 not recognized
> dpkg-shlibdeps: warning: format of libnco.0 not recognized
> dpkg-shlibdeps: warning: format of libnco.0 not recognized
> dpkg-shlibdeps: warning: format of libnco.0 not recognized
> dpkg-shlibdeps: warning: format of libnco.0 not recognized
> dpkg-shlibdeps: warning: format of libnco.0 not recognized
Safe to ignore?
The deb seems to build with an "i386" identity, e.g., it calls GCC
with "i386-linux-gcc":
Is this what we want or can we have it produce i686 .debs or are
they in fact the same thing and i386 is just a generic name for
intel debs and the binaries in the deb are being compiling with
i686 optimizations since my machine is i686 (actually pentium IV)?
I've tagged and released 2.8.0. I linked the .deb files to the homepage.
It seems like there are no serious issues preventing us from uploading
these debs to ftp://ftp.debian.org. Will you clear up any issues
involved with Brian Mays, rebuild the debs if necessary, and see if
Mays will let you take over as Debian maintainer and start uploading?
I understand that there may be a few months of transition, but the
new .debs are much better than what's in the current Debian so we
should start uploading ASAP.
Thanks!
Charlie
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Charlie,
I emailed Brian Mays a few weeks ago and never heard from him regarding maintenance of NCO on Debian. Did you? I also don't see any posts by him on the debian-devel forum since last September.
Making debian/rules +x on CVS seems fine to me.
Because of incompatible libtool versions I had on my machine, the shared object names were being dropped, that causes these errors:
> dh_shlibdeps -l debian/nco/usr/lib
> dpkg-shlibdeps: warning: format of libnco.0 not recognized
> dpkg-shlibdeps: warning: format of libnco.0 not recognized
1. SGI with native compilers fails to build ncap and then dies.
ncap does not build because somehow HAVE_GETOPT_H is defined,
but getopt_long is not available. I think the configure.in
should be that if getopt() or getopt_long() is not found, then
we should #include nco_getopt.h.
With SGI native compilers, <getopt.h> exists even though it does
not contain getopt_long(). As a result, HAVE_GETOPT_H is defined
and then the build fails. So if we make HAVE_GETOPT_H be true
only when <getopt.h> exists and defines getopt_long() I think
that will fix the problem. The other option is to use
#if NEED_GETOPT_LONG
#include "nco_getopt.h"
#endif
independent of HAVE_GETOPT_H. What do you think?
> I emailed Brian Mays a few weeks ago and never heard from him
> regarding maintenance of NCO on Debian. Did you?
Nope, not since March when we started Debianizing.
> I also don't see
> any posts by him on the debian-devel forum since last September.
Maybe he's stressing over his thesis.
> Making debian/rules +x on CVS seems fine to me.
Hmmm....I could not figure out how to change modes on files
in SF's CVS repository, so I deleted it, changed it mode and
then re-added it, but the permissions did not change.
Yikes. I did not realize how gory shared libraries support was.
I trust you understand the details better than I, so implement
whatever versioning seems best. libnco and libnco_c++ will probably
never be used for anything except NCO, so don't sweat it too much.
> As I understand it, i386 is the Intel chip in general, the other
> options are sparc, alpha, etc.
Yes
> Maybe I'll post on debian-devel and see if anyone has heard from
> Brian lately.
Good idea
Thanks,
Charlie
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Charlie,
I fixed some stuff up in the debian/ dir. I decided to only make it a single package with the documentation (as html) included. I also fixed the naming scheme for the shared library. Since it is not used by any other packages except nco, we should be able to skip the [version].[release].[age] naming scheme. I think we should go with libnco-2.8.0.so for version 2.8.0, etc. That way the library will change names every time the version of nco changes, which it probably should since they are not intercompatible with other versions, usually. The agree with Debian policy, AFAIK. The package is lintian and linda clean currently, but I am still new to things.
The UDUnits stuff is statically compiled into the .deb, which might be the best way to handle it instead of trying to introduce yet another package into debian. If you could update the .deb on the NCO webpage, that would be a good idea.
I emailed Brian Mays one more time about NCO. If I still don't hear back anytime soon, I'll send an MIA report to debian-devel and debian-qa. I'll take over maintenance if he wants.
rorik
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I like your library naming scheme.
The changes you made seem to help my system
autotools work again, so that's good.
I will update the .debs on the homepage when
I put out 2.8.1. This should be within two weeks,
once Henry's wildcard parser stabilizes.
Now that the autotools are more or less working
consistently for me, the advantages of using them are becoming clear. e.g., 'make tags' and
search and replace is easy. I continue to maintain bld/Makefile for systems where autotools is not working.
One thing I like about bld/Makefile is that
I can implement strict compiler flags, e.g.,
'-Wall'. This helps keeping the code clean.
Do you think flags like this should be in
configure.in, or should be specified in the
CFLAGS argument to ./configure by each user?
Thanks,
Charlie
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
> One thing I like about bld/Makefile is that
> I can implement strict compiler flags, e.g.,
> '-Wall'. This helps keeping the code clean.
> Do you think flags like this should be in
> configure.in, or should be specified in the
> CFLAGS argument to ./configure by each user?
My opinion on this is simply to use CFLAGS. I do this regularly with my own projects when I need to debug it. I like to turn off optimization as well, so I can step through the code easier, thus I use
However, different people like different things, and using CFLAGS is most flexible, yet easy to use.
> One more thing....the library versioning decision
> rekindles a problem we never solved originally, namely,
> how/where do we specify the NCO version?
> I would like to specify it only in one place for all the
> source code. I prefer to use doc/VERSION.
Currently, doc/VERSION is the only place the version number is specified. That file is read by 'configure', which then sets $VERSION via AM_INIT_AUTOMAKE, and that is passed through to all the generated Makefiles. The shared library is constructed using $VERSION as is the gzip'ed tar file of 'make dist', so simply changing doc/VERSION should do it all. Let me know if you see any different behavior.
rorik
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Charlie,
The last time I revised the .deb builds, I did away with the nco-doc package and included documentation in .info and .html format with the binary package, see Policy 12.4 http://www.debian.org/doc/debian-policy/ch-docs.html#s12.4
I wasn't sure if it was worth creating and maintaining a separate package with the same information in a different format since people can generate those other formats from the html, or get it from the NCO homepage. If you feel strongly that there should be a separate nco-doc package, I can revert to that.
rorik
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I forgot you did this.
I think one package is fine, simpler, better.
Let's add the .ps and .pdf files to the nco .deb
in 2.8.2, though, so that people get everything
when they update.
Thanks,
Charlie
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I heard back from Brian Mays. He's quite busy with a new job and writing, but said he might get around to updating the nco package in September or so. I guess there is/was a problem with moving the source code that exacerbated things, and it sounds like the webpage has some mistakes. I let him know he can disregard any of the (potentially harmful) changes we've made to the debian/ build mechanism since the old one works fine. I attached his response below.
rorik
>
>
>I am not quite MIA, but I do have quite a few things going on in my life
>since this spring. Finishing a PhD, moving, and starting a new job take
>up quite a bit of one's time, and while I typically can make minor
>improvements to packages without much difficulty, I haven't been able to
>spare the time -- which most likely would require a couple of days -- to
>update the NCO packages.
>
>Right now, I have several other Debian packages that need changes, I
>need to write at least two papers for publication in the Journal of
>Computational Physics, and I am settling down into my new job. Since
>only one of these activities results in my rent being paid, it receives
>the highest priority. The others will be done when I have time.
>
>The main problem in the past year or so is that the upstream maintainers
>totally changed the way in which the upstream code is distributed (i.e.,
>they moved away from the usual ftp site where I was downloading the
>source) without telling anyone. Therefore, I was unaware that ongoing
>development was occurring upstream until I receive an email out of the
>blue in March. (Sorry, but I don't have the time to scan Google
>periodically for NCO to find where the source code has gone.) Note that
>even the NCO sourceforge website lists the location of the upstream
>sources incorrectly.
>
>Thus, I receive an email informing me of all sorts of upstream
>development and pointing out that the Debian package is out of date.
>That in itself would not be a problem, but the message also included a
>fairly detailed list of changes in the way the code is built, including
>a special "debian" directory provided in the upstream source. This
>immediately signaled to me that significant work (more than a day) will
>be required to incorporate these changes into the current Debian
>package. These "improvements" will need to be carefully read, digested,
>understood, and modified to produce a quality result. Too many times, I
>have seen well-meaning, but not entirely competent, developers take a
>stab at doing their own version of a Debian package, and the results
>often require extensive effort to work around their code so that one can
>produce a real, quality package. I'm sorry, but the job used upstream
>to "debianize" NCO just looks sloppy to me, and they would have received
>a faster response from me if they had left well enough alone.
>
>Thus, the situation is as follows. If someone can point me to an new
>upstream source that is fairly similar to the source in the current
>Debian package, then I can incorporate these changes into Debian in a
>very short time. If, however, the upstream source contains "significant
>improvements" to the way the software is built (i.e., it has extensive
>changes), then don't expect anything from me before the middle of
>September.
>
>Respectfully,
>
>- Brian
>
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Rorik forwarded me Brian's response about debianizing NCO.
I'm not sure what broken links to the source Brian is talking about.
They all seem to work for me.
There certainly have been broken links in the past though.
Brian, if you are really busy, then consider allowing Rorik to take
over maintainership. He's an active NCO developer and, in my opinion,
has done a great first stab at debianizing NCO.
I (and possibly Rorik) don't care who is the official Debian maintainer.
I do want the up-to-date NCO releases to enter Sid, since I have
switched to Debian and want to contribute what I can to making Debian
better.
The best way to stay apprised of important NCO releases is to join
nco-announce:
Uploading new releases to sourceforge is a PITA, their system makes
no sense to me and is too frustrating to use. I do it once every few years.
> I'm sorry, but the job used upstream
> to "debianize" NCO just looks sloppy to me, and they would have received
> a faster response from me if they had left well enough alone.
Have you checked recently, in the last few months?
My first attempts were a bit hackish, but Rorik has debianized NCO
completely by the book as far as I can tell.
What bothers you about the implementation?
>If someone can point me to an new
>upstream source that is fairly similar to the source in the current
>Debian package, then I can incorporate these changes into Debian in a
>very short time.
>If, however, the upstream source contains "significant
>improvements" to the way the software is built (i.e., it has extensive
>changes), then don't expect anything from me before the middle of
>September.
My guess is you would call these extensive changes.
You may download current .debs directly from the NCO homepage.
Give 'em a look see and tell us how to make them better.
Thanks and best wishes in your new job,
Charlie
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Courtesy post of a message sent to the Debian NCO maintainer:
Hi Brian,
I've taken a first step in Debian'izing the NCO upstream source.
Your files from the nco-2.2.0 deb package are now in the upstream
source. This includes all files in nco/debian as well as
nco/bld/linux1 and nco/bld/linux2.
There have been many significant changes to the Makefile since the NCO
library was restructured, so I have not tried to port any improvements
you made to the Debian bld/Makefile except to insert the LINUXARM
architecture into the LINUX* rules.
I wanted to warn you that the upstream source (in CVS, not released
yet) now has a debian directory.
I hope this does not create difficulties for you!
My plan is to try to make the upstream source Debian
compatible by about version 2.8.0 (summer?).
At that point, I envision two distinct build mechanisms:
1. nco/bld/Makefile
This is what I will continue to use for NCO development.
It works by default on all platforms I have access to and is
very useful for porting to new platforms and ones on which
I do not have root priveleges and so which may lack the GNU
toolchain. It is currently the only way to build certain
NCO configurations (e.g., AIX, OSF1, NEC, CRAY, DODS...).
I do not plan to merge your Debian-specific Makefile changes
into the generic portion of bld/Makefile, but am happy to
merge any generic stuff that is useful.
Also, if you wanted to make Debian-specific changes activate
with a keyword, e.g., DBN, then we could certainly include
Debian-specific flags/rule/target modification under that
mechanism. If you are interested in modifying the upstream
source yourself I will add you to the developers list on
sourceforge. Let me know. Otherwise I will modify your debian/* files
until they are synchronized with the current NCO and the packages
build fine on my home debian machine.
2. Generic ./configure mechanism
Rorik Peterson added a generic ./configure mechanism to NCO.
It works great on Linux but has problems on most other machines.
I could not use this in maintainer mode until I switched to
Debian (NCO is the main reason I switched from RedHat to Debian).
I think the default Debian build mechanism should be migrated into
the ./configure mechanism. This will allow Debian users to benefit
from the shared libraries etc. that autoconf/libtool/automake
support. However, I want the ./configure mechanism to continue to
work for non-Debian systems. Since Debian builds have to adhere
to the Debian policy guidelines, I can imagine that modifying
the current ./configure mechanism to fit those guidelines would
result in a ./configure mechanism that does not work for non-Debian
users (e.g., users who want to install into /usr/local by default).
I really do not know enought about autotools or Debian guidelines
to know if this will be a problem, but I'm assuming that any
problems can be addressed by adding a --enable-debian to configure.in
if necessary.
Anyway, that's the plan. The ulimate goal is to be able to upload
tagged versions of NCO (e.g., 2.8.1) to debian unstable with
a single command. Please let me know if you have any suggestions on
how to make this transition go smoothly.
Thanks,
Charlie
I made a few somewhat major changes to the debian/ directory. What are your opinions on these?
* several new files were added that various debhelper scripts use, and make maintainance/modification fairly painless. Most are described below.
* 'rules' has been changes substantially. It builds using configure since that seems to work fine on Linux machines. I used the default skeleton file and modified it somewhat, so many of the debhelper programs are used now. For example, the dependencies are determined automatically by dh_shlibdeps and dh_makeshlibs (and turn out to only be libnetcdf3g and libc6), and info files are installed/ uninstalled by default.
* I made a second package called nco-doc, which includes the documentation in html and ps. We could add/change that to pdf, dvi, or whatever. The html is a single page, it could be broken down into chapters (see doc/Makefile.am). All the documentation could also simply be included in the main nco package (installed in /usr/share/doc/nco/doc/ instead of /usr/share/nco-doc/). However, the info and man pages already get installed with the main nco package, and the info pages contain the same information, so I'm ambivalent
* the documentation will be registered with doc-base.
* There are no dependencies for nco-doc, but we could recommend a postscript viewer. I think dpkg requires these and apt-get ignores them, but I'm not sure.
* The doc/NEWS file is terribly out of date, so I didn't include it in the nco package although it was there in the last release (2.2.0). I put doc/ChangeLog in there. See debian/docs for modifications and additions to this list. doc/nco.shtml was in the last debian distribution.
* We could add the test script bld/nco_tst.sh to a nco-doc/examples directory (?)
* when making the .deb file, ther version number is determined automatically from the main directory name (i.e. nco-2.7.2/) This seems like the easiest way to make it, particularly since 'make dist' creates nco-2.7.2.tar.gz automatically.
* since documentation and info files are handled automatically with the various debhelper scripts, I think 'prerm' and 'postinst' are not needed anymore.
I added '--disable-nco_cplusplus" to the configure options so the c++ library can be conditionally built and installed since there is no reason to inlcude it in the .deb file for now.
* the UDUnits will not be included since there is not a debian version of the library and the dependency could not be met. I'm not sure if Brian would want to maintain that aditional package on or not. I'm not a Debian maintainer.
I created nco_2.7.2-1_i386.deb on my machine and it installed fine.
rorik
Hi Rorik,
Thanks! All your changes sound like improvements
but I still want to examine the specific modifications.
I will try to do so until this weekend.
I also had a working .deb building before your
commits so I may merge some changes I made.
Specific comments inserted below.
Charlie
SourceForge.net wrote:
> * several new files were added that various debhelper scripts use, and make
> maintainance/modification fairly painless. Most are described below.
Good, we want NCO to use the relevant debhelper scripts.
> * 'rules' has been changes substantially. It builds using configure since that
> seems to work fine on Linux machines. I used the default skeleton file and
> modified it somewhat, so many of the debhelper programs are used now. For example,
> the dependencies are determined automatically by dh_shlibdeps and dh_makeshlibs
> (and turn out to only be libnetcdf3g and libc6), and info files are installed/
> uninstalled by default.
Good, I don't expect to build debs on machines where the configure
mechanism does not work for NCO.
> * I made a second package called nco-doc, which includes the documentation in
> html and ps. We could add/change that to pdf, dvi, or whatever. The html is
> a single page, it could be broken down into chapters (see doc/Makefile.am).
> All the documentation could also simply be included in the main nco package
> (installed in /usr/share/doc/nco/doc/ instead of /usr/share/nco-doc/). However,
> the info and man pages already get installed with the main nco package, and
> the info pages contain the same information, so I'm ambivalent
Do you mean that you can install nco but not nco-doc?
I suppose this is a good idea, but what do other packages do?
Make the docs recommended?
I just want the PDF, DVI, Postscript, .txt and/or XML available in a deb somehow.
The man pages, by the way, are completely out of date since they are
based on NCO 2.2 and there is no automated way (that I know of) of
generating them from the TeXInfo source.
> * the documentation will be registered with doc-base.
Great.
> * There are no dependencies for nco-doc, but we could recommend a postscript
> viewer. I think dpkg requires these and apt-get ignores them, but I'm not sure.
I would leave viewers as recommended rather than dependencies.
The .txt format does not require a viewer.
> * The doc/NEWS file is terribly out of date, so I didn't include it in the nco
> package although it was there in the last release (2.2.0). I put doc/ChangeLog
> in there. See debian/docs for modifications and additions to this list.
> doc/nco.shtml was in the last debian distribution.
> * We could add the test script bld/nco_tst.sh to a nco-doc/examples directory
> (?)
It is a good example but it's also part of the build mechanism,
so does Debian policy dictate where to put it?
> * when making the .deb file, ther version number is determined automatically
> from the main directory name (i.e. nco-2.7.2/) This seems like the easiest
> way to make it, particularly since 'make dist' creates nco-2.7.2.tar.gz
> automatically.
I would prefer it if version numbers came from doc/VERSION.
> * since documentation and info files are handled automatically with the various
> debhelper scripts, I think 'prerm' and 'postinst' are not needed anymore.
I have no idea, and will try to think about prem and postinst this weekend.
> I added '--disable-nco_cplusplus" to the configure options so the c++ library
> can be conditionally built and installed since there is no reason to inlcude
> it in the .deb file for now.
Fine.
> * the UDUnits will not be included since there is not a debian version of the
> library and the dependency could not be met. I'm not sure if Brian would want
> to maintain that aditional package on or not. I'm not a Debian maintainer.
We should debianize UDUnits.
Let me take a crack at that this weekend because I want to become
a Debian maintainer in order to keep NCO snapshots up-to-date on
Sid. Udunits is a perfect first project for me to run by Brian Mays.
> I created nco_2.7.2-1_i386.deb on my machine and it installed fine.
I will try to build a 2.7.4 deb this weekend and if it works we can
release that.
Hi Rorik,
One more thing....the library versioning decision
rekindles a problem we never solved originally, namely, how/where do we specify the NCO version?
I would like to specify it only in one place for all the
source code. I prefer to use doc/VERSION. Perhaps
there is a better way. Is there a way to make macros etc. so that whatever is in doc/VERSION gets used for the library name? Or must we somehow manually and separately specify the
library version number (which is bound to fail
due to human error eventually)?
Thanks,
Charlie
Just a few more comments:
Concerning man pages, Debian policy (13.1) requires man pages. If they are not provided upstream, the maintainer either provides them or keeps an open bug report about it. If we don't want to maintain the man pages, we could either:
(1) simply add a warning that they might be out of date and the info file should be referenced for accurate documentation, or
(2) simply them all so they don't get out of date and again reference the info pages.
I don't think there is a reasonable way to convert texinfo to man pages with the structure of NCO, for example, there are multiple executeables.
Concerning nco-doc, yes you could install nco without nco-doc, or viseversa, although why someone would want the documenatation and not the program, I'm not sure. It seems to me that most large programs with lots of documentation, break it into documentation packages, see octave for example. I think it might also help with doc-base, but I'm not positive.
Concerning version numbers, if the source directory is not named program-version, dpkg-buildpackage will retrieve it from the last entry in debian/changelog. I'm not sure if you can force dpkg-buildpackage to directly use doc/VERSION. Now, 'make dist' does directly use doc/VERSION. However, if you're building out of a CVS snapshot, and not something generated from 'make dist', everything will be in the package-version.orig.tag.gz file. For NCO, that means the c++ stuff will be there although it is not part of the debian package, and any other files laying around in CVS. I'm not saying one is right or wrong, the maintainer should just decide what they want to do. I did add the bld/ directory to 'make dist'.
Concerning nco_tst.sh, you said this is part of the build mechanism. I don't see how, it is not _required_ for either bld/make or configure;make to work. Therefore, I don't think there is a debian policy concerning it. It really seems like a test to me.
Concerning UDUnits, we could statically link it in the nco binaries until such time as it is available in debian; it increases the library size less than 1%, and is licensed GPL.
rorik
Charlie,
What is the status of updating the version of NCO in Debian? Did you ever hear from Brian? The NetCDF library really needs to be updated on Debian as well, since it won't link with gcc/g++ versions < 3.0. I can try and help if you want.
rorik
Hi Rorik,
Thanks for reminding me about Debianization.
I did not realize that the current netCDF .deb was so problematic.
I have not done anything on Debianization in a few
months, and I have lost interest in doing it myself
in the near future. I'm sure Brian Mays will be happy
to transfer maintainership of NCO to one of us.
I hope you will consider taking on this role since
you have done almost all of the Debianization until now.
Brian will probably transfer netCDF as well, but you would have to ask
him. He maintains _a lot_ of packages so he'll probably be happy if
you want to take some over. I said I was going to Debianize
UDUnits as well, but I've lost motivation for that too. So the field
is wide open for anyone.
I'm debugging a new operator, ncbnr, which supercedes ncdiff, and, in
addition, does addition (and division and multiplication) of whole
files. I'm happier putting my energies into features and gearing up
to write an NCO-HDF funding proposal for NASA.
Thanks,
Charlie
Hi Rorik,
Here are some questions on the Debian side of things:
> chmod +x debian/rules
Should we change this in the CVS repository so that it always comes in
executable mode on new checkouts?
I built a .deb of 2.8.0. It produced some warnings, but looks very
clean and complete. I think we're ready to start uploading to Sid.
> libtool: install: warning: remember to run `libtool --finish /usr/lib'
Should I worry about this? Maybe it's referring to the fact that
I do not have the NCO .deb installed on this machine, even though
I'm building the deb on this machine.
> dh_shlibdeps -l debian/nco/usr/lib
> dpkg-shlibdeps: warning: format of libnco.0 not recognized
> dpkg-shlibdeps: warning: format of libnco.0 not recognized
> dpkg-shlibdeps: warning: format of libnco.0 not recognized
> dpkg-shlibdeps: warning: format of libnco.0 not recognized
> dpkg-shlibdeps: warning: format of libnco.0 not recognized
> dpkg-shlibdeps: warning: format of libnco.0 not recognized
> dpkg-shlibdeps: warning: format of libnco.0 not recognized
> dpkg-shlibdeps: warning: format of libnco.0 not recognized
> dpkg-shlibdeps: warning: format of libnco.0 not recognized
Safe to ignore?
The deb seems to build with an "i386" identity, e.g., it calls GCC
with "i386-linux-gcc":
> i386-linux-gcc -DHAVE_CONFIG_H -I. -I. -I../.. -I/usr/local/include -g
> -O2 -MT nco_var_scv.lo -MD -MP -MF .deps/nco_var_scv.Tpo -c
> nco_var_scv.c -o nco_var_scv.o >/dev/null 2>&1
Is this what we want or can we have it produce i686 .debs or are
they in fact the same thing and i386 is just a generic name for
intel debs and the binaries in the deb are being compiling with
i686 optimizations since my machine is i686 (actually pentium IV)?
I've tagged and released 2.8.0. I linked the .deb files to the homepage.
It seems like there are no serious issues preventing us from uploading
these debs to ftp://ftp.debian.org. Will you clear up any issues
involved with Brian Mays, rebuild the debs if necessary, and see if
Mays will let you take over as Debian maintainer and start uploading?
I understand that there may be a few months of transition, but the
new .debs are much better than what's in the current Debian so we
should start uploading ASAP.
Thanks!
Charlie
Charlie,
I emailed Brian Mays a few weeks ago and never heard from him regarding maintenance of NCO on Debian. Did you? I also don't see any posts by him on the debian-devel forum since last September.
Making debian/rules +x on CVS seems fine to me.
Because of incompatible libtool versions I had on my machine, the shared object names were being dropped, that causes these errors:
> dh_shlibdeps -l debian/nco/usr/lib
> dpkg-shlibdeps: warning: format of libnco.0 not recognized
> dpkg-shlibdeps: warning: format of libnco.0 not recognized
I uploaded a new autobld/ltmain.sh to fix this. I guess we should decide an initial version number for libnco.so.x.x.x. There is more information about libtool versioning information here:
http://www.gnu.org/manual/libtool-1.4.2/html_mono/libtool.html#SEC34
As I understand it, i386 is the Intel chip in general, the other options are sparc, alpha, etc.
This libtool error I don't recognize off hand, I'll look into it.
> libtool: install: warning: remember to run `libtool --finish /usr/lib'
Maybe I'll post on debian-devel and see if anyone has heard from Brian lately.
rorik
Hi Rorik,
I noticed one minor autobuild problem on SGI.
1. SGI with native compilers fails to build ncap and then dies.
ncap does not build because somehow HAVE_GETOPT_H is defined,
but getopt_long is not available. I think the configure.in
should be that if getopt() or getopt_long() is not found, then
we should #include nco_getopt.h.
With SGI native compilers, <getopt.h> exists even though it does
not contain getopt_long(). As a result, HAVE_GETOPT_H is defined
and then the build fails. So if we make HAVE_GETOPT_H be true
only when <getopt.h> exists and defines getopt_long() I think
that will fix the problem. The other option is to use
#if NEED_GETOPT_LONG
#include "nco_getopt.h"
#endif
independent of HAVE_GETOPT_H. What do you think?
> I emailed Brian Mays a few weeks ago and never heard from him
> regarding maintenance of NCO on Debian. Did you?
Nope, not since March when we started Debianizing.
> I also don't see
> any posts by him on the debian-devel forum since last September.
Maybe he's stressing over his thesis.
> Making debian/rules +x on CVS seems fine to me.
Hmmm....I could not figure out how to change modes on files
in SF's CVS repository, so I deleted it, changed it mode and
then re-added it, but the permissions did not change.
> I uploaded a new autobld/ltmain.sh to fix this. I guess we should
> decide an initial version number for libnco.so.x.x.x. There is more
> information about libtool versioning information here:
> http://www.gnu.org/manual/libtool-1.4.2/html_mono/libtool.html#SEC34
Yikes. I did not realize how gory shared libraries support was.
I trust you understand the details better than I, so implement
whatever versioning seems best. libnco and libnco_c++ will probably
never be used for anything except NCO, so don't sweat it too much.
> As I understand it, i386 is the Intel chip in general, the other
> options are sparc, alpha, etc.
Yes
> Maybe I'll post on debian-devel and see if anyone has heard from
> Brian lately.
Good idea
Thanks,
Charlie
Charlie,
I fixed some stuff up in the debian/ dir. I decided to only make it a single package with the documentation (as html) included. I also fixed the naming scheme for the shared library. Since it is not used by any other packages except nco, we should be able to skip the [version].[release].[age] naming scheme. I think we should go with libnco-2.8.0.so for version 2.8.0, etc. That way the library will change names every time the version of nco changes, which it probably should since they are not intercompatible with other versions, usually. The agree with Debian policy, AFAIK. The package is lintian and linda clean currently, but I am still new to things.
The UDUnits stuff is statically compiled into the .deb, which might be the best way to handle it instead of trying to introduce yet another package into debian. If you could update the .deb on the NCO webpage, that would be a good idea.
I emailed Brian Mays one more time about NCO. If I still don't hear back anytime soon, I'll send an MIA report to debian-devel and debian-qa. I'll take over maintenance if he wants.
rorik
Hi Rorik,
I like your library naming scheme.
The changes you made seem to help my system
autotools work again, so that's good.
I will update the .debs on the homepage when
I put out 2.8.1. This should be within two weeks,
once Henry's wildcard parser stabilizes.
Now that the autotools are more or less working
consistently for me, the advantages of using them are becoming clear. e.g., 'make tags' and
search and replace is easy. I continue to maintain bld/Makefile for systems where autotools is not working.
One thing I like about bld/Makefile is that
I can implement strict compiler flags, e.g.,
'-Wall'. This helps keeping the code clean.
Do you think flags like this should be in
configure.in, or should be specified in the
CFLAGS argument to ./configure by each user?
Thanks,
Charlie
> One thing I like about bld/Makefile is that
> I can implement strict compiler flags, e.g.,
> '-Wall'. This helps keeping the code clean.
> Do you think flags like this should be in
> configure.in, or should be specified in the
> CFLAGS argument to ./configure by each user?
My opinion on this is simply to use CFLAGS. I do this regularly with my own projects when I need to debug it. I like to turn off optimization as well, so I can step through the code easier, thus I use
./configure --disable-shared CFLAGS="-g -Wall -O0"
However, different people like different things, and using CFLAGS is most flexible, yet easy to use.
> One more thing....the library versioning decision
> rekindles a problem we never solved originally, namely,
> how/where do we specify the NCO version?
> I would like to specify it only in one place for all the
> source code. I prefer to use doc/VERSION.
Currently, doc/VERSION is the only place the version number is specified. That file is read by 'configure', which then sets $VERSION via AM_INIT_AUTOMAKE, and that is passed through to all the generated Makefiles. The shared library is constructed using $VERSION as is the gzip'ed tar file of 'make dist', so simply changing doc/VERSION should do it all. Let me know if you see any different behavior.
rorik
OK, I'll use CFLAGS like you.
Awesome, VERSION already works. I did not know that.
Hi Rorik,
I released 2.8.1.
How do you generate updated nco-doc .debs?
It does not happen automaticallly when I use
cd ~/nco;dpkg-buildpackage -rfakeroot
Thanks,
Charlie
Charlie,
The last time I revised the .deb builds, I did away with the nco-doc package and included documentation in .info and .html format with the binary package, see Policy 12.4
http://www.debian.org/doc/debian-policy/ch-docs.html#s12.4
I wasn't sure if it was worth creating and maintaining a separate package with the same information in a different format since people can generate those other formats from the html, or get it from the NCO homepage. If you feel strongly that there should be a separate nco-doc package, I can revert to that.
rorik
I forgot you did this.
I think one package is fine, simpler, better.
Let's add the .ps and .pdf files to the nco .deb
in 2.8.2, though, so that people get everything
when they update.
Thanks,
Charlie
I heard back from Brian Mays. He's quite busy with a new job and writing, but said he might get around to updating the nco package in September or so. I guess there is/was a problem with moving the source code that exacerbated things, and it sounds like the webpage has some mistakes. I let him know he can disregard any of the (potentially harmful) changes we've made to the debian/ build mechanism since the old one works fine. I attached his response below.
rorik
>
>
>I am not quite MIA, but I do have quite a few things going on in my life
>since this spring. Finishing a PhD, moving, and starting a new job take
>up quite a bit of one's time, and while I typically can make minor
>improvements to packages without much difficulty, I haven't been able to
>spare the time -- which most likely would require a couple of days -- to
>update the NCO packages.
>
>Right now, I have several other Debian packages that need changes, I
>need to write at least two papers for publication in the Journal of
>Computational Physics, and I am settling down into my new job. Since
>only one of these activities results in my rent being paid, it receives
>the highest priority. The others will be done when I have time.
>
>The main problem in the past year or so is that the upstream maintainers
>totally changed the way in which the upstream code is distributed (i.e.,
>they moved away from the usual ftp site where I was downloading the
>source) without telling anyone. Therefore, I was unaware that ongoing
>development was occurring upstream until I receive an email out of the
>blue in March. (Sorry, but I don't have the time to scan Google
>periodically for NCO to find where the source code has gone.) Note that
>even the NCO sourceforge website lists the location of the upstream
>sources incorrectly.
>
>Thus, I receive an email informing me of all sorts of upstream
>development and pointing out that the Debian package is out of date.
>That in itself would not be a problem, but the message also included a
>fairly detailed list of changes in the way the code is built, including
>a special "debian" directory provided in the upstream source. This
>immediately signaled to me that significant work (more than a day) will
>be required to incorporate these changes into the current Debian
>package. These "improvements" will need to be carefully read, digested,
>understood, and modified to produce a quality result. Too many times, I
>have seen well-meaning, but not entirely competent, developers take a
>stab at doing their own version of a Debian package, and the results
>often require extensive effort to work around their code so that one can
>produce a real, quality package. I'm sorry, but the job used upstream
>to "debianize" NCO just looks sloppy to me, and they would have received
>a faster response from me if they had left well enough alone.
>
>Thus, the situation is as follows. If someone can point me to an new
>upstream source that is fairly similar to the source in the current
>Debian package, then I can incorporate these changes into Debian in a
>very short time. If, however, the upstream source contains "significant
>improvements" to the way the software is built (i.e., it has extensive
>changes), then don't expect anything from me before the middle of
>September.
>
>Respectfully,
>
>- Brian
>
Hi Rorik and Brian,
Rorik forwarded me Brian's response about debianizing NCO.
I'm not sure what broken links to the source Brian is talking about.
They all seem to work for me.
There certainly have been broken links in the past though.
Brian, if you are really busy, then consider allowing Rorik to take
over maintainership. He's an active NCO developer and, in my opinion,
has done a great first stab at debianizing NCO.
I (and possibly Rorik) don't care who is the official Debian maintainer.
I do want the up-to-date NCO releases to enter Sid, since I have
switched to Debian and want to contribute what I can to making Debian
better.
The best way to stay apprised of important NCO releases is to join
nco-announce:
http://lists.sourceforge.net/mailman/listinfo/nco-announce
Uploading new releases to sourceforge is a PITA, their system makes
no sense to me and is too frustrating to use. I do it once every few years.
> I'm sorry, but the job used upstream
> to "debianize" NCO just looks sloppy to me, and they would have received
> a faster response from me if they had left well enough alone.
Have you checked recently, in the last few months?
My first attempts were a bit hackish, but Rorik has debianized NCO
completely by the book as far as I can tell.
What bothers you about the implementation?
>If someone can point me to an new
>upstream source that is fairly similar to the source in the current
>Debian package, then I can incorporate these changes into Debian in a
>very short time.
The source is at http://dust.ess.uci.edu/nco/nco.tar.gz.
>If, however, the upstream source contains "significant
>improvements" to the way the software is built (i.e., it has extensive
>changes), then don't expect anything from me before the middle of
>September.
My guess is you would call these extensive changes.
You may download current .debs directly from the NCO homepage.
Give 'em a look see and tell us how to make them better.
Thanks and best wishes in your new job,
Charlie