distmake-general Mailing List for distmake
Brought to you by:
mblythe86
You can subscribe to this list here.
2006 |
Jan
(5) |
Feb
(12) |
Mar
(9) |
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
---|---|---|---|---|---|---|---|---|---|---|---|---|
2008 |
Jan
|
Feb
|
Mar
|
Apr
|
May
(2) |
Jun
(10) |
Jul
(5) |
Aug
(1) |
Sep
|
Oct
(2) |
Nov
|
Dec
|
2009 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
(20) |
2015 |
Jan
|
Feb
(2) |
Mar
|
Apr
|
May
|
Jun
(1) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
From: Blythe, M. <Mat...@am...> - 2015-06-09 20:53:17
|
HI Bruno, Sorry about taking so long to reply. I've finally convinced my company to let me contribute to distmake. I'll be happy to take over for you, if you'd like. My sourceforge username is mblythe86. Thanks, Matt Blythe From: Bruno Fleisch [mailto:bfl...@fr...] Sent: Saturday, February 21, 2015 1:41 AM To: dis...@li... Subject: Re: [Distmake-general] Does anyone still maintain distmake? Hi Matthew I do not maintain this project anymore due to lack of time and because I do not work as system engineer anymore. I'll be more than happy if someone wants to take over from me. Any volunteer ? Thank you Regards Bruno Le 20/02/2015 01:02, Blythe, Matthew a écrit : Hi, I see there haven't been any releases on this project since 2008. Does anyone still actively maintain this project? If so, since 2008 GNU make has progressed quite a lot. Are there any plans to bring distmake up-to-date with a recent release of GNU make? Thanks, Matt Blythe ------------------------------------------------------------------------------ Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server from Actuate! Instantly Supercharge Your Business Reports and Dashboards with Interactivity, Sharing, Native Excel Exports, App Integration & more Get technology previously reserved for billion-dollar corporations, FREE http://pubads.g.doubleclick.net/gampad/clk?id=190641631&iu=/4140/ostg.clktrk _______________________________________________ Distmake-general mailing list Dis...@li...<mailto:Dis...@li...> https://lists.sourceforge.net/lists/listinfo/distmake-general -- Bruno Fleisch http://www.linkedin.com/in/brunofleisch |
From: Bruno F. <bfl...@fr...> - 2015-02-21 09:11:42
|
Hi Matthew I do not maintain this project anymore due to lack of time and because I do not work as system engineer anymore. I'll be more than happy if someone wants to take over from me. Any volunteer ? Thank you Regards Bruno Le 20/02/2015 01:02, Blythe, Matthew a écrit : > > Hi, > > I see there haven’t been any releases on this project since 2008. > Does anyone still actively maintain this project? > > If so, since 2008 GNU make has progressed quite a lot. Are there any > plans to bring distmake up-to-date with a recent release of GNU make? > > Thanks, > > Matt Blythe > > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=190641631&iu=/4140/ostg.clktrk > > > _______________________________________________ > Distmake-general mailing list > Dis...@li... > https://lists.sourceforge.net/lists/listinfo/distmake-general -- Bruno Fleisch http://www.linkedin.com/in/brunofleisch |
From: Blythe, M. <Mat...@am...> - 2015-02-20 00:35:14
|
Hi, I see there haven't been any releases on this project since 2008. Does anyone still actively maintain this project? If so, since 2008 GNU make has progressed quite a lot. Are there any plans to bring distmake up-to-date with a recent release of GNU make? Thanks, Matt Blythe |
From: Glen B. <Gle...@ja...> - 2009-12-18 12:42:36
|
On 12/18/09 2:17 AM, "John Coiner" <joh...@am...> wrote: Is it possible that two processes were both trying to "mv -f $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po" around the same time? That could lead to problems. $* would evaluate to something different for each .c file that gets compiled with this rule: foo.c genereates $(DEPDIR)/foo.Tpo, and then does the move with $* evaluating to "foo", so the mv should only be called once on foo.Tpo, unless foo.c were getting compiled by two different processes around the same time -- Glen L. Beane Software Engineer The Jackson Laboratory Phone (207) 288-6153 |
From: John C. <joh...@am...> - 2009-12-18 07:18:14
|
Hi Glen, Is it possible that two processes were both trying to "mv -f $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po" around the same time? That could lead to problems. I committed some changes you might want to try. They are on the 'nfs_safe_dec09' branch of distmake in CVS. These changes add two things: * a README.NFS file * a '--nfs-safe' flag. This enables an NFS-hardening scheme which README.NFS talks about. Many Makefiles should be able to run on distmake with '--nfs-safe' without the noac mount option. (Not all Makefiles work: if a Make rule tries to read a file that's not in its prerequisite list, --nfs-safe provides no protection for that.) If you try this, I'd like to hear about what you find. Thanks, John Glen Beane wrote: > > > > no, this file is not a symlink. If you are not familiar with .Tpo/.Po > files, this is how they are generated/used > > > the gnu autotools generate a rule like this to compile a .c file into a .o: > > .c.o: > $(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< > mv -f $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po > > The .Tpo file is generated during this compile process (actually by the > C preprocessor) from all the –MX flags. After it is done, it moves the > file so it has a .Po extension. The file contains a single make rule > containing all of the #include dependencies of the file being compiled. > > When you run ./configure it generates all of these .Po files with the > contents of something like #dummy, but after you have run make once, > they get populated as described above. They are included in the > Makefile like this: > > include ./$(DEPDIR)/rand.Po > > immediately before the .c.o: rule (so the first time around this just > includes the comment #dummy into the Makefile, the second time around it > includes the dependencies generated during the previous run of make) > > The contents looks something like: > > rand.o: rand.c /usr/include/stdlib.h /usr/include/features.h \ > /usr/include/sys/cdefs.h /usr/include/bits/wordsize.h \ > /usr/include/gnu/stubs.h /usr/include/gnu/stubs-64.h \ > /usr/lib64/gcc/x86_64-suse-linux/4.3/include/stddef.h \ > /usr/include/sys/types.h /usr/include/bits/types.h \ > /usr/include/bits/typesizes.h /usr/include/time.h /usr/include/endian.h \ > /usr/include/bits/endian.h /usr/include/bits/byteswap.h \ > /usr/include/sys/select.h /usr/include/bits/select.h \ > /usr/include/bits/sigset.h /usr/include/bits/time.h \ > /usr/include/sys/sysmacros.h /usr/include/bits/pthreadtypes.h \ > /usr/include/alloca.h /usr/include/stdio.h /usr/include/libio.h \ > /usr/include/_G_config.h /usr/include/wchar.h \ > /usr/lib64/gcc/x86_64-suse-linux/4.3/include/stdarg.h \ > /usr/include/bits/stdio_lim.h /usr/include/bits/sys_errlist.h \ > /usr/include/bits/stdio.h /usr/include/math.h \ > ... > > and the whole point is so that if you run make again and any one of > those header files changes then make knows it needs to recompile that > target, otherwise it only recompiles a .o file if the .c files it > depends on have changed. > > > Sometimes I get the mv error, other times it works fine. I suspect it > has something to do with the attribute caching, since I haven’t seen > that yet using that NFS share. I have no idea how you could fix that in > distmake, but this is a common thing (pretty much anything with an > Autotools based build system will make these files). I added a cat > > /dev/null for both the .Tpo and .Po before the mv –f and that seems to > work all (most?) of the time. > > For now I’ll probably stay fat dumb and happy and run distmake on a NFS > share with “noac” so I don’t have to change my Makefiles (which we > definitely do not want to do for the Illumina Pipeline). > > > > -- > Glen L. Beane > Software Engineer > The Jackson Laboratory > Phone (207) 288-6153 > |
From: Glen B. <Gle...@ja...> - 2009-12-17 20:28:24
|
On 12/17/09 2:40 PM, "John Coiner" <joh...@am...> wrote: Glen Beane wrote: > > So I have made two modifications to my makefile that seem to work around > my issue: > > first I tried a sleep here, (no luck) but switching it to the cat > helped, no more *.o not found: > > hmmSNP$(EXEEXT): $(hmmSNP_OBJECTS) $(hmmSNP_DEPENDENCIES) > cat *.o > /dev/null > @rm -f hmmSNP$(EXEEXT) > $(LINK) $(hmmSNP_OBJECTS) $(hmmSNP_LDADD) $(LIBS) > > then I was getting file not found errors for some of the mv -f > .deps/*.Tpo .deps/*.Po commands. I made the following modification > (added the cat command), and it took care of that problem... > > .c.o: > $(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< > cat $(DEPDIR)/$*.Tpo > /dev/null > mv -f $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po > Hi Glen, This trick works because when the shell expands "*.o" and "*.Tpo", it must reopen the directory, which fetches a fresh copy of the directory from the NFS server into the NFS client cache. I looked at whether it's feasible to have distmake send the list of prereqs over the wire to rpc.bldserver, so that rpc.bldserver can do NFS-cache-thwacking to make all prereqs visible. It looks like it would not be difficult. Specifically it does not require any changes to the source files of Gnu Make proper; it could be done entirely within the distmake extensions. It changes the client/server protocol a bit, I still need to study whether it's possible to do this such that newer server versions would still work with older client versions; that's a requirement for my site. So I'm tempted to add this feature now. > > then I got the following error that really doesn't make sense to me > > mv -f .deps/hmmMatrix.Tpo .deps/hmmMatrix.Po > === building 'hmmMatrix.o' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-2' > > mv -f .deps/hmmUtil.Tpo .deps/hmmUtil.Po > === building 'hmmUtil.o' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-1' > > mv: `.deps/hmmMatrix.Tpo' and `.deps/hmmMatrix.Po' are the same file That means one of these is a symlink to the other (or they are both a symlink to the same thing?) For example $ touch foo $ ln -s foo bar $ mv bar foo mv: `bar' and `foo' are the same file So this probably is not related to distmake per se. John no, this file is not a symlink. If you are not familiar with .Tpo/.Po files, this is how they are generated/used the gnu autotools generate a rule like this to compile a .c file into a .o: .c.o: $(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< mv -f $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po The .Tpo file is generated during this compile process (actually by the C preprocessor) from all the -MX flags. After it is done, it moves the file so it has a .Po extension. The file contains a single make rule containing all of the #include dependencies of the file being compiled. When you run ./configure it generates all of these .Po files with the contents of something like #dummy, but after you have run make once, they get populated as described above. They are included in the Makefile like this: include ./$(DEPDIR)/rand.Po immediately before the .c.o: rule (so the first time around this just includes the comment #dummy into the Makefile, the second time around it includes the dependencies generated during the previous run of make) The contents looks something like: rand.o: rand.c /usr/include/stdlib.h /usr/include/features.h \ /usr/include/sys/cdefs.h /usr/include/bits/wordsize.h \ /usr/include/gnu/stubs.h /usr/include/gnu/stubs-64.h \ /usr/lib64/gcc/x86_64-suse-linux/4.3/include/stddef.h \ /usr/include/sys/types.h /usr/include/bits/types.h \ /usr/include/bits/typesizes.h /usr/include/time.h /usr/include/endian.h \ /usr/include/bits/endian.h /usr/include/bits/byteswap.h \ /usr/include/sys/select.h /usr/include/bits/select.h \ /usr/include/bits/sigset.h /usr/include/bits/time.h \ /usr/include/sys/sysmacros.h /usr/include/bits/pthreadtypes.h \ /usr/include/alloca.h /usr/include/stdio.h /usr/include/libio.h \ /usr/include/_G_config.h /usr/include/wchar.h \ /usr/lib64/gcc/x86_64-suse-linux/4.3/include/stdarg.h \ /usr/include/bits/stdio_lim.h /usr/include/bits/sys_errlist.h \ /usr/include/bits/stdio.h /usr/include/math.h \ ... and the whole point is so that if you run make again and any one of those header files changes then make knows it needs to recompile that target, otherwise it only recompiles a .o file if the .c files it depends on have changed. Sometimes I get the mv error, other times it works fine. I suspect it has something to do with the attribute caching, since I haven't seen that yet using that NFS share. I have no idea how you could fix that in distmake, but this is a common thing (pretty much anything with an Autotools based build system will make these files). I added a cat > /dev/null for both the .Tpo and .Po before the mv -f and that seems to work all (most?) of the time. For now I'll probably stay fat dumb and happy and run distmake on a NFS share with "noac" so I don't have to change my Makefiles (which we definitely do not want to do for the Illumina Pipeline). -- Glen L. Beane Software Engineer The Jackson Laboratory Phone (207) 288-6153 |
From: John C. <joh...@am...> - 2009-12-17 19:40:42
|
Glen Beane wrote: > > So I have made two modifications to my makefile that seem to work around > my issue: > > first I tried a sleep here, (no luck) but switching it to the cat > helped, no more *.o not found: > > hmmSNP$(EXEEXT): $(hmmSNP_OBJECTS) $(hmmSNP_DEPENDENCIES) > cat *.o > /dev/null > @rm -f hmmSNP$(EXEEXT) > $(LINK) $(hmmSNP_OBJECTS) $(hmmSNP_LDADD) $(LIBS) > > then I was getting file not found errors for some of the mv –f > .deps/*.Tpo .deps/*.Po commands. I made the following modification > (added the cat command), and it took care of that problem... > > .c.o: > $(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< > cat $(DEPDIR)/$*.Tpo > /dev/null > mv -f $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po > Hi Glen, This trick works because when the shell expands "*.o" and "*.Tpo", it must reopen the directory, which fetches a fresh copy of the directory from the NFS server into the NFS client cache. I looked at whether it's feasible to have distmake send the list of prereqs over the wire to rpc.bldserver, so that rpc.bldserver can do NFS-cache-thwacking to make all prereqs visible. It looks like it would not be difficult. Specifically it does not require any changes to the source files of Gnu Make proper; it could be done entirely within the distmake extensions. It changes the client/server protocol a bit, I still need to study whether it's possible to do this such that newer server versions would still work with older client versions; that's a requirement for my site. So I'm tempted to add this feature now. > > then I got the following error that really doesn’t make sense to me > > mv -f .deps/hmmMatrix.Tpo .deps/hmmMatrix.Po > === building 'hmmMatrix.o' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-2' > > mv -f .deps/hmmUtil.Tpo .deps/hmmUtil.Po > === building 'hmmUtil.o' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-1' > > mv: `.deps/hmmMatrix.Tpo' and `.deps/hmmMatrix.Po' are the same file That means one of these is a symlink to the other (or they are both a symlink to the same thing?) For example $ touch foo $ ln -s foo bar $ mv bar foo mv: `bar' and `foo' are the same file So this probably is not related to distmake per se. John |
From: Glen B. <Gle...@ja...> - 2009-12-17 19:17:08
|
On 12/17/09 1:58 PM, "Glen Beane" <gle...@ja...> wrote: I'm having the sysadmin setup a test NFS volume that is mounted with the noac option to turn off file attribute caching. It worked fine with this test NFS volume, so the problem has to do with file attribute caching. -- Glen L. Beane Software Engineer The Jackson Laboratory Phone (207) 288-6153 |
From: Glen B. <Gle...@ja...> - 2009-12-17 18:58:38
|
I'm having the sysadmin setup a test NFS volume that is mounted with the noac option to turn off file attribute caching. -- Glen L. Beane Software Engineer The Jackson Laboratory Phone (207) 288-6153 |
From: Glen B. <Gle...@ja...> - 2009-12-17 17:46:23
|
So I have made two modifications to my makefile that seem to work around my issue: first I tried a sleep here, (no luck) but switching it to the cat helped, no more *.o not found: hmmSNP$(EXEEXT): $(hmmSNP_OBJECTS) $(hmmSNP_DEPENDENCIES) cat *.o > /dev/null @rm -f hmmSNP$(EXEEXT) $(LINK) $(hmmSNP_OBJECTS) $(hmmSNP_LDADD) $(LIBS) then I was getting file not found errors for some of the mv -f .deps/*.Tpo .deps/*.Po commands. I made the following modification (added the cat command), and it took care of that problem... .c.o: $(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< cat $(DEPDIR)/$*.Tpo > /dev/null mv -f $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po then I got the following error that really doesn't make sense to me mv -f .deps/hmmMatrix.Tpo .deps/hmmMatrix.Po === building 'hmmMatrix.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' mv -f .deps/hmmUtil.Tpo .deps/hmmUtil.Po === building 'hmmUtil.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' mv: `.deps/hmmMatrix.Tpo' and `.deps/hmmMatrix.Po' are the same file distmake[2]: *** [hmmMatrix.o] Error 1 distmake[2]: *** Waiting for unfinished jobs.... -- Glen L. Beane Software Engineer The Jackson Laboratory Phone (207) 288-6153 |
From: Glen B. <Gle...@ja...> - 2009-12-17 16:46:25
|
On 12/17/09 11:30 AM, "John Coiner" <joh...@am...> wrote: You're right -- it's not necessary to delete the directory to see the failure. Anytime the file is removed and recreated on host A, the failure is possible, where host B has a stale copy of the directory that it assumes is valid, and so it asks the server about the old file identifier and never knows about the new file identifier. So I gave an example that was more complicated than it needed to be. Oops. To solve this for the general case, you need to assume that directories have also been removed and re-created. You need to assume the more complicated example. You're right, this would be nice to integrate with distmake. Distmake can only refresh the NFS cached copy of files that it knows about, so this would rely on your Makefile dependencies being complete. Are they? I believe so. This really is a trivial build. There is one executable that depends on less than 20 object files. This makefile is generated by autotools, but the dependencies for the single executable are hmmSNP$(EXEEXT): $(hmmSNP_OBJECTS) $(hmmSNP_DEPENDENCIES) $ hmmSNP_OBJECTS contains all the .o files that I need the buildserver to reopen before trying to build the executable m_hmmSNP_OBJECTS = main.$(OBJEXT) dataIO.$(OBJEXT) CLI.$(OBJEXT) \ hmm_em.$(OBJEXT) hmmMatrix.$(OBJEXT) dataFilter.$(OBJEXT) \ filter.$(OBJEXT) path.$(OBJEXT) fill.$(OBJEXT) graph.$(OBJEXT) \ prune.$(OBJEXT) hmmUtil.$(OBJEXT) rand.$(OBJEXT) \ sort_max_trace.$(OBJEXT) csv.$(OBJEXT) \ temp_file_cleanup.$(OBJEXT) version.$(OBJEXT) \ state_sorting.$(OBJEXT) hmmSNP_OBJECTS = $(am_hmmSNP_OBJECTS) But again, this is only my simple test case for getting distmake working with our TORQUE (PBS) cluster. I don't really care about compiling this with distmake (It takes seconds with make -j 4). The real application will be for the Illumina Genome Analyzer pipeline which does work with SGE's qmake (so I am assuming the Illumina folks have complete dependencies in their Makefiles). This can take 12 hours to run on a single node. |
From: John C. <joh...@am...> - 2009-12-17 16:30:41
|
You're right -- it's not necessary to delete the directory to see the failure. Anytime the file is removed and recreated on host A, the failure is possible, where host B has a stale copy of the directory that it assumes is valid, and so it asks the server about the old file identifier and never knows about the new file identifier. So I gave an example that was more complicated than it needed to be. Oops. To solve this for the general case, you need to assume that directories have also been removed and re-created. You need to assume the more complicated example. You're right, this would be nice to integrate with distmake. Distmake can only refresh the NFS cached copy of files that it knows about, so this would rely on your Makefile dependencies being complete. Are they? John Glen Beane wrote: > > > > On 12/17/09 10:37 AM, "John Coiner" <joh...@am...> wrote: > > * Never delete directories in your build output tree, only delete the > files during a "make clean". I have not tried this but I believe it > would work. > > > By the way, I am not deleting any directories. The files it is not > finding are .o files that are getting built in the same directory as the > .c files. > > > -- > Glen L. Beane > Software Engineer > The Jackson Laboratory > Phone (207) 288-6153 > |
From: Glen B. <Gle...@ja...> - 2009-12-17 16:29:56
|
On 12/17/09 11:24 AM, "Christophe LYON" <chr...@st...> wrote: On 17.12.2009 17:01, Glen Beane wrote: > On 12/17/09 10:37 AM, "John Coiner" <joh...@am...> wrote: > > * Never delete directories in your build output tree, only delete the > files during a "make clean". I have not tried this but I believe it > would work. > > By the way, I am not deleting any directories. The files it is not > finding are .o files that are getting built in the same directory as the > .c files. > I have noted that files such as csv.o seem to get built twice (ie you get twice the same message: building 'csv.o' in /home/gbeane/hmmSNP/dev/src). Is this expected? no, this is not expected. If the program you are currently building does not take too long to rebuild from scratch, I'd second John's suggestion about prefixing your rules with "sleep 10", so as to make sure the problem comes from NFS, not from missing dependencies. You may also try something like "make -j 50" to exhibit dependencies problems. make -j 50 works fine. As does distmake with a single buildserver. I really don't think there are missing dependencies. The Makefiles are pretty simple and generated from gnu autotools. One executable and a bunch of .c files that can be compiled into object files independently. Christophe. ------------------------------------------------------------------------------ This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev _______________________________________________ Distmake-general mailing list Dis...@li... https://lists.sourceforge.net/lists/listinfo/distmake-general -- Glen L. Beane Software Engineer The Jackson Laboratory Phone (207) 288-6153 |
From: Christophe L. <chr...@st...> - 2009-12-17 16:25:08
|
On 17.12.2009 17:01, Glen Beane wrote: > On 12/17/09 10:37 AM, "John Coiner" <joh...@am...> wrote: > > * Never delete directories in your build output tree, only delete the > files during a "make clean". I have not tried this but I believe it > would work. > > By the way, I am not deleting any directories. The files it is not > finding are .o files that are getting built in the same directory as the > .c files. > I have noted that files such as csv.o seem to get built twice (ie you get twice the same message: building 'csv.o' in /home/gbeane/hmmSNP/dev/src). Is this expected? If the program you are currently building does not take too long to rebuild from scratch, I'd second John's suggestion about prefixing your rules with "sleep 10", so as to make sure the problem comes from NFS, not from missing dependencies. You may also try something like "make -j 50" to exhibit dependencies problems. Christophe. |
From: Glen B. <Gle...@ja...> - 2009-12-17 16:01:32
|
On 12/17/09 10:37 AM, "John Coiner" <joh...@am...> wrote: * Never delete directories in your build output tree, only delete the files during a "make clean". I have not tried this but I believe it would work. By the way, I am not deleting any directories. The files it is not finding are .o files that are getting built in the same directory as the .c files. -- Glen L. Beane Software Engineer The Jackson Laboratory Phone (207) 288-6153 |
From: Glen B. <Gle...@ja...> - 2009-12-17 15:56:40
|
On 12/17/09 10:37 AM, "John Coiner" <joh...@am...> wrote: Hi Glen, Are you using NFS? If so, what mount options? What OS do the distmake servers run on? Yes, I am using NFS with Linux clients. I got the mount options from the sysadmin (pasted below). We have two NFS shares that we would likely run distmake in. One (/home) is exported from the head node of our cluster (Linux), the other is a 4-node Isilon system (also NFS). /home is default (rw,addr=10.9.4.253,nfsvers=3,proto=tcp,mountproto=udp) /hpcdata has bigger frames (rw,rsize=65536,wsize=65536,bg,intr,addr=10.9.4.203,nfsvers=3,proto=tcp,mountproto=udp) I'd like to get this to work without having to touch Makefiles, since the end goal is to use this for a make-based analysis pipeline for an Illumina Genome Analyzer and we don't want to have to modify the Makefiles they produce. I'm not sure if these pipelines will have the same problems or not. In general these take hours, not seconds, to run. * Write a little program that takes in the list of prerequisites for each make rule. The program will stat() each prereq, and if the prereq doesn't exist, the program will re-open() every directory component from / on down to the file, forcing the NFS client to obtain a fresh copy of each directory from the NFS server. (Close-to-open coherency applies to directories as well.) Prefix each Make rule that has trouble with NFS coherency with a call to this program. This is what we do at my site. It's not ideal, you need to modify some of your Make rules, but it does work and it's faster than "sleep 2". Could we get something like this built into distmake? Maybe make it an optional step that the build server would do before executing a make rule? -- Glen L. Beane Software Engineer The Jackson Laboratory Phone (207) 288-6153 |
From: John C. <joh...@am...> - 2009-12-17 15:38:14
|
Hi Glen, Are you using NFS? If so, what mount options? What OS do the distmake servers run on? Some mount options are worse than others, for example on linux, "nocto" tells the NFS client to not even try to be coherent across hosts. You don't want "nocto" for a distmake setup. Even with "pretty good" mount options, there are still some coherency considerations with NFS on linux. NFS claims "close-to-open" coherency. This means if a file is written and closed on host A, and later opened on host B, then host B will see the newest contents of the file. This coherency model can break down. For example NFS fails to handle this situation (at least on linux) regardless of your mount options. This was a problem at my site until we found a workaround: Suppose host A removes directory dir/ containing file F. Then host A creates a new dir and a new dir/F. Shortly afterward, an attempt to open dir/F on host B can fail. Why's that? This is what I suspect is going on based on what I've read about linux NFS and some experiments: The new "dir" and the new "F" have different identifiers on the NFS server, compared to the old "dir" and the old "F", despite occupying the same pathnames. Now suppose host B's NFS client cache has a cached copy of the old dir and the old F. When you try to open F on host B, B's NFS client assumes that its cached copy of dir is valid, while it assumes its cached copy of dir/F might not be valid. So B will call the NFS server and ask about the status of old-F. The server says that old-F does not exist, so the open() will fail. (If B had reverified each directory in the path with the server, it would have picked up the new dir and realized that it needed to ask the server for the new F -- it makes a difference because new-F uses a different identifier on the server than old-F.) Did that makes sense? So this suggests some possible workarounds: * Use an NFS cache attribute timeout of 1 second (this is a mount option) and preface all your make rules with "sleep 2". This way, all stale data will expire from the NFS client cache before each rule starts running. I've tried this, it works. * Never delete directories in your build output tree, only delete the files during a "make clean". I have not tried this but I believe it would work. * Write a little program that takes in the list of prerequisites for each make rule. The program will stat() each prereq, and if the prereq doesn't exist, the program will re-open() every directory component from / on down to the file, forcing the NFS client to obtain a fresh copy of each directory from the NFS server. (Close-to-open coherency applies to directories as well.) Prefix each Make rule that has trouble with NFS coherency with a call to this program. This is what we do at my site. It's not ideal, you need to modify some of your Make rules, but it does work and it's faster than "sleep 2". I should probably write this up in a README.NFS and add it to the distmake distribution. Hope this helps. Cheers, John Glen Beane wrote: > my Makefile works fine with make -j, however distmake is trying to link all > the .o files without all of them being built. Occasionally it works > correctly. > > For example, I see a message that it is building csv.o, but then later when > it tries to build the executable it complains that csv.o odes not exist. It > not always the same object file(s) that it can not find. > > Any ideas? > > > gbeane@host> less submit_make.sh.o60580 > === building 'all-recursive' in /home/gbeane/hmmSNP/dev on host > 'cs-short-1' > > Making all in src > distmake[1]: Entering directory `/home/gbeane/hmmSNP/dev/src' > === building 'all-recursive' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-1' > > Making all in include > distmake[2]: Entering directory `/home/gbeane/hmmSNP/dev/src/include' > === building 'config.h' in /home/gbeane/hmmSNP/dev/src/include on host > 'cs-short-1' > > distmake all-am > === building 'all' in /home/gbeane/hmmSNP/dev/src/include on host > 'cs-short-1' > > distmake[3]: Entering directory `/home/gbeane/hmmSNP/dev/src/include' > === building 'config.h' in /home/gbeane/hmmSNP/dev/src/include on host > 'cs-short-1' > > distmake[3]: Leaving directory `/home/gbeane/hmmSNP/dev/src/include' > distmake[2]: Leaving directory `/home/gbeane/hmmSNP/dev/src/include' > distmake[2]: Entering directory `/home/gbeane/hmmSNP/dev/src' > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c > === building 'main.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT dataIO.o -MD -MP -MF .deps/dataIO.Tpo -c -o dataIO.o dataIO.c > === building 'dataIO.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT CLI.o -MD -MP -MF .deps/CLI.Tpo -c -o CLI.o CLI.c > === building 'CLI.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT hmm_em.o -MD -MP -MF .deps/hmm_em.Tpo -c -o hmm_em.o hmm_em.c > === building 'hmm_em.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT hmmMatrix.o -MD -MP -MF .deps/hmmMatrix.Tpo -c -o hmmMatrix.o > hmmMatrix.c > === building 'hmmMatrix.o' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-2' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT dataFilter.o -MD -MP -MF .deps/dataFilter.Tpo -c -o dataFilter.o > dataFilter.c > === building 'dataFilter.o' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-2' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT filter.o -MD -MP -MF .deps/filter.Tpo -c -o filter.o filter.c > === building 'filter.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT path.o -MD -MP -MF .deps/path.Tpo -c -o path.o path.c > === building 'path.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' > > mv -f .deps/dataFilter.Tpo .deps/dataFilter.Po > === building 'dataFilter.o' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-2' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT fill.o -MD -MP -MF .deps/fill.Tpo -c -o fill.o fill.c > === building 'fill.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' > > mv -f .deps/filter.Tpo .deps/filter.Po > === building 'filter.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT graph.o -MD -MP -MF .deps/graph.Tpo -c -o graph.o graph.c > === building 'graph.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' > > mv -f .deps/fill.Tpo .deps/fill.Po > === building 'fill.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT prune.o -MD -MP -MF .deps/prune.Tpo -c -o prune.o prune.c > === building 'prune.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' > > mv -f .deps/CLI.Tpo .deps/CLI.Po > === building 'CLI.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT hmmUtil.o -MD -MP -MF .deps/hmmUtil.Tpo -c -o hmmUtil.o hmmUtil.c > === building 'hmmUtil.o' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-1' > > mv -f .deps/main.Tpo .deps/main.Po > === building 'main.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c > === building 'rand.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' > > mv -f .deps/path.Tpo .deps/path.Po > === building 'path.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT sort_max_trace.o -MD -MP -MF .deps/sort_max_trace.Tpo -c -o > sort_max_trace.o sort_max_tra > ce.c > === building 'sort_max_trace.o' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-2' > > mv -f .deps/graph.Tpo .deps/graph.Po > === building 'graph.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT csv.o -MD -MP -MF .deps/csv.Tpo -c -o csv.o csv.c > === building 'csv.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' > > mv -f .deps/prune.Tpo .deps/prune.Po > === building 'prune.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT temp_file_cleanup.o -MD -MP -MF .deps/temp_file_cleanup.Tpo -c -o > temp_file_cleanup.o tem > p_file_cleanup.c > === building 'temp_file_cleanup.o' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-2' > > mv -f .deps/rand.Tpo .deps/rand.Po > === building 'rand.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' > > mv -f .deps/hmmUtil.Tpo .deps/hmmUtil.Po > === building 'hmmUtil.o' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-1' > > mv -f .deps/hmm_em.Tpo .deps/hmm_em.Po > === building 'hmm_em.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -c ../src/version.c > === building 'version.o' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-1' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT state_sorting.o -MD -MP -MF .deps/state_sorting.Tpo -c -o > state_sorting.o state_sorting.c > === building 'state_sorting.o' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-1' > > mv -f .deps/state_sorting.Tpo .deps/state_sorting.Po > === building 'state_sorting.o' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-1' > > mv -f .deps/temp_file_cleanup.Tpo .deps/temp_file_cleanup.Po > === building 'temp_file_cleanup.o' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-1' > > mv -f .deps/csv.Tpo .deps/csv.Po > === building 'csv.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' > > mv -f .deps/hmmMatrix.Tpo .deps/hmmMatrix.Po > === building 'hmmMatrix.o' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-1' > > mv -f .deps/sort_max_trace.Tpo .deps/sort_max_trace.Po > === building 'sort_max_trace.o' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-1' > > mv -f .deps/dataIO.Tpo .deps/dataIO.Po > === building 'dataIO.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' > > === building 'hmmSNP' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' > > gcc -std=gnu99 -Wall -pedantic -Werror -g -O2 -o hmmSNP main.o dataIO.o > CLI.o hmm_em.o hmmMatri > x.o dataFilter.o filter.o path.o fill.o graph.o prune.o hmmUtil.o rand.o > sort_max_trace.o csv.o te > mp_file_cleanup.o version.o state_sorting.o -lm > === building 'hmmSNP' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' > > gcc: graph.o: No such file or directory > gcc: prune.o: No such file or directory > gcc: sort_max_trace.o: No such file or directory > gcc: csv.o: No such file or directory > gcc: temp_file_cleanup.o: No such file or directory > distmake[2]: *** [hmmSNP] Error 1 > distmake[2]: Leaving directory `/home/gbeane/hmmSNP/dev/src' > distmake[1]: *** [all-recursive] Error 1 > distmake[1]: Leaving directory `/home/gbeane/hmmSNP/dev/src' > |
From: Glen B. <Gle...@ja...> - 2009-12-17 15:04:53
|
On 12/17/09 10:03 AM, "Sean Davis" <sd...@ma...> wrote: On Thu, Dec 17, 2009 at 10:00 AM, Glen Beane <Gle...@ja...> wrote: > my Makefile works fine with make -j, however distmake is trying to link all > the .o files without all of them being built. Occasionally it works > correctly. > > For example, I see a message that it is building csv.o, but then later when > it tries to build the executable it complains that csv.o odes not exist. It > not always the same object file(s) that it can not find. > > Any ideas? Is all this going on on a shared file system where all the paths are the same on all the hosts? yes, it is a shared file system |
From: Sean D. <sd...@ma...> - 2009-12-17 15:03:23
|
On Thu, Dec 17, 2009 at 10:00 AM, Glen Beane <Gle...@ja...> wrote: > my Makefile works fine with make -j, however distmake is trying to link all > the .o files without all of them being built. Occasionally it works > correctly. > > For example, I see a message that it is building csv.o, but then later when > it tries to build the executable it complains that csv.o odes not exist. It > not always the same object file(s) that it can not find. > > Any ideas? Is all this going on on a shared file system where all the paths are the same on all the hosts? Sean > > gbeane@host> less submit_make.sh.o60580 > === building 'all-recursive' in /home/gbeane/hmmSNP/dev on host > 'cs-short-1' > > Making all in src > distmake[1]: Entering directory `/home/gbeane/hmmSNP/dev/src' > === building 'all-recursive' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-1' > > Making all in include > distmake[2]: Entering directory `/home/gbeane/hmmSNP/dev/src/include' > === building 'config.h' in /home/gbeane/hmmSNP/dev/src/include on host > 'cs-short-1' > > distmake all-am > === building 'all' in /home/gbeane/hmmSNP/dev/src/include on host > 'cs-short-1' > > distmake[3]: Entering directory `/home/gbeane/hmmSNP/dev/src/include' > === building 'config.h' in /home/gbeane/hmmSNP/dev/src/include on host > 'cs-short-1' > > distmake[3]: Leaving directory `/home/gbeane/hmmSNP/dev/src/include' > distmake[2]: Leaving directory `/home/gbeane/hmmSNP/dev/src/include' > distmake[2]: Entering directory `/home/gbeane/hmmSNP/dev/src' > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c > === building 'main.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT dataIO.o -MD -MP -MF .deps/dataIO.Tpo -c -o dataIO.o dataIO.c > === building 'dataIO.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT CLI.o -MD -MP -MF .deps/CLI.Tpo -c -o CLI.o CLI.c > === building 'CLI.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT hmm_em.o -MD -MP -MF .deps/hmm_em.Tpo -c -o hmm_em.o hmm_em.c > === building 'hmm_em.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT hmmMatrix.o -MD -MP -MF .deps/hmmMatrix.Tpo -c -o hmmMatrix.o > hmmMatrix.c > === building 'hmmMatrix.o' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-2' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT dataFilter.o -MD -MP -MF .deps/dataFilter.Tpo -c -o dataFilter.o > dataFilter.c > === building 'dataFilter.o' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-2' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT filter.o -MD -MP -MF .deps/filter.Tpo -c -o filter.o filter.c > === building 'filter.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT path.o -MD -MP -MF .deps/path.Tpo -c -o path.o path.c > === building 'path.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' > > mv -f .deps/dataFilter.Tpo .deps/dataFilter.Po > === building 'dataFilter.o' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-2' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT fill.o -MD -MP -MF .deps/fill.Tpo -c -o fill.o fill.c > === building 'fill.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' > > mv -f .deps/filter.Tpo .deps/filter.Po > === building 'filter.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT graph.o -MD -MP -MF .deps/graph.Tpo -c -o graph.o graph.c > === building 'graph.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' > > mv -f .deps/fill.Tpo .deps/fill.Po > === building 'fill.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT prune.o -MD -MP -MF .deps/prune.Tpo -c -o prune.o prune.c > === building 'prune.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' > > mv -f .deps/CLI.Tpo .deps/CLI.Po > === building 'CLI.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT hmmUtil.o -MD -MP -MF .deps/hmmUtil.Tpo -c -o hmmUtil.o hmmUtil.c > === building 'hmmUtil.o' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-1' > > mv -f .deps/main.Tpo .deps/main.Po > === building 'main.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c > === building 'rand.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' > > mv -f .deps/path.Tpo .deps/path.Po > === building 'path.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT sort_max_trace.o -MD -MP -MF .deps/sort_max_trace.Tpo -c -o > sort_max_trace.o sort_max_tra > ce.c > === building 'sort_max_trace.o' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-2' > > mv -f .deps/graph.Tpo .deps/graph.Po > === building 'graph.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT csv.o -MD -MP -MF .deps/csv.Tpo -c -o csv.o csv.c > === building 'csv.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' > > mv -f .deps/prune.Tpo .deps/prune.Po > === building 'prune.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT temp_file_cleanup.o -MD -MP -MF .deps/temp_file_cleanup.Tpo -c -o > temp_file_cleanup.o tem > p_file_cleanup.c > === building 'temp_file_cleanup.o' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-2' > > mv -f .deps/rand.Tpo .deps/rand.Po > === building 'rand.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' > > mv -f .deps/hmmUtil.Tpo .deps/hmmUtil.Po > === building 'hmmUtil.o' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-1' > > mv -f .deps/hmm_em.Tpo .deps/hmm_em.Po > === building 'hmm_em.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -c ../src/version.c > === building 'version.o' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-1' > > gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 > -Wall -pedantic -Werror -g > -O2 -MT state_sorting.o -MD -MP -MF .deps/state_sorting.Tpo -c -o > state_sorting.o state_sorting.c > === building 'state_sorting.o' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-1' > > mv -f .deps/state_sorting.Tpo .deps/state_sorting.Po > === building 'state_sorting.o' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-1' > > mv -f .deps/temp_file_cleanup.Tpo .deps/temp_file_cleanup.Po > === building 'temp_file_cleanup.o' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-1' > > mv -f .deps/csv.Tpo .deps/csv.Po > === building 'csv.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' > > mv -f .deps/hmmMatrix.Tpo .deps/hmmMatrix.Po > === building 'hmmMatrix.o' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-1' > > mv -f .deps/sort_max_trace.Tpo .deps/sort_max_trace.Po > === building 'sort_max_trace.o' in /home/gbeane/hmmSNP/dev/src on host > 'cs-short-1' > > mv -f .deps/dataIO.Tpo .deps/dataIO.Po > === building 'dataIO.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' > > === building 'hmmSNP' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' > > gcc -std=gnu99 -Wall -pedantic -Werror -g -O2 -o hmmSNP main.o dataIO.o > CLI.o hmm_em.o hmmMatri > x.o dataFilter.o filter.o path.o fill.o graph.o prune.o hmmUtil.o rand.o > sort_max_trace.o csv.o te > mp_file_cleanup.o version.o state_sorting.o -lm > === building 'hmmSNP' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' > > gcc: graph.o: No such file or directory > gcc: prune.o: No such file or directory > gcc: sort_max_trace.o: No such file or directory > gcc: csv.o: No such file or directory > gcc: temp_file_cleanup.o: No such file or directory > distmake[2]: *** [hmmSNP] Error 1 > distmake[2]: Leaving directory `/home/gbeane/hmmSNP/dev/src' > distmake[1]: *** [all-recursive] Error 1 > distmake[1]: Leaving directory `/home/gbeane/hmmSNP/dev/src' > > -- > Glen L. Beane > Software Engineer > The Jackson Laboratory > Phone (207) 288-6153 > > > > ------------------------------------------------------------------------------ > This SF.Net email is sponsored by the Verizon Developer Community > Take advantage of Verizon's best-in-class app development support > A streamlined, 14 day to market process makes app distribution fast and easy > Join now and get one step closer to millions of Verizon customers > http://p.sf.net/sfu/verizon-dev2dev > _______________________________________________ > Distmake-general mailing list > Dis...@li... > https://lists.sourceforge.net/lists/listinfo/distmake-general > |
From: Glen B. <Gle...@ja...> - 2009-12-17 15:00:44
|
my Makefile works fine with make -j, however distmake is trying to link all the .o files without all of them being built. Occasionally it works correctly. For example, I see a message that it is building csv.o, but then later when it tries to build the executable it complains that csv.o odes not exist. It not always the same object file(s) that it can not find. Any ideas? gbeane@host> less submit_make.sh.o60580 === building 'all-recursive' in /home/gbeane/hmmSNP/dev on host 'cs-short-1' Making all in src distmake[1]: Entering directory `/home/gbeane/hmmSNP/dev/src' === building 'all-recursive' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' Making all in include distmake[2]: Entering directory `/home/gbeane/hmmSNP/dev/src/include' === building 'config.h' in /home/gbeane/hmmSNP/dev/src/include on host 'cs-short-1' distmake all-am === building 'all' in /home/gbeane/hmmSNP/dev/src/include on host 'cs-short-1' distmake[3]: Entering directory `/home/gbeane/hmmSNP/dev/src/include' === building 'config.h' in /home/gbeane/hmmSNP/dev/src/include on host 'cs-short-1' distmake[3]: Leaving directory `/home/gbeane/hmmSNP/dev/src/include' distmake[2]: Leaving directory `/home/gbeane/hmmSNP/dev/src/include' distmake[2]: Entering directory `/home/gbeane/hmmSNP/dev/src' gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 -Wall -pedantic -Werror -g -O2 -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c === building 'main.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 -Wall -pedantic -Werror -g -O2 -MT dataIO.o -MD -MP -MF .deps/dataIO.Tpo -c -o dataIO.o dataIO.c === building 'dataIO.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 -Wall -pedantic -Werror -g -O2 -MT CLI.o -MD -MP -MF .deps/CLI.Tpo -c -o CLI.o CLI.c === building 'CLI.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 -Wall -pedantic -Werror -g -O2 -MT hmm_em.o -MD -MP -MF .deps/hmm_em.Tpo -c -o hmm_em.o hmm_em.c === building 'hmm_em.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 -Wall -pedantic -Werror -g -O2 -MT hmmMatrix.o -MD -MP -MF .deps/hmmMatrix.Tpo -c -o hmmMatrix.o hmmMatrix.c === building 'hmmMatrix.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 -Wall -pedantic -Werror -g -O2 -MT dataFilter.o -MD -MP -MF .deps/dataFilter.Tpo -c -o dataFilter.o dataFilter.c === building 'dataFilter.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 -Wall -pedantic -Werror -g -O2 -MT filter.o -MD -MP -MF .deps/filter.Tpo -c -o filter.o filter.c === building 'filter.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 -Wall -pedantic -Werror -g -O2 -MT path.o -MD -MP -MF .deps/path.Tpo -c -o path.o path.c === building 'path.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' mv -f .deps/dataFilter.Tpo .deps/dataFilter.Po === building 'dataFilter.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 -Wall -pedantic -Werror -g -O2 -MT fill.o -MD -MP -MF .deps/fill.Tpo -c -o fill.o fill.c === building 'fill.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' mv -f .deps/filter.Tpo .deps/filter.Po === building 'filter.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 -Wall -pedantic -Werror -g -O2 -MT graph.o -MD -MP -MF .deps/graph.Tpo -c -o graph.o graph.c === building 'graph.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' mv -f .deps/fill.Tpo .deps/fill.Po === building 'fill.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 -Wall -pedantic -Werror -g -O2 -MT prune.o -MD -MP -MF .deps/prune.Tpo -c -o prune.o prune.c === building 'prune.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' mv -f .deps/CLI.Tpo .deps/CLI.Po === building 'CLI.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 -Wall -pedantic -Werror -g -O2 -MT hmmUtil.o -MD -MP -MF .deps/hmmUtil.Tpo -c -o hmmUtil.o hmmUtil.c === building 'hmmUtil.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' mv -f .deps/main.Tpo .deps/main.Po === building 'main.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 -Wall -pedantic -Werror -g -O2 -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c === building 'rand.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' mv -f .deps/path.Tpo .deps/path.Po === building 'path.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 -Wall -pedantic -Werror -g -O2 -MT sort_max_trace.o -MD -MP -MF .deps/sort_max_trace.Tpo -c -o sort_max_trace.o sort_max_tra ce.c === building 'sort_max_trace.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' mv -f .deps/graph.Tpo .deps/graph.Po === building 'graph.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 -Wall -pedantic -Werror -g -O2 -MT csv.o -MD -MP -MF .deps/csv.Tpo -c -o csv.o csv.c === building 'csv.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' mv -f .deps/prune.Tpo .deps/prune.Po === building 'prune.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 -Wall -pedantic -Werror -g -O2 -MT temp_file_cleanup.o -MD -MP -MF .deps/temp_file_cleanup.Tpo -c -o temp_file_cleanup.o tem p_file_cleanup.c === building 'temp_file_cleanup.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-2' mv -f .deps/rand.Tpo .deps/rand.Po === building 'rand.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' mv -f .deps/hmmUtil.Tpo .deps/hmmUtil.Po === building 'hmmUtil.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' mv -f .deps/hmm_em.Tpo .deps/hmm_em.Po === building 'hmm_em.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 -Wall -pedantic -Werror -g -O2 -c ../src/version.c === building 'version.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' gcc -DHAVE_CONFIG_H -I. -I../src/include -I../src/include -std=gnu99 -Wall -pedantic -Werror -g -O2 -MT state_sorting.o -MD -MP -MF .deps/state_sorting.Tpo -c -o state_sorting.o state_sorting.c === building 'state_sorting.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' mv -f .deps/state_sorting.Tpo .deps/state_sorting.Po === building 'state_sorting.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' mv -f .deps/temp_file_cleanup.Tpo .deps/temp_file_cleanup.Po === building 'temp_file_cleanup.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' mv -f .deps/csv.Tpo .deps/csv.Po === building 'csv.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' mv -f .deps/hmmMatrix.Tpo .deps/hmmMatrix.Po === building 'hmmMatrix.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' mv -f .deps/sort_max_trace.Tpo .deps/sort_max_trace.Po === building 'sort_max_trace.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' mv -f .deps/dataIO.Tpo .deps/dataIO.Po === building 'dataIO.o' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' === building 'hmmSNP' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' gcc -std=gnu99 -Wall -pedantic -Werror -g -O2 -o hmmSNP main.o dataIO.o CLI.o hmm_em.o hmmMatri x.o dataFilter.o filter.o path.o fill.o graph.o prune.o hmmUtil.o rand.o sort_max_trace.o csv.o te mp_file_cleanup.o version.o state_sorting.o -lm === building 'hmmSNP' in /home/gbeane/hmmSNP/dev/src on host 'cs-short-1' gcc: graph.o: No such file or directory gcc: prune.o: No such file or directory gcc: sort_max_trace.o: No such file or directory gcc: csv.o: No such file or directory gcc: temp_file_cleanup.o: No such file or directory distmake[2]: *** [hmmSNP] Error 1 distmake[2]: Leaving directory `/home/gbeane/hmmSNP/dev/src' distmake[1]: *** [all-recursive] Error 1 distmake[1]: Leaving directory `/home/gbeane/hmmSNP/dev/src' -- Glen L. Beane Software Engineer The Jackson Laboratory Phone (207) 288-6153 |
From: Glen B. <Gle...@ja...> - 2009-12-17 15:00:10
|
I actually discovered it is a problem with my "rsh replacement" script that is trying to start the remote bldservers using the TORQUE task manager API (I'm trying to run this through the TORQUE batch system). I do not get the error if I use ssh to startup the remote bldservers, but the problem is then TORQUE does not own the processes and can not clean them up if the batch job is terminated before distmake shuts down the remote servers. I'll keep working on finding a solution. On 12/17/09 9:47 AM, "John Coiner" <joh...@am...> wrote: Glen When running with persistent rpc.bldserver daemons, I've generally seen that error when one of the daemons is offline. (For example if the server has rebooted and the daemon does not start up again, and then a client tries to connect.) Hope this helps. John Glen Beane wrote: > I see a lot of " RPC: Remote system error - Connection refused" messages > when I use distmake, but the build seems to complete OK. What causes these > error messages, and if the errors are recoverable is there a way to suppress > the message? We'll be using these to drive the execution of an analysis > pipeline and I'd rather not have the biostatisticians see all of these error > messages. > > > -- Glen L. Beane Software Engineer The Jackson Laboratory Phone (207) 288-6153 |
From: John C. <joh...@am...> - 2009-12-17 14:47:56
|
Glen When running with persistent rpc.bldserver daemons, I've generally seen that error when one of the daemons is offline. (For example if the server has rebooted and the daemon does not start up again, and then a client tries to connect.) Hope this helps. John Glen Beane wrote: > I see a lot of " RPC: Remote system error - Connection refused" messages > when I use distmake, but the build seems to complete OK. What causes these > error messages, and if the errors are recoverable is there a way to suppress > the message? We'll be using these to drive the execution of an analysis > pipeline and I'd rather not have the biostatisticians see all of these error > messages. > > > |
From: Glen B. <Gle...@ja...> - 2009-12-17 02:32:03
|
I see a lot of " RPC: Remote system error - Connection refused" messages when I use distmake, but the build seems to complete OK. What causes these error messages, and if the errors are recoverable is there a way to suppress the message? We'll be using these to drive the execution of an analysis pipeline and I'd rather not have the biostatisticians see all of these error messages. -- Glen L. Beane Software Engineer The Jackson Laboratory Phone (207) 288-6153 |
From: Christophe L. <chr...@st...> - 2008-10-17 15:14:09
|
Hi Cristian, Thanks for using distmake! > cserpell@pc-3:~/disttest$ distmake all I guess you have forgotten to use "-j": $ distmake -j all Christophe. |
From: Cristián S. <cse...@di...> - 2008-10-17 15:06:23
|
Hi I have been trying to use distmake, but it sends every job to the same server, resulting in waiting one by one. We have 5 machines here running the bldserver , and I wrote a .bldhosts with a list of them. Do you know what 'm doing wrong? This is my Makefile: task1: cat file1 > file1.new task2: cat file2 > file2.new task3: cat file3 > file3.new task4: sort --key=1,2 mid-size-file.txt > /dev/null task5: sort --key=3,3 mid-size-file.txt > /dev/null task6: sort --key=5,5 mid-size-file.txt > /dev/null all: task1 task2 task3 task4 task5 task6 @echo Finished And this is my .bldhosts : pc-0.chile.cl pc-1.chile.cl pc-2.chile.cl pc-3.chile.cl pc-4.chile.cl and this is an example output (without debug): cserpell@pc-3:~/disttest$ distmake all cat file1 > file1.new === building 'task1' in /cserpell/disttest on host 'pc-4.chile.cl' cat file2 > file2.new === building 'task2' in /cserpell/disttest on host 'pc-4.chile.cl' cat file3 > file3.new === building 'task3' in /cserpell/disttest on host 'pc-4.chile.cl' sort --key=1,2 mid-size-file.txt > /dev/null === building 'task4' in /cserpell/disttest on host 'pc-4.chile.cl' sort --key=3,3 mid-size-file.txt > /dev/null === building 'task5' in /cserpell/disttest on host 'pc-4.chile.cl' sort --key=5,5 mid-size-file.txt > /dev/null === building 'task6' in /cserpell/disttest on host 'pc-4.chile.cl' === building 'all' in /cserpell/disttest on host 'pc-4.chile.cl' Finished Thanks a lot! Cristián Serpell |