Re: [Tack-devel] Prime Mover commands

[tim k. <gt...@di...>]

At 6:56 PM -0400 8/2/06, David Given wrote:
>Unfortunately, that's not what the intermediate cache is for --- it's not the
>final destination of the built files, it's where pm stores (for its own
>nefarious purposes) the *intermediate files* it uses to build things. It's not
>supposed to be public, which is why it defaults to a hidden directory. (In
>fact, the only reason the files in there have human-readable names is to make
>debugging easier. The first version called everything 1.o, 2.o, 3.o, 4.a,
>etc.)

What happens in a large build, like a kernel or userland worldbuild, where
the .o files can number into the hundreds and even thousands of files,
normally distributed throughout the build tree?  If the intermediate cache
is the location for all of these, and the README for pm indicates that the
intermediate cache is used to speed building (using the indexing), this
centralized location is going to be extremely full.

Using ls -a /tmp/ack-temp/pmcache > temp.txt and determining the number of
lines in temp.txt, the pmcache directory for ACK appears to have 1931
items.

I've done some reading on incremental linking (ild) and even got my local
library to get me a copy of Russel Quong and Mark Linton's seminal paper on
incremental linking, in order to research techniques to decrease
compilation time.  I'm in favor of your solution to use an intermediate
cache, and I like the idea of indexing the builds, but if the intermediate
cache is flat instead of hierarchical, this makes me concerned about larger
builds as described above.  Given the number of files from the ACK build, a
userland world build would be staggering.

>From my perspective, I don't mind having the indexed .o files in my
designated object directory; others might find it better to have only the
actual .o files used to create the final executable residing in their
object directory.  A hierarchical intermediate cache could meet both needs.


Is it possible to modify outputs for cfile somewhat like:

ext_cfile = cfile {
        class = "ext_cfile",
        outputs = {"lib/libsa/%U%-%I%.o"},
}

so that the cached files can have relative paths?

>If you want access to anything pm makes, you have to export it using 'install
>= pm.install()'. If you want access to all .o files, you should be able to do
>this by subclassing the cfile{} rule.
>
>[long example removed --- it actually turned out to be more interesting than I
>expected, so I checked it in. See:
>
>http://svn.sourceforge.net/viewvc/*checkout*/primemover/pm/test-pmfiles/complex
>-c-program.pm?revision=33

This example to me helps justify extending the cfile definition to
facilitate adding objects separate from groups.  There are two cfiles that
add test4.c, with different CDEFINES, but in order to build either one the
entire program has to be built.  The alternative pmfile I would envision
would be something akin to:

include "lib/c.pm"

local d = "test-source/"

ext_cfile = simple_with_clike_dependencies {

        class = "ext_cfile",
        outputs = {o.."%U%-%I%.o"},
        inputs = inputs..".c",
        inputs = d..inputs,
}


test = ext_cfile ("test")
test1 = ext_cfile ("test1")
test2 = ext_cfile ("test2")
test3 = ext_cfile ("test3")

-- Compile these two files with a modified definition of CDEFINES ---
-- they should get both test1 and test4 defined.

test4A = ext_cfile {
            CDEFINES = {PARENT, "-Dtest4=test4A"},
           "test4",
},

test4B = ext_cfile {
           CDEFINES = {PARENT, "-Dtest4=test4B"},
           "test4",
},

default = cprogram {
          CDEFINES = {"-Dtest1=fnord"},

           test,
           test1,
           test2,
           test3,
           test4A,
           test4B,

           install = pm.install("test-source/complex-c-program")
}

which would allow building 4A and 4B seperately, without building default.
The ext_cfile won't run, of course, because the concatentation doesn't work
(I'm not up to speed on modifying inputs yet), but overall the goal would
be to simplify specifying target source file and object file.  Not that
it's complicated; I'm just trying to reduce the amount of typing and number
of characters in the pmfile for ease of reading.

>I also found a bug in c.pm, so you'll need a new version of that as well if
>you want rebuilding to work properly.]

Ok, thanks for the heads up.  Unfortunately, at least on NetBSD, Subversion
(svn) requires Apache Portable Runtime, which appears to require installing
Apache httpd, which I don't see a need to do, so I have not yet been able
to sync up to files committed to the subversion tree at sourceforge.

>> How do I do a "Save and Restore" for the pm.intermediate_cache_dir setting,
>> so that it is local in scope wrt to the pmfile, or even the object being
>> built?
>
>You can't, I'm afraid --- once execution of your pmfile script has finished,
>whatever value you last set it as sticks. It *might* be possible to trick pm
>into running code where you change it, but all that's likely to happen is
>horribly confusing pm.

Then perhaps the pmcache/outputs connection could be modified to be more
hierarchical.  Just a thought.

thanks,
tim

Gregory T. (tim) Kelly
Owner
Dialectronics.com

P.O. Box 606
Newberry, SC 29108

"Anything war can do, peace can do better."  -- Bishop Desmond Tutu