Finding CC compiler if it is not on the PATH

Help
2004-06-08
2004-08-25
  • Shankar Unni
    Shankar Unni
    2004-06-08

    Because we have multiple Visual C++ versions installed, we don't put any of them in the system PATH.

    How do we make the "msvc" compiler find the one we want it to pick up? It just blindly forks "cl". Is there any way to set a PATH for the executed task?

     
    • Curt Arnold
      Curt Arnold
      2004-06-08

      The most common way is to run the appropriate "vcvars32.bat" or setenv.bat before running Ant.  The bat file is placed in the same directory as the CL.EXE (usually \Program Files\Microsoft Visual Studio*\VC*\bin") and temporarily sets the PATH, INCLUDE and LIB environment variables that CL needs to execute properly.

      p.s. There is an open enhancement request to support more than one version of a compiler in a single build script, but it is a whole lot more difficult than it seems and it will be undertaken as part of a refactoring needed to support general purpose cross-compilation.

       
      • Shankar Unni
        Shankar Unni
        2004-06-10

        > The most common way is to run the appropriate
        > "vcvars32.bat" or setenv.bat before running Ant.

        Right. In fact, I ended up creating a template antrc_pre.bat and .antrc (for cygwin bash users) for my users, but it's still a manual step to make them put it in their home directory.

        If you're refactoring, it'll be really useful if you can provide any or all of the following:

        1. An <env> hook to export additional environment variables before forking off the compilers, linkers, etc., to be passed on to the "env" task.

        This will also allow users to add to the PATH, by setting it in an "env" clause with the original ${java.library.path} plus whatever components.

        2. Provide standard properties for "cl", etc., that can be overridden by the user before <typedef>ing the cpptasks resources, which would be used from the <cc> task.

        The importance of this is that all this can be scripted in Ant itself, making the build script  independent of the user's environment (which is a generally good thing in controlled build environments).

        Any deviations from the build file should only come from explicit command-line Defines, not implicitly from the environment..

         
        • clem
          clem
          2004-07-29

          I'd like to second the call for a <env> tag.  It would help a great deal with a problem I'm currently facing with cpptask on a Windows 2000 platform. 

          One particular library that we're compiling has so many include directories that it's going over the character limit of the command-line.  One way around this would be to set the INCLUDE environment variable.  Is there any way around this?

           
          • saber850
            saber850
            2004-07-29

            I too am encountering problems w/ going over the character limit of the command line on WinXP.  I'm getting the error:
              Extremely long file name, can't fit on command line

            I thought at one point I noticed cpptask putting the command line in a file and handing that off to cl.  Perhaps that functionality has been lost from v1.0b2 (I'm now using 1.0b3)?

             
            • saber850
              saber850
              2004-07-30

              I found the symptom of my character-limit issue.  It has something to do w/ specifying include paths that don't exist.  My setup dynamically generates include paths, and so there were some args like:
              /Ih:\path\to\my\project\${SOME_PROJECT_PATH}
              because SOME_PROJECT_PATH was not defined.  Somehow this resulting in the error listed above.  Adding some wits to my build script so it doesn't specify such paths fixed it.  And it seems cpptasks is smart enough to split the compilation line if the command line length is going to be exceeded.  So it compiles a few files at a time--pretty slick.
              As for the linker, cpptasks _does_ use a separate file.  It packs all the linker args into a file w/ a .rsp extension (placed in the output dir).
              So far so good.

               
              • clem
                clem
                2004-07-30

                Good info -- we have a similar build setup where paths are dynamically generated.  However, problem we're having is not too many imaginary include paths, but too many actual ones.  We run out of space before we even get to the source files. 

                Finally found a workaround by getting the <env> tag to work.  Turns out that the tag has to be placed with a <compiler> tag to work, whereas I was placing it within the <cc> tag.  So here's how to set your INCLUDE variable dynamically:

                <cc>
                    <compiler name="msvc">
                        <env key="INCLUDE" path="${inc.path}"/>
                    </compiler>
                </cc>

                Bear in mind, there's also a fixed limit (bless you, Microsoft) as to how large the environment can be.  The exact number escapes me, but it is significantly larger than the command line's capacity.

                On this train of thought, another nice feature for <cc> would be the ability to identify when it has been passed duplicates in the include paths and filter them out automagically.

                 
                • saber850
                  saber850
                  2004-07-30

                  Thanks for the info regarding the env tag!  I didn't know it was possible.  That may make some of my stuff _much_ simpler.
                  Any chance I can prepend/append to the existing one (ie. INCLUDE=$INCLUDE:/more/paths) ?  For the INCLUDE env var it's not such an issue.  But I need to modify PATH.  Setting it to my stuff will break everything else (MS compiler, etc).
                  Leave it to M$ to take something that works well and royally screw it up.
                  I agree w/ the duplicate path elimination--would be very helpfull.

                   
                  • clem
                    clem
                    2004-07-30

                    Something along the following should do what you need:

                    <property environment="env"/>
                    <property name="cc.path" value="${env.PATH};${path.1};${path.2};${path.etc}"/>

                    <cc>
                    <compiler name="msvc">
                       <env key="PATH" path="${cc.path}"/>
                    </compiler>
                    </cc>

                     
                    • saber850
                      saber850
                      2004-07-30

                      That looks good; I'll give it a try next week.  Thanks a lot!

                       
                    • Markus Wankus
                      Markus Wankus
                      2004-08-23

                      Has anyone gotten this to work?  I can't seem to get any environment propagating into my compiler or cc task.  Debugging shows that env is  always null for my tasks.

                      Mark.

                       
                      • clem
                        clem
                        2004-08-23

                        It's been working for me.  Make sure the <env> tag is included between your <compiler></compiler> tags.  Can you post an example?

                         
                        • Markus Wankus
                          Markus Wankus
                          2004-08-24

                          Hopefully I am doing something wrong, but here is my sample code:

                          <?xml version="1.0" encoding="UTF-8"?>
                          <project name="myTest" default="build" basedir="..">
                              <taskdef resource="cpptasks.tasks"/>
                              <typedef resource="cpptasks.types"/>

                              <property environment="env"/>
                              <property name="path.to.tool" value="C:/PROGRA~1/DSPFAC~1/bin"/>
                              <property name="cc.path" value="${path.to.tool};${env.PATH}"/>

                              <target name="build">
                                  <cc outfile="outputfile">
                                      <compiler classname="com.dspfactory.myCustomCompiler">
                                          <env key="PATH" path="${cc.path}"/>
                                      </compiler>
                                  <fileset dir="C:/__TEMP_CRAP__/Test" includes="*.ss"/>
                                  </cc>
                              </target>
                             
                          </project>

                          I have mucked with names a bit to obfuscate our corporate info a little, but this is what I am doing.  myCustomCompiler is a simple compiler I made very similar to the TI Clxx DSP compiler/linker included with cpptasks.  It calls an executable to compile the code which resides in the directory "C:/PROGRA~1/DSPFAC~1/bin", but when I run Ant I get an error 2, like it can't find the executable.  The compiler is trying to execute the appropriate command line from what I can see.

                          Mark.

                           
                          • Curt Arnold
                            Curt Arnold
                            2004-08-24

                            I can't offer much immediate encouragement.  The support for nested environment variables was not completed due to lack of interest by the original requestor a couple of years ago.  Obviously there is interest now and it would fit into a significant refactoring that I have good intentions to do if funded or if I have some downtime.

                            Nested environment variables works for some degree for some compilers (but I don't believe linker support was completed), but I never tested the TI compilers.  

                             
                            • Markus Wankus
                              Markus Wankus
                              2004-08-24

                              No problem.  I guess I can always add more support in my custom compiler/linker by looking at the source of another one that does support it. 

                              Thanks,
                              Mark.

                               
                            • Markus Wankus
                              Markus Wankus
                              2004-08-25

                              Well, I thought I could figure this out, but I am ready to throw my damn PC out the window at this point.  I don't think this is a cpptasks issue - but an Ant issue.

                              I have stepped through code until I am blue in the face.  I end up in the Execute() task provided by Ant.  I can see that it is calling Runtime.exec() on my executable, with a full environment, and all appropriate command line args.  The PATH is set properly in the env String[] that is passed to the exec() method, and it still fricking bails with an IOException/error=2, indicating it cannot find the executable.  I think I have officially gone insane.....  I give up!  Serenity Now! Serenity Now!

                              Mark.

                               
                              • saber850
                                saber850
                                2004-08-25

                                My workaround was to use a script that called my compiler.
                                I suspect the ENV you're specifying in Ant is not applied to the Ant process, but to the spawned one.  That means Ant's path isn't changed so it can't find the program.  I found the spawned process _does_ get the env that's specified.
                                The Java docs for Execute make this clear.

                                 
                                • Markus Wankus
                                  Markus Wankus
                                  2004-08-25

                                  Yes, it appears this is the case.  Although - my Java docs for Execute say nothing at all.  Where did you find the Javadocs that make it clear?  I would like to read them...

                                  Thanks,
                                  Mark.

                                   
                          • clem
                            clem
                            2004-08-24

                            One thing I'd suggest is using the key "Path" rather than "PATH".  I'm not sure how exactly Ant handles case-sensitive issues for Win32 environments, but I've heard of issues creeping up in the past.

                             
                            • Markus Wankus
                              Markus Wankus
                              2004-08-24

                              No luck.  Can you post an example?  I am debugging with Eclipse, and when I put an <env tag iinside my cc task, it is calling the CCTask.addEnv() method, but no matter what I put in there, the "key" and "value" fields of the Environment.Variable object are always null.

                              It is very weird...  I can see the Environment.Variable object set it's key/value pairs, *AFTER* the CCTask calls addEnv().

                              Mark.

                               
                              • clem
                                clem
                                2004-08-24

                                Afraid I've only just realized what you are trying to do.  All my previous experience using <env> was in setting compiler specific environment variables.  I've never tried to set the path to the toolset I was trying to use with <cc>.

                                Is it possible for you to set the path in a script which wraps the call to ant?

                                 
                                • Markus Wankus
                                  Markus Wankus
                                  2004-08-25

                                  Well, I am *almost* there.  By implementing the changeEnvironment() method, now my custom compiler actually gets the environment variable change when I place an <env> tag inside of my <compiler> tag.  It is still not quite working but I think I am close to figuring out why...

                                  I don't want to wrap it in a calling script because I am trying to integrate cpptasks into Eclipse as a builder and everything will be done programmatically from Java.  I want to auto-generate the build file as well.

                                  Thanks for your help.
                                  Mark.

                                   
                • Curt Arnold
                  Curt Arnold
                  2004-07-30

                  I have never found a definitive reference on the allowable command line length for the various Windows flavors.  I believe that the current length is conservative for the NT derived Windows platforms.  If someone could provide a decent reference, I could tweak those values.

                  Using an env element to pass include paths may not make those paths available to the dependency analyzer.  It would be better to enhance the Microsoft compiler adapters so it better handles cases where the compile switches exhaust the available command line length.

                   
                  • clem
                    clem
                    2004-07-30

                    Here's what Google provided on the subject.  It's not official Microsoft documentation, so take it with a few grains of salt (courtesy of http://www.ss64.com/nt/cmd.html\):

                    Under Windows NT, the command line is limited to 256 characters.
                    Under Windows 2000, the command line is limited to 2046 characters.
                    Under Windows XP, the command line is limited to 8190 characters