#252 Problems using foamExec in Ubuntu

new
nobody
None
normal
minor
Always
none
Bug
Ubuntu
Linux
other
2014-08-15
2014-08-11
Bruno Santos
No

Original issue was reported on the forum, associated to what was thought to be an installation problem: http://www.cfd-online.com/Forums/openfoam-installation/137496-source-fe3-bashrc-parallel-run.html

These issues are mostly related to Ubuntu using dash for /bin/sh, instead of bash.

Example from that thread:

$ foamJob -p -s icoFoam

Parallel processing using SYSTEMOPENMPI with 6 processors
Executing: mpirun -np 6 /opt/foam/foam-extend-3.1/bin/foamExec icoFoam -parallel | tee log
Sourcing: /opt/foam/foam-extend-3.1/etc/prefs.sh
Sourcing: /opt/foam/foam-extend-3.1/etc/prefs.sh
Sourcing: /opt/foam/foam-extend-3.1/etc/prefs.sh
Sourcing: /opt/foam/foam-extend-3.1/etc/prefs.sh
Sourcing: /opt/foam/foam-extend-3.1/etc/prefs.sh
Sourcing: /opt/foam/foam-extend-3.1/etc/prefs.sh
/opt/foam/foam-extend-3.1/bin/foamExec: 400: /opt/foam/foam-extend-3.1/etc/bashrc: [[: not found
/opt/foam/foam-extend-3.1/bin/foamExec: 400: /opt/foam/foam-extend-3.1/etc/bashrc: [[: not found
/opt/foam/foam-extend-3.1/bin/foamExec: 400: /opt/foam/foam-extend-3.1/etc/bashrc: [[: not found
Sourcing: /opt/foam/foam-extend-3.1/etc/settings.sh
/opt/foam/foam-extend-3.1/bin/foamExec: 400: /opt/foam/foam-extend-3.1/etc/bashrc: [[: not found
Sourcing: /opt/foam/foam-extend-3.1/etc/settings.sh
/opt/foam/foam-extend-3.1/bin/foamExec: 400: /opt/foam/foam-extend-3.1/etc/bashrc: [[: not found
/opt/foam/foam-extend-3.1/bin/foamExec: 400: /opt/foam/foam-extend-3.1/etc/bashrc: [[: not found
Sourcing: /opt/foam/foam-extend-3.1/etc/settings.sh

Related:

/opt/foam-extend-3.0/bin/foamExec: 398: /opt/foam-extend-3.0/etc/bashrc: [[: not found
/opt/foam-extend-3.0/bin/foamExec: 21: export: -pthread: bad variable name

After fully diagnosing the problem, it all boils down to 3 issues:

  1. foamExec is broken. The arguments given to it are not passed to the final exec.
  2. etc/bashrc has got non-universal sh-type shell script code. [[ is usually only available in Bash and derived shells.
  3. The custom build of Open-MPI for foam-extend has at least one broken export in the shell environment script, which is why the error about -pthread occurs.

A merge request will will pushed soon.

Discussion

  • Dominik Christ
    Dominik Christ
    2014-08-12

    One question: Are you running on Ubuntu? If yes, could you please test is replacing /bin/sh with /bin/bash in foamJob solves the problem as well?

    The undelying reasoning is that Ubuntu uses /bin/dash for /bin/sh instead /bin/bash and this has substantial limitations.

     
  • Bruno Santos
    Bruno Santos
    2014-08-12

    I am very well aware of the limitations that Dash has got, as well as the limitations that Bash has got ;)
    And I do vaguely remember reading that Zsh and Fish are actually a couple of the best shells there are, but I haven't seen it being used all that much :)

    The reason why Ubuntu is using Dash is thoroughly explained on this wiki page: https://wiki.ubuntu.com/DashAsBinSh - and I'm guessing that there are system administrators that have Ubuntu on their systems and that will not be pleased with the requirement of foam-extend needing Bash to be the default shell. Because, as this wiki page implies, it means that the booting process and other scripted parts of the system will not work with Bash as fast as they do with Dash.

    Which would lead me to suggest that if Bash is now a strict requirement for using foam-extend, then it's best to change the shebang line in all shell scripts in the source code to use "/bin/bash", to avoid future confusion ;)

    The reason why I reported this and provided this particular solution of downgrading the scripting requirements to the lowest common shell features, was because it has been a tradition for sometime now in OpenFOAM to have all scripts always compatible with the common shell features.

     
  • I think Dash in Ubuntu is again a case of "Not invented here" (for which Ubuntu has a bit of a tendency). The only hard data on their claim "dash is faster and smaller" that I found is this quite old and german article http://www.documentroot.net/linux/bash-vs-dash and it says "improvements are nothing to write home about"

    But of course you're right about /bin/bash in the shebang-line

    And I think bash should be a requirement for foam-extend but other shells should be able to live with it: https://sourceforge.net/p/openfoam-extend/foam-extend-3.1/merge-requests/7/

     
  • Thank you Bruno for your proposed fixes, but you are basically fixing the wrong problem.

    Unless I am mistaken, bash is always installed and is available on Ubuntu by default. Of course, we don't need bash to become the default shell on Ubuntu, that would be crazy and totally unnecessary.

    Bash is certainly not a strict requirement for foam-extend, but finding all means necessary to stay away from Ubuntu Dash should certainly be one! The amount of time myself and others have wasted on Ubuntu because of dash-related problems is way too large...

    The only good fix for this problem is to use /bin/bash in all the shebang lines. This is a very minor requirement since Bash is available everywhere.

    This way, we control the Bourne Shell syntax being used in all the foam-extend scripts (more robustness), and we make sure to keep away from crazy people like the Ubuntu guys that thinks replacing /bin/sh by an half-baked Bourne Shell like dash is making any sense.

     
  • Dominik Christ
    Dominik Christ
    2014-08-15

    What is the bottom-line then?
    Is a "search for all /bin/sh and replace with /bin/dash" enough to resolve this bug completely?

     
  • Not /bin/dash but rather replace with "/bin/bash".

    I think it would be a good idea to do this.
    I don't know if this will fix the foamExec problem for good, but that would be a good start.

     
  • Bruno Santos
    Bruno Santos
    2014-08-15

    Sorry for not giving feedback on this sooner, but since it seemed that everything was going in the right direction, I didn't worry too much about it.

    Either way, yes, changing "/bin/sh" to "/bin/bash" should also fix the issue with "foamExec".

    I just hope you don't end up regretting this. I've seen OpenFOAM staying faithful to the lower shell-feature specifications of "sh", so I'm a bit wary of making such a strict change from "sh" to "bash".

    From my experience, using dash instead of bash resulted in faster building when using Cygwin to cross-compile OpenFOAM for Windows: http://openfoamwiki.net/index.php/Installation/Windows/Outdated/Using_Cygwin_for_cross-compiling_OpenFOAM - I forgot to write on the wiki how much faster it was (3h30 down to 2h30, if my memory serves me right), but according to the records, it looks like cross-compiling GCC+MinGW in Cygwin was a requirement to use bash... then we could switch back to dash for building OpenFOAM...

    Anyway, the performance benefit on Linux and Mac OS X is likely not as great as in Cygwin: http://unix.stackexchange.com/a/67133 - and with today's CPU technology, it probably won't trim down more than 3-5 minutes for every hour of building OpenFOAM from source code.