build with bash Files

I am still putting this site together. Any enquiries welcome: robert.durkacz@gmail.com
-R.D.

Introduction
============
 
Before 'make' it must have been the case that people built their projects with shell scripts. When make appeared it was immediately and completely accepted but slowly it began to lose favour and there are now many elaborate alternatives. This project shows by demonstration that shell scripts could be a good way to do builds after all. A major advantage is that since everyone knows the shell there is nothing new to learn.
Make does two things. It cleverly infers a process to build a target from available resources, but as a distinct matter it skips the execution of any step in the process where it so happens that the product of the step exists already and is up to date. The first part is not usually what the user needs whereas the second part, which can avoid large amounts of redundant processing, is exactly what is wanted. The second part is simple enough to achieve with the shell.

In this project we simply write wrapper functions for the compiler which test whether the output is up to date before going ahead to execute the relevant command. A build script for a project invokes commands to do every step to build the project as if the source had been freshly checked out. On first use every command would be executed and all products of the build created. If the script were to be run again immediately after a successful first use there would be no effect because all files are up to date. In this way and in these cases we achieve the same result as 'make all', followed by a second invocation of make all, would do. Like make, our script prints out the command lines as they are executed. Unlike make, the user deals with a normal procedural bash script, one which makes good use of shell functions to store the details of commands as well as provided the conditional execution feature and various other conveniences.

As an example, from the command line or from a script, this is how the user compiles the file jobs.c-
  cc jobs
The response is to print and run this command line-
  gcc -c -g -O2 -DSHELL -DHAVE_CONFIG_H -I. -I. -I./include -I./lib -Wno-parentheses -Wno-format-security jobs.c
except if jobs.o is already present and up to date then there is no response at all.
In this way we have the same results that
  make jobs.o
would give.

This project does not offer a software package but merely the idea, particularly simple as it is, supported by a demonstration, using a recent release of bash itself as a use case.

To give it a try unpack the bash-4.4.tar.gz package from https://www.gnu.org/software/bash/ or here and add build.sh, build-project.src and build-general.src to the top level directory. Run ./configure to obtain files necessary for the buiild. Run build.sh instead of make to get the same result that make would give. The first thing to be aware of however is that there are no measures taken to ensure portability between platforms as we are offering a way to do the job of make not a way to replace the configure step. bash itself is likely to be portable enough but there may be a problem with the necessary command line invocations varying somewhat. Only the user can deal with this. The files supplied work on Ubuntu 20.

Subject to the build being made to work at all and successfully completed firs time, the following command sequences demonstrate our build system.

  touch jobs.c
  build.sh

All being well the result will be as follows-

	cd ./.
	cd ./builtins
	cd ../.
	gcc -c -g -O2 -DSHELL -DHAVE_CONFIG_H -DPROGRAM="bash" -DCONF_HOSTTYPE="x86_64" -DCONF_OSTYPE="linux-gnu" -DCONF_MACHTYPE="x86_64-unknown-linux-gnu" -DCONF_VENDOR="unknown" -DLOCALEDIR="/usr/local/share/locale" -DPACKAGE="bash" -I. -I. -I./include -I./lib -Wno-parentheses -Wno-format-security jobs.c
	cd ./builtins
	cd ../lib/glob
	cd ../../lib/sh
	cd ../../lib/readline
	cd ../../lib/termcap
	cd ../../lib/tilde
	cd ../../lib/malloc
	cd ../../.
	#    jobs.o nt bash
	gcc -L./builtins -L./lib/glob -L./lib/sh -L./lib/readline -L./lib/termcap -L./lib/tilde -L./lib/malloc -rdynamic -o bash shell.o eval.o y.tab.o general.o make_cmd.o print_cmd.o dispose_cmd.o execute_cmd.o variables.o copy_cmd.o error.o expr.o flags.o jobs.o subst.o hashcmd.o hashlib.o mailcheck.o trap.o input.o unwind_prot.o pathexp.o sig.o test.o version.o alias.o array.o arrayfunc.o assoc.o braces.o bracecomp.o bashhist.o bashline.o list.o stringlib.o locale.o findcmd.o redir.o pcomplete.o pcomplib.o syntax.o xmalloc.o -lbuiltins -lglob -lsh -lreadline -lhistory -ltermcap -ltilde -lmalloc -ldl
	./lib/sh/libsh.a(tmpfile.o): In function `sh_mktmpname':
	/home/tux/bash/bash-4.4_alt/lib/sh/tmpfile.c:152: warning: the use of `mktemp' is dangerous, better use `mkstemp' or `mkdtemp'
	cd ./support
	cd ../.

The build.sh visits all directories announcing the cd step as it does so. Unlike the usual case with make there is no recursion but environment variables are adjusted on the fly (whereas make variables are fixed). It is no mystery why the jobs.c file was recompiled whereas the bash executable had to be relinked and in this case a comment is made to give the reason, ie it is because jobs.o is newer than bash.

Next,
  touch trap.h
  build.sh

trap.h is included by jobs.c. We expect again jobs.c to be recompiled. Result should be, with many lines omitted since trap.h is included in other c files, -

	cd ./.
	#    trap.h nt shell.o
	#    rm shell.o
	gcc -c -g -O2 -DSHELL -DHAVE_CONFIG_H -DPROGRAM="bash" -DCONF_HOSTTYPE="x86_64" -DCONF_OSTYPE="linux-gnu" -DCONF_MACHTYPE="x86_64-unknown-linux-gnu" -DCONF_VENDOR="unknown" -DLOCALEDIR="/usr/local/share/locale" -DPACKAGE="bash" -I. -I. -I./include -I./lib -Wno-parentheses -Wno-format-security shell.c
	etc ...
	#    ../trap.h nt trap.o
	#    rm trap.o
	gcc -c -g -O2 -DSHELL -DHAVE_CONFIG_H -I. -I.. -I../include -I../lib -Wno-parentheses -Wno-format-security trap.c
	#    common.o nt libbuiltins.a
	ar cr libbuiltins.a builtins.o alias.o bind.o break.o builtin.o caller.o cd.o colon.o command.o common.o declare.o echo.o enable.o eval.o evalfile.o evalstring.o exec.o exit.o fc.o fg_bg.o hash.o help.o history.o jobs.o kill.o let.o mapfile.o pushd.o read.o return.o set.o setattr.o shift.o source.o suspend.o test.o times.o trap.o type.o ulimit.o umask.o wait.o getopts.o shopt.o printf.o getopt.o bashgetopt.o complete.o
	cd ../lib/glob
	etc ...
	cd ../../.
	#    shell.o nt bash
	gcc -L./builtins -L./lib/glob -L./lib/sh -L./lib/readline -L./lib/termcap -L./lib/tilde -L./lib/malloc -rdynamic -o bash shell.o eval.o y.tab.o general.o make_cmd.o print_cmd.o dispose_cmd.o execute_cmd.o variables.o copy_cmd.o error.o expr.o flags.o jobs.o subst.o hashcmd.o hashlib.o mailcheck.o trap.o input.o unwind_prot.o pathexp.o sig.o test.o version.o alias.o array.o arrayfunc.o assoc.o braces.o bracecomp.o bashhist.o bashline.o list.o stringlib.o locale.o findcmd.o redir.o pcomplete.o pcomplib.o syntax.o xmalloc.o -lbuiltins -lglob -lsh -lreadline -lhistory -ltermcap -ltilde -lmalloc -ldl

To deal with indirect dependencies caused by included header files, the compile instruction creates a dependency file as a side-effect. This file is intended for the use of make. For our use we have a shell function try(). In the above case the compiler function cc() called try with argument trap. try() examined the side-effect dependency file trap.d and found that trap.o was out of date with respect to trap.h and so it deleted trap.o. trap.o being absent, cc() rebuilt it. Other object files were rebuilt, requiring an intermediate library to be rebuilt. Comments are issued when a file is found to be out of date with respect to any one of a list of files. Only the first encountered file is mentioned.

The are shell functions for building archives, arch() wrapping ar; linking executables, link(); wrapping gcc; and compiling, being cc() wrapping gcc -c.

Additional tricks built into the shell functions are inspired by make. Setting environmental variable clean to any value results in the shell function deleting the usual product instead of creating it. Setting dryrun results in the command being printed but not executed. (This is much weaker than the make dry-run option; it only detects immediate effects of out of date files and not follow-on results.) The side-effect feature can be disabled by unsetting Tryopt. Dependencies for the link() function are supplied explicitly, ie Deps=... link(arg)

The make.sh file is documentation for how to build specific targets from the command line using the shell functions. You need to take into account environment variables that are set when the command is issued. For instance to compile trap.c manually this is how:
  . build.src
	A1=`echo -DPROGRAM=\"bash\" -DCONF_HOSTTYPE=\"x86_64\" -DCONF_OSTYPE=\"linux-gnu\" -DCONF_MACHTYPE=\"x86_64-unknown-linux-gnu\" -DCONF_VENDOR=\"unknown\" -DLOCALEDIR=\"/usr/local/share/locale\" -DPACKAGE=\"bash\"`
	Dopts_var=$A1
  cc trap
Perhaps a more convenient way could be found.