[Compilercache-general] oh boy...

[Erik T. <er...@er...>]

hi.

i got a problem that drives me nuts.

it's all about temporary files creation in bash scripts.
compilercache needs to do two things.

1) create a temporary file that is maximally fast

2) create a temporary file in the cache directory that can be renamed
to the real entry by a single atomic "rename" system call. (i.e. mv
needn't copy) this is important because otherwise other users of your
cache will get half-ready (i.e. wrong) .o files.

debian has the nicest fix for this task. it's the "tempfile"
program. unfortunately it is not installed in other distributions like
redhat, or even other unixes.

in compilercache-1.0.9 i was using

FASTTMPFILE=/tmp/compilercache_$$

which is a VERY severe security hole since other users could install
link files there and manipulate the temporary files of other users...

there exists a "mktemp" program which is not portable to HP-UX at least.

ok. i NEED the functionality (especially the --directory parameter) of
debian "tempfile". either it is there or not. if not, i must include a
tempfile program in the compilercache distribution and tell the user
he must install it. but where to ??? /bin is not an option for
non-root users. so the user must put it somewhere in his path and set
the PATH variable accordingly in the .compilercacherc config file. but
where will he put it ? in the compilercache bin dir?

and who writes the tempfile program? maybe a perl script? i wrote one,
(without the --directory functionality) but it is very slow:

use File::Temp qw/tempfile/;
($fh,$filename)=tempfile();
print $filename."\n";

and by the way.... the current 1.0.10 version fixes this problem in a
somehow funny way. you just set your own private temporary directory
and all problems are gone away.

so.. i decided that 1.0.10 is a good version. even though i told some
of you it's not good yet since i'll change the temp stuff again, but i
guess i won't. security, anti-complexity and speed go first :)


a note to the debian package maintainer of compilercache:

you did the following patch in compilercache-1.0.9-2.deb:

283,286c277,278
<   if ! TMPFILE=$(tempfile); then
<     echo "compilercache: error: cannot create temporary file" 1>&2
<     exit 1
<   fi
---
>   TMPFILE="$CACHEDIR/tmp_$(hostname)_$$"
>   rm -f "$TMPFILE"

this opens a new bug. if you run two compilercaches at the same time
on the same file and same user, one will write the .o file, the other
one sees, ah it's already there and copies it. but as it is only half
written, the second compiler will retrieve a half .o file only.

TMPFILE will be mv'ed to the real cache entry. that must be an atomic
operation. if /tmp is mounted on tmpfs for example and not your cache
harddisk, mv won't be atomic and the above effect will occur. this
problem does not exist in original 1.0.9 and in 1.0.10 since both have
the temporary .o file in the cache directory.

so... have fun with 1.0.10 :-)

cu
erik

-- 
Name:  Erik Thiele                                        \\\\
Email: er...@er...                                 o `QQ'_
WWW:   http://www.erikyyy.de/                              /   __8
                                                           '  `