Aubit 4GL SVN-Code

dotnet: answer the fglproto question in the README

Records what the prototype pass is actually for, since it is the first
thing anyone hits: for a single module it is redundant - the generator
infers return types from the bodies it can see, and the prototypes only
save a defensive As* conversion - but for anything multi-module it is
required, and a function returning several values is the case that
cannot be guessed at all.

Also adds the two things the new arithmetic probe turned up to the list
of mistakes the probes caught, and records DISPLAY of an unassigned
expression as a known gap rather than leaving it implied by what the
probe does not test.

dotnet: 4GL division does not truncate

Found while answering whether the C# backend still needs fglproto: a
cross-module test came out with c=3.00 where the C runtime said 3.50.

4GL division never truncates. 7/2 is 3.5 whatever the operands are
declared as, and it is the assignment afterwards that rounds or
truncates - LET i = 7/2 gives 3 because i is an INTEGER, not because
the division did anything. Checked against the C runtime:

dec 7/2 = 3.5000
flt 7/2 = 3.50
int 7/2 = 3
lit 7/2 = 3.5000
neg -7/2 = -3.5000

The generator emitted a bare C# "/", which truncates at the division
itself when both sides are integers, so a DECIMAL target held 3.00.
Division now goes through Fgl.Divide, which promotes first. Division by
zero returns NULL rather than throwing, as the C runtime does - a 4GL
program tests the result, it does not catch anything.

Two things fell out of fixing it:

assign_wrapper had no case for FLOAT or SMALLFLOAT, so assigning any
decimal-valued expression to one would not compile. It has one now,
the same shape as the CHAR, DECIMAL and SMALLINT cases.

DISPLAY of a MONEY lost its currency symbol. MONEY and DECIMAL are
both decimal? at runtime - that is deliberate, the difference is
presentational - so the type has to be named where it is still known,
and the generator now emits Fgl.DispMoney for it. The symbol comes
from DBMONEY, which can also place it after the number and change the
decimal separator, and it is extra to the declared width: a
MONEY(10,2) shows in 13 columns where a DECIMAL(10,2) shows in 12.

New end-to-end probe arith.4gl covers all of it: division into DECIMAL,
FLOAT, INTEGER and SMALLINT, negative and literal division, MONEY, and
the other three operators plus MOD.

Every result in it is assigned to a declared variable before being
displayed, deliberately. DISPLAYing a raw expression is a separate
question - an unassigned division has no declared scale, and 4GL prints
it at DECIMAL's maximum width - and that gap is recorded in the README
rather than quietly folded into this probe.

Fidelity: 27 semantics, 9 UI messages, 30 USING, the 66-line report and
6 end-to-end programs all agree with C. 203 runtime tests pass, d4 still
builds clean, and the standalone corpus is unchanged at 30 of 33.

etc/containers: check the installed tree, not just the build

Every install bug of the last few days - the missing aubitrc, the
generated headers that were never installed, libSQL_pg8 built and then
left behind - had the same shape: make install reported success and
produced a tree nothing could be built against. None of it is visible
to anyone working in the source tree, which is where we all work.

So the container check now has two phases:

in-tree compile and run a 4GL program from the source tree
installed make install to a prefix, verify what actually landed,
then compile and run again with ONLY the installed tree
on PATH and LD_LIBRARY_PATH

The second phase does not trust the exit status. It checks that 4glc,
aubit-config, libaubit4gl, the public headers, json.h, at least eight
generated a4gl_API_*.h and an aubitrc are all present, and that every
libSQL_* plug-in that was built was also installed. Then it runs
aubit-config, which is the first thing to notice a config it cannot
read, and finally compiles and runs a program with a REPORT and checks
the output is right rather than merely that the compiler exited 0.

The container-side half moved into its own checks.sh, since it had
outgrown being a string inside the run command.

Both negative-tested by reintroducing a bug in a scratch export and
confirming the harness fails, by name and before the compile:

dropping json.h from the header list
MISSING: incl/json.h (exit 1)

dropping SQL_pg8 from the install list
BUILT BUT NOT INSTALLED: libSQL_pg8.so (exit 1)

In both, the in-tree phase still passed - which is the whole point.

Clean tree passes on both images.

configure: make an installed tree usable

Compiling a 4GL program against a --prefix install failed three times
over. Each fault hid the next, so they only came out one at a time.

1. The runtime could not find its own aubitrc.

resource.c has had an #ifdef AUBITETC for this all along, but nothing
ever defined it, so it fell back to a hardcoded /etc/opt/aubit4gl -
correct only when the prefix is the default. configure computes
AUBITETC as $sysconfdir/aubit4gl and installs there, so with
--prefix=/opt/aubit4gl the file went to
/opt/aubit4gl/etc/aubit4gl/aubitrc while every tool looked in
/etc/opt/aubit4gl/aubitrc and reported

*********** ERROR IN CONFIGURATION ***********
No aubitrc file has been found on your system

which surfaced as "ERROR: no TARGET_OS STOP." from bin/aubit, because
TARGET_OS comes from aubit-config, which could not read its config.
AC_DEFINE_UNQUOTED now tells the C code what configure chose.

2. None of the a4gl_API_*.h headers were installed.

The list filters each header on existence, so that one since removed
cannot break the install. That is right for the hand-written ones and
wrong for these: make generates them from the .spec files, long after
configure has run, so the test always failed and quietly dropped all
eight. Since a4gl_4gl_callable.h - which IS installed - includes
a4gl_API_exreport.h, the first compile against the install died with
"a4gl_API_exreport.h: No such file or directory". They are now listed
unconditionally; the filter stays on the headers it makes sense for.

3. json.h was not installed either, though a4gl_4gl_callable.h includes
it. It is as public as the rest of them, whatever its name suggests.
Found by copying only the listed headers into an empty directory and
compiling generated output against it until it stopped complaining -
json.h was the only one left, the rest of the include closure being
behind ifdefs that are not taken.

Verified on AlmaLinux 8: configure, make and make install with
--prefix=/opt/aubit4gl, then from the installed tree alone -
aubit-config answers TARGET_OS and A4GL_SQLTYPE, 18 headers are in
place including all eight generated ones, both PostgreSQL drivers are
present, and a program with a REPORT compiles and runs and prints what
it should. The source-tree path is unchanged and the fidelity probes
still pass.

configure: install libSQL_pg8, and make it the PostgreSQL default

Reported: a compile against an installed tree dies with

Error: can't open DLL /opt/aubit4gl/plugins-1.8.1/libSQL_pg8.so - STOP
Error msg: ... cannot open shared object file: No such file or directory

Reproduced in a container on AlmaLinux 8 with postgresql-devel: the
plug-in builds, and then is not installed. lib/libsql/postgresql builds
both libSQL_pg and libSQL_pg8, but only SQL_pg was in ALL_PLUGINS,
which is what drives the install. So libSQL_pg8.so sat in the source
tree and never reached the install, and a program that asked for it got
the error above from an otherwise complete installation.

The default was wrong as well, and more seriously.

libSQL_pg and libSQL_pg8 are not two versions of the same driver.
libSQL_pg is built from simple.c - "This simple PG plug-in with limited
functionality", as its own Makefile comment says - and it has no
cursor, scroll or transaction support at all: 22 entry points against
pg8.c's 78, and not one mention of CURSOR, SCROLL or COMMIT in 1074
lines. A 4GL program that does a FOREACH or a BEGIN WORK cannot run on
it. libSQL_pg8 is the real driver, at 4616 lines.

configure was defaulting A4GL_SQLTYPE to pg, so a PostgreSQL user got
the limited driver by default and could not select the other one,
because it was not installed.

Now: both are installed, both are listed in A4GL_SQLTYPE_DETECTED, and
the default is pg8. Both still sort below Informix, so a box with both
still picks esql as it did.

Verified on Alma 8: configure reports "SQL backend: pg8", make install
puts libSQL_pg.so and libSQL_pg8.so in the plug-in directory, and the
installed aubitrc carries A4GL_SQLTYPE=pg8 with
A4GL_SQLTYPE_DETECTED="nosql pg pg8".

common/dataio: drop the dead menu_x rpcgen rules

You were right, and it goes further than the rpcgen path: nothing in
the default build consumes common/dataio/menu_x.x at all.

- menu_x_xdr.o was a prerequisite of nothing. The four XDRPACKER
plug-ins are struct_form, report, module and module_definition.
- lib/libmenu is not wired into lib/Makefile, so no libMENU_* is
built - configure already says so in a comment, and aubitrc reports
A4GL_MENUTYPE_DETECTED="NOMENU GENERIC" with no XDR in the list.
- The one place that could still have wanted the rpcgen header,
compilers/menus/a4gl_mcompile_int.h, takes the xgen branch:
_USE_RPCGEN_HEADERS_ is not defined anywhere in the tree.
- compilers/menus/Makefile had already commented its own rpcgen rules
out, with "Should be obsolete - using xgen instead of rpcgen".
- Even the xgen side is unused here: libDATA_any is the only thing
that wants menu_x.xio.o, and it is commented out of ALL. An Alma
build produces libDATA_struct_form, _report, _module and
_module_definition, and no menu_x object of any kind.

So the RPCGEN_MENU definitions and the menu_x_xdr.c and menu_x_xdr.o
rules are gone, including the fallback line I added to them in r13066 -
which never fired, since nothing asked for the file.

One distinction worth keeping straight, because the two files have the
same name: compilers/menus/menu_x.x IS live. xgen turns it into
menu_x.xs.h, which mcompile includes, and bin/mcompile is built. Only
the common/dataio copy is dead. Nothing has been removed from
compilers/menus.

tools/cygwin/menu_x.h and menu_x_xdr.c are now referenced by nothing.
They are still the odd ones out there - hand-adapted rather than
rpcgen output - and deleting versioned files is your call, so they are
left alone; refresh.norpcgen does not touch them either way.

Verified: common/dataio and compilers/menus both still build, a clean
Alma 8 container build with no rpcgen still configures, makes and runs
a compiled 4GL program, and the fidelity probes still pass.

Makefile: make refresh.norpcgen work again, and refresh what had drifted

The target was there, as you said. It had not copied anything in a very
long time, for two reasons:

if test "$x" = "1"

"$x" is expanded by make, not by the shell, so the shell saw
test "" = "1" and the branch never ran. It wanted "$$x" - which
refresh.noyacc, twenty lines below, gets right.

The sense was inverted anyway. diff -q exits 0 when two files are the
SAME, so

if diff -q common/dataio/form_x.h tools/cygwin/form_x.h ; then x=1; fi

set "needs updating" precisely when they already matched, and left it
clear when they had diverged. Both bugs had to be fixed for either to
show.

Rewritten as a loop over the file list, so a new .x file is one word to
add rather than three lines, and it now says which files it copied.

Then run, which refreshed three files that had drifted from the .x
sources:

form_x.h FA_S_ALLOWEDCHARS, an enum used by lib/libui/ui_curses
fgl.h the s_expr_json type and a struct's config field
fgl_xdr.c the matching xdr_s_expr_json serialiser

All additions; nothing was removed. report.h, report_xdr.c, npcode.h
and npcode_xdr.c were already current.

menu_x stays out of the list, as it always was. Unlike the other four,
tools/cygwin/menu_x.h is not rpcgen output - it is hand-adapted to need
no RPC headers, declaring bool_t itself rather than including
<rpc/rpc.h>. Copying rpcgen's version over it would throw that away.
That is now written down in the target rather than being implicit in
the omission.

Verified: a second run reports everything up to date; the fidelity
probes still pass here; and the no-rpcgen Alma 8 container build still
configures, makes and runs a compiled 4GL program against the refreshed
files - which is the case these copies exist for.

etc/containers: build on Alma 8 without needing an Alma 8

We have been fixing RHEL-family build failures blind, one report at a
time, because they do not happen on a current Debian. Three in a row now
- --with-pg-all being silently ignored, configure dying with no message
where there is no controlling terminal, and the missing -ldl - were all
invisible here and obvious there.

./etc/containers/build-in-container.sh # alma8, no rpcgen
./etc/containers/build-in-container.sh alma8-rpcgen # with rpcgen
./etc/containers/build-in-container.sh alma8 --with-mysql

It exports the tree rather than mounting the working copy, so a
generated file or an object file from a host build cannot mask a
missing dependency - which is the entire point of building elsewhere.
Then it configures, makes, and compiles and runs a 4GL program, so a
green result means the toolchain works rather than merely that it
linked. Uses podman when it is there and docker otherwise; podman needs
no daemon and no group membership, so it works without setup.

Alma 8 is worth the trouble because of how it differs:

glibc 2.28 libdl.so.2 is still a separate object
gcc 8.5 stricter in places than what we develop against
autoconf 2.69 older than configure.ac wants, so the image installs
no autoconf at all and builds with the configure that
ships in the tree - what a release user actually does
rpcgen in PowerTools, off by default; the plain image leaves
it out so the pre-generated files get exercised, and
the -rpcgen image puts it back to cover the other path

Both images verified: configure, make and the smoke test pass on each.

build: rpcgen is not a requirement again

I made rpcgen a hard configure error. It never was one, and it should
not have become one - this reverses that.

tools/cygwin holds a checked-in copy of every file rpcgen produces:
form_x.h, fgl.h, menu_x.h, npcode.h, report.h and their _xdr.c
partners. They are not raw rpcgen output - they are hand-adapted to
compile with no RPC headers at all, defining bool_t themselves rather
than including <rpc/rpc.h>. That is exactly what a machine without
rpcgen needs, and it is how this built for years before rpcgen was on
anyone's dependency list.

common/dataio now generates when rpcgen is there and copies those files
when it is not. configure says which it is doing instead of stopping.

The error was an over-correction on my part. What actually broke on
Debian 13 was the ancient bundled RPC runtime headers under
tools/no_rpc/cygwin/rpc, where XDR_INLINE() returns long * while
rpcgen output declares int32_t * - a hard error with gcc 14. That is a
different directory and a different problem from the pre-generated
output in tools/cygwin, and the second was made to pay for the first.

The requirement also landed hardest on the people least able to satisfy
it: on RHEL 8, Alma and Rocky rpcgen lives in PowerTools, which is not
enabled by default, so the advice the error gave - "dnf install rpcgen"
- does not work as written. The notice now says
--enablerepo=powertools.

Verified in a container on the reporter's platform, AlmaLinux 8.10 with
glibc 2.28:

- with no rpcgen installed at all: configure and make both succeed,
and the resulting compiler builds and runs a 4GL program
- with rpcgen present, here on Debian: form_x.h is still generated
from form_x.x, and the end-to-end probes still pass

The pre-generated copies have drifted from the .x files - form_x.h is
missing FA_S_ALLOWEDCHARS, menu_x more than that. It does not break the
build, because the enum that matters reaches the code through
incl/dataio/form_x.xs.h, which xgen generates and which is versioned.
Worth refreshing, but it needs doing by hand to keep the no-RPC-headers
adaptation, so it is not being done blind here.

build: link -ldl where dlclose() is called directly

Reported from a RHEL-family build:

/bin/ld: API_lex.o: undefined reference to symbol 'dlclose@@GLIBC_2.2.5'
//usr/lib64/libdl.so.2: error adding symbols: DSO missing from command line
make[2]: *** [Makefile:192: ../../bin/4glc] Error 1

4glc calls dlclose() itself - the .spec-generated API_lex.c and
API_parse.c declare their own prototype for it and call it from
A4GLLEX_clrlibptr() - so it has to link against libdl in its own right,
not lean on libaubit4gl having pulled it in. Nothing put -ldl on the
line: configure has worked out USE_LIBDL correctly all along, but no
Makefile ever used it, and EXTRALIBS - which the link lines do use - is
substituted empty.

It only shows on some distributions because glibc 2.34 merged libdl
into libc. On Debian and Ubuntu dlclose resolves out of libc and the
missing library is invisible; on RHEL, Alma and Rocky 8, where
libdl.so.2 is still its own object, the link fails. compilers/pcode has
the same direct calls and so had the same latent problem.

USE_LIBDL now goes into EXTRASYSTEMLIBS, which every link rule already
uses, so this covers the executables rather than just the one that was
reported. It is empty on macOS, where dlopen is in libc anyway, and
-ldl still links harmlessly on a merged-libc glibc - the stub library
is kept for compatibility - so nothing needs to be conditional beyond
what configure already decides.

Verified: -ldl now appears on the 4glc link line, 4glc rebuilds and
runs, and the end-to-end probes still pass.