tack-devel — Discussion for developers of the ACK.

[Tack-devel] pm runtime script

[tim k. <gt...@di...>]

Hi David,
Just for kicks I modified pm to 'machine' instead of 'arch'.  It's surely
of no surprise to you that I ruined the zcat CRC check upon saving the
file.  pm doesn't complain about arch not found, though (and as a side
note, OpenBSD's arch(1) does not yield the same results as its machine(1)
call and both are present).

It would possibly be useful to spin off the binary part from the script
part, if possible (and making it self-building would be great, too) :-)

thanks,
tim

Gregory T. (tim) Kelly
Owner
Dialectronics.com

P.O. Box 606
Newberry, SC 29108

"Anything war can do, peace can do better."  -- Bishop Desmond Tutu

Re: [Tack-devel] Trouble Installing ACK

[David G. <dg...@co...>]

Arun Ramachandran wrote:
[...]
> I am involved in a project to build a backend for a forth based
> microprocessor. Since I am new to compiler design, my project guide has=

> suggested use of ACK as it has a stack based intermediate representatio=
n
> EM.

Very cool; what sort of processor is it? Commercial or self-developed? At=
 one
point I wrote a code generator (for a proprietry operating system, ask me=
 for
a boring lecture) for the PSC1000 (ne=E9 ShBoom), which was... interestin=
g. It
had two stacks, a weird instruction encoding scheme was varied depending =
where
the instruction was in memory, one and a half processor cores, about 40% =
of a
floating point unit, and sank without trace. I'm not sure they even paid =
us.

He's right, in that EM would make a good choice. EM does assume that the
equivalent of 'n pick' is very cheap, and will do it a lot, but you could=

probably map most EM instructions onto one or two Forth primitives...

[...]
> I am trying to install ACK in my linux system. Here is what I have
> done...
[...]
> cd $SRC_HOME
> find . -type d -perm -555 -print > $CONFIG/dir_list
[...]
> Could you please confirm that "perm -555" is indeed the correct option?=


I will be honest and admit that I have absolutely no idea what that means=
=2E

When I first got hold of the ACK CVS repository, I did spend some time an=
d
managed to beat the build system into behaving reasonably well on Linux, =
and I
don't know why it would fail on your system --- although it might be wort=
h
playing with your default umask settings.

The old build system is incomprehensible, labyrinthine, and would give
M.C.Escher a fit. I'm a little scared to look into it too deeply in case =
I
find one of the Elder Gods looking back at me... since then, I've been wo=
rking
on a new and improved build system that ought to be a whole lot easier to=
 cope
with. It's not complete, but builds all the code generators and optimiser=
s and
the ANSI C, K&R C and Pascal front ends, so it ought to suffice for what =
you
want; I'm still working on it. If you're interested in giving this a try,=
 look
back in the mailing list archives for a message titled 'CVS update' for
instructions. I'd appreciate any feedback.

(Apart from anything else, the old build system does not support incremen=
tal
builds --- or at least, I've never found any way of making it do one --- =
and
so would be an exercise in frustration for actually doing development on.=
)

--=20
+- David Given --McQ-+ "Gaping from its single obling socket was
|  dg...@co...    | scintillating, many fauceted scarlet emerald..."
| (dg...@ta...) | --- Jim Theis, _The Eye of Argon_ (spelling
+- www.cowlark.com --+ original)

Re: [Tack-devel] CVS & the new build system

[tim k. <gt...@di...>]

At 9:52 AM -0400 7/21/06, David Given wrote:
>Incidentally, NetBSD appears not to have an 'arch' command
>--- what's the portable equivalent?

Try machine(1).  Man pages says equivalent to uname -m

> man machine
D O........n......o..P....eCVSP....O......o..Root1/4pESRepository
V....EntriesU........Entries.LogV....Entries.Backup':pserv-
MACHINE(1)                BSD General Commands Manual               MACHINE(1)

er:an...@an...:/cvsroot

NAME
     machine -- print machine type

SYNOPSIS
     machine

DESCRIPTION
     The machine command displays the machine type.  It is equivalent to
     `uname -m'.

SEE ALSO
     make(1), uname(1)

HISTORY
     A machine command appeared in 4.3BSD-Reno.

BSD                              May 24, 2003                              BSD
> machine
macppc

(yeah, I don't know what the junk is in my man pages.  It shows up after I
build and install userland, possibly because of differences in kernel
version, it's a long story)

tim

Gregory T. (tim) Kelly
Owner
Dialectronics.com

P.O. Box 606
Newberry, SC 29108

"Anything war can do, peace can do better."  -- Bishop Desmond Tutu

Re: [Tack-devel] CVS & the new build system

[David G. <dg...@co...>]

tim kelly wrote:
[...]
> Is LLgen cvs'd?  Can I just update a "LLgen" module?

It is; it's in Ack/util/LLgen. *However*, there's a slight complication -=
-- in
order to build LLgen from CVS, you need to have LLgen installed... yeah, =
I
know this sucks. The external download contains pregenerated copies of th=
e
relevant files.

However, the only change is to the 'pmfile' file, so you could always pul=
l
that out of cvsweb and patch a 1.0.2 version manually. LLgen is very stat=
ic
code; once it's installed, you'll probably never have to touch it again.

(LLgen is the only package in the ACK which does this, which was why I sp=
lit
it off --- I'm beginning to suspect that was, in fact, the wrong solution=
 and
instead I should have checked in pregenerated copies of the files and lef=
t
LLgen as an ACK-only tool. Oh, well.)

--=20
+- David Given --McQ-+ "Gaping from its single obling socket was
|  dg...@co...    | scintillating, many fauceted scarlet emerald..."
| (dg...@ta...) | --- Jim Theis, _The Eye of Argon_ (spelling
+- www.cowlark.com --+ original)

Re: [Tack-devel] CVS & the new build system

[tim k. <gt...@di...>]

At 5:23 PM -0400 7/23/06, David Given wrote:
>OSX does this too --- I've just uploaded LLgen 1.0.3 with a workaround for
>this. I don't know why it's failing; those header files are all in the same
>directory as LLgen.c, so it should be seeing them. Odd. But hopefully the new
>LLgen should help. (I now have the ACK running on OSX.)

Is LLgen cvs'd?  Can I just update a "LLgen" module?

I'm getting ready to run through the process again and post the error
messages when I use the -DPREFIX= flag, so I thought I'd sync up first (and
test again on OpenBSD).

thanks,
tim

Gregory T. (tim) Kelly
Owner
Dialectronics.com

P.O. Box 606
Newberry, SC 29108

"Anything war can do, peace can do better."  -- Bishop Desmond Tutu

Re: [Tack-devel] Trouble Installing ACK

[Gregory T. (t. K. <gt...@di...>]

Hi Arun,

>I am involved in a project to build a backend for a forth based
>microprocessor. Since I am new to compiler design, my project guide has
>suggested use of ACK as it has a stack based intermediate representation
>EM.

Awesome!  However, if I may suggest, it would be of value to investigate
how difficult it would be to make the microprocessor EM-based instead of
Forth.  Then you skip a step, that of writing  the EM->Forth
assembler/converter (unless one already exists).

BTW, I am a big fan of Forth, so I'm not advocating a EM is better than
Forth position.  It's just that EM can work in hardware, instead of needing
to go through an assembler for a target architecture.

>I am trying to install ACK in my linux system. Here is what I have
>done...

I would recommend using CVS to check out the Ack module.  David Given (the
project head) has done a lot of work making the ACK package more portable.
I can confirm it runs on NetBSD, is close to running on OpenBSD, and David
posted a day or so ago that it runs on OS X now, too.  The new build
process works well and effectively supercedes the 5.6 download
(paraphrasing David's comments and suggestions regarding how much further
time he wishes to spend on the 5.6 download).

http://sourceforge.net/mailarchive/forum.php?thread_id=26484335&forum_id=45107

As part of the build process you will be installing David's Prime Mover
(pm), his replacement for make.

http://sourceforge.net/projects/primemover

Overall the process will be easier if you go the CVS route.

hope this helps,
tim

Gregory T. (tim) Kelly
Owner
Dialectronics.com

P.O. Box 606
Newberry, SC 29108

"Anything war can do, peace can do better."  -- Bishop Desmond Tutu

[Tack-devel] Trouble Installing ACK

[Arun R. <aru...@sl...>]

Hi!

I am involved in a project to build a backend for a forth based
microprocessor. Since I am new to compiler design, my project guide has
suggested use of ACK as it has a stack based intermediate representation
EM.

I am trying to install ACK in my linux system. Here is what I have
done...

1) downloaded and extracted ack-5.6.tar.bz2 to my home directory.
2) created a directory ack in tmp (/tmp/ack) to be used as configuration
directory.
3) from the directory /tmp/ack I ran the first script (as suggested in
the install manual) and gave the following options

a) ACK source root /home/username/ack-5.6/
b) root of configuration tree /tmp/ack/
c) ACK binary root /home/username/Project/ACK
d) system type ANY
e) running on system : yes
f) Default machine : em22 (EM with 2byte wide integers and pointers)
g) Unix target system SYS_V
h) limit installation : no
i) Library : libsysV_2

And then I run the INSTALL script that is generated. However, I get the
following error

cp: cannot create regular file
`/tmp/ack//lang/cem/cemcom.ansi/Parameters': No such file or directory

I did some investigation to find that the only directory that is created
by the INSTALL script is the (/tmp/ack/)bin with the ack_sys, cp_dir,
create_dir, mk_makefile and TakeAction executable script.

Interestingly the size of the file dir_list in /tmp/ack/ is 0 bytes.
A little more digging into mk_config script in the first directory of
the source root, it has the line :

cd $SRC_HOME
find . -type d -perm -555 -print > $CONFIG/dir_list

I tried the "find" command line manually and saw that due to the "perm
-555" option, no directory list is generated!! Evidently, because of the
lack of any entries in the dir_list file, no tree structure is created
in the configuration root and hence all copy and move or create
operation fail.

Could you please confirm that "perm -555" is indeed the correct option?
If so, what should be done to get a correct installation?

Thanks in advance
Arun Ramachandran

Student
Institute for System Level Integration
Livingston
UK

Re: [Tack-devel] bad ack-5.6.tar.bz2

[David G. <dg...@co...>]

Jonas Sundstr=F6m wrote:
[...]
> /bin/tar: ack-5.6/mach/sparc_solaris/libem/LIST:=20
> Could not link to `ack-5.6/mach/sparc/libem/LIST':=20
> No such file or directory
>=20
> /bin/tar: ack-5.6/mach/sparc_solaris/libem/libem_s.a:=20
> Could not link to `ack-5.6/mach/sparc/libem/libem_s.a':=20
> No such file or directory
>=20
> If these items are hardlinks, it fails due to BeOS not
> supporting them. We've only got symlinks.

Yup; it's trying to create hardlinks. You should be able to manually fix
things by creating the appropriate symlinks.

(I assume that the reason why it's using hardlinks is because the
authors mainly did development on Minix, which doesn't support
symlinks... yes, I agree with you that hardlinks are evil except in
specific circumstances.)

Alternatively, you could always try building the head of CVS --- it's
using a radically new build system that, er, may compile, and has never
been tested on BeOS...

--=20
+- David Given --McQ-+ "You cannot truly appreciate _Atlas Shrugged_
|  dg...@co...    | until you have read it in the original Klingon."
| (dg...@ta...) | --- Sea Wasp on r.a.sf.w
+- www.cowlark.com --+

Re: [Tack-devel] bad ack-5.6.tar.bz2

[David G. <dg...@co...>]

Jonas Sundstr=F6m wrote:
> I think ack-5.6.tar.bz2 is corrupt.
> I tried two sourceforge mirrors with the same result.
[...]
> ack-5.6/include/_tail_cc/time.h
> ack-5.6/include/_tail_cc/setjmp.h
> ack-5.6/include/_tail_cc/assert.h
> ack-5.6/Action
> /bin/tar: Error exit delayed from previous errors=20

When I try it all seems to work fine.

That 'error exit delayed' message is indicating that tar came across a
non-fatal error further up the file, but could continue --- you have, in =
fact,
successfully reached the end of the archive. If you look for an earlier e=
rror
message you'll probably find a harmless timestamp or ownership warning.

--=20
+- David Given --McQ-+ "Gaping from its single obling socket was
|  dg...@co...    | scintillating, many fauceted scarlet emerald..."
| (dg...@ta...) | --- Jim Theis, _The Eye of Argon_ (spelling
+- www.cowlark.com --+ original)

Re: [Tack-devel] CVS & the new build system

[David G. <dg...@co...>]

Gregory T. (tim) Kelly wrote:
[...]
> gcc -g -Os    -DLIBDIR=3D\"/usr/local/share/LLgen\" -DNON_CORRECTING -c=
 -o
> ".pm-cache/25-LLgen.o" "src/LLgen.c"
> LLgen.g:21: types.h: No such file or directory
> LLgen.g:22: io.h: No such file or directory
> LLgen.g:23: extern.h: No such file or directory
> LLgen.g:25: cclass.h: No such file or directory

OSX does this too --- I've just uploaded LLgen 1.0.3 with a workaround fo=
r
this. I don't know why it's failing; those header files are all in the sa=
me
directory as LLgen.c, so it should be seeing them. Odd. But hopefully the=
 new
LLgen should help. (I now have the ACK running on OSX.)

[...]
> Also, I can't get ACK to build on NetBSD if the PREFIX directory is not=

> /usr/local/.  There are files in paths that are not picked up.

Is this the PREFIX in config.pm? Could you post the errors it's producing=
?
Your /tmp directory isn't set non-executable, is it? (The build process w=
ill
try to build and then run helper tools.)

[...]
> The pmfiles seem quite a bit easier to read than Makefiles.  Do I need =
to
> start learning some of this "Extended Context Free syntax" I read about=
 in
> the LLgen parser generator document?  :-)

Actually, pmfiles are just Lua scripts --- see www.lua.org. LLgen is a to=
ol
used by the ACK to process grammars; it's very much like yacc or bison. I=
t's
not used by pm at all.

--=20
+- David Given --McQ-+ "Gaping from its single obling socket was
|  dg...@co...    | scintillating, many fauceted scarlet emerald..."
| (dg...@ta...) | --- Jim Theis, _The Eye of Argon_ (spelling
+- www.cowlark.com --+ original)

Re: [Tack-devel] CVS & the new build system

[Gregory T. (t. K. <gt...@di...>]

At 9:52 AM -0400 7/21/06, David Given wrote:
>Prerequisites: you will first need to download and install LLgen 1.0.2
>from the ACK website.

LLgen doesn't build on OpenBSD/macppc 3.5:

gcc -g -Os    -DLIBDIR=\"/usr/local/share/LLgen\" -DNON_CORRECTING -c -o
".pm-cache/25-LLgen.o" "src/LLgen.c"
LLgen.g:21: types.h: No such file or directory
LLgen.g:22: io.h: No such file or directory
LLgen.g:23: extern.h: No such file or directory
LLgen.g:25: cclass.h: No such file or directory
rm -f ".pm-cache/25-LLgen.o"
pm: object 'cfile', defined at pmfile:38, failed to build with return code 256

Also, I can't get ACK to build on NetBSD if the PREFIX directory is not
/usr/local/.  There are files in paths that are not picked up.  This is
with buildiung LLgen to /usr/local/ and to the alternatively specified path.

The pmfiles seem quite a bit easier to read than Makefiles.  Do I need to
start learning some of this "Extended Context Free syntax" I read about in
the LLgen parser generator document?  :-)

tim

Gregory T. (tim) Kelly
Owner
Dialectronics.com

P.O. Box 606
Newberry, SC 29108

"Anything war can do, peace can do better."  -- Bishop Desmond Tutu

Re: [Tack-devel] [tack-devel] a.out vs. ELF

[Gregory T. (t. K. <gt...@di...>]

David Given wrote:

>...I suspect a lot of the build confusion comes from mistaking headers
> intended to be used by programs compiled *by* the ACK with headers used
>by the
> ACK itself.

Well, I did not understand why ack needed any knowledge it was going to be
in ELF format in order to even compile.  I can understand this at the
output from ack layer, but that wasn't the case.  I could even understand
it if ack was bootstrapping itself, but it wasn't completely clear to me it
was.  I didn't understand why the particular header was being picked up; I
just assumed it had some POSIX functionality in it that ACK needed and for
some reason wasn't provided with the source code.

>  As a result of this, the ACK knows absolutely nothing about shared
>libraries.  However,
> there's no reason why we have to use the ACK's platform. It's
> perfectly possible to compile a program referring to headers in /usr/include,
> link it with led into a partially linked object file (because led can't link
> to shared libraries, of course), exporting it to ELF (using a program yet to
> be written), and then linking the result to shared libraries using the
> platform linker.  If you do this then you'll lose the portability across
>platforms that the ACK
> was originally aiming for, but frankly, that's not particularly important
> these days.

I would say that portability across platforms is critical to maintain.  ACK
is exceptionally well positioned for bootstrapping alternative operating
systems.  I would say that ACK's main draw is going to be for new OSes and
not currently existing ones which are probably quite unlikely to be able to
compile on any toolchain other than gcc.  As such, I would recommend
delaying the file format generation decision as late as possible in the
assembling process.

I am aware that ACK currently does not generate code in high enough quality
to be desirable for kernels.  However, I'd rather fix that problem than
build a kernel tree that is tied very tightly to gcc functionality and
directives that then can not be ported to being built with ACK, XL C, et al.

> I am planning to write two ack.out converters at some point, one for ELF and
> one for a simple binary image; however, I don't know a lot about ELF and so
> this would require a lot of reading up on... it may be easier instead to
> persuade someone who knows about such things to add ack.out support to
>the GNU
> binutils toolchain instead, so that ld can read them in directly. ack.out
> doesn't look very complicated.

ack.out appears to very closely resemble xcoff.  It'd be easy to modify ld
to recognize this format and as Perry indicated in an earlier post, NetBSD
can be compiled for xcoff compatibility.  In conjunction with my statement
about ACK's main draw, simple three section file formats are about as easy
as they get for linking and loading (short of straight binary with entry at
byte 0).  While most kernels are ELF, I know that the NetBSD and OpenBSD
bootloaders strip off extra sections when booting, and are only interested
in the .text, .rodata, and .bss sections when loading the kernel into
memory (and maybe the symbols section if it hasn't been stripped).

/usr/src/sys/lib/libsa/loadfile.c
/usr/src/sys/lib/libsa/loadfile_elf32.c

So while supporting ELF may be desirable and reasonable, I think there are
a lot of arguments for having parallel file formats.  I am concerned that
converters from ELF or to ELF will lose information if other file formats
come into play (like the shared library-friendly PEF format, for example).

tim

Gregory T. (tim) Kelly
Owner
Dialectronics.com

P.O. Box 606
Newberry, SC 29108

"Anything war can do, peace can do better."  -- Bishop Desmond Tutu

[Tack-devel] stack based vs. register based em/cg/ncg

[Gregory T. (t. K. <gt...@di...>]

Hi All,

David Given wrote:

>[...]
> Gregory T. (tim) Kelly wrote:
>> It's occured to me that the em virtual machine needs to be rewritten, too,
>> to use an infinite number of registers (as per W. L. Van der Poel's dictum
>> referenced in the em documentation).  This would make RISC assemblers much
>> easier to write and schedule with full consideration of pipelining.
>
>You may have trouble. em doesn't use registers at all --- it's entirely stack
>based. The optimisers can provide register hinting, but it's the backend
>itself that does all the register allocation.

Yes, I understand this.  If instead of em using an infinitely deep stack it
used an infinite number of registers for local variables and parameters, I
theorize that em's operational outcomes would be the same with only minor
changes to the logic.

Consider the em output of the count function.  One of the steps necessary
for the stack based operation was to restore the value of max to the stack
each pass, since comparing the current value of i to max consumed both i
and max.  The ARM code reproduced this behavior, when it should have stuck
max into a register and then kept comparing that.  I recognize that some of
ARM's output comes from the the assembler being coded to a stack instead of
registers, but it does hint towards deeper problems getting an assembler to
order code efficiently when building line by line.

>I suspect that it may be easiest to implement the pipelining system as a
>separate stage after code generation; top, the target optimiser, already
>features a peephole optimisation system. It may be possible to extend this to
>reorder instructions to take account of pipelining.

The scheduling in the assembler will have to handle the pipelining, in
order to be most accurate.  Originally I thought getting pipelining hints
from em would be very helpful, but now I realize that it should be treated
like a one cycle-one instruction RISC engine and let a global optimizer in
the target assembler deal with pipelining.  The biggest help would be for
em to use registers instead of a stack so that the target assembler could
"look ahead" to see how soon the register or branch conditional is needed.
It seems to me that looking ahead on a stack based single line code rule
structure is difficult.

>I also have a nasty feeling that the ACK really, really likes to pass
>parameters on the stack, rather than in registers, and I'm not sure if this
>can be changed. Certainly, this C code:
>
>arg0();
>arg1(1);
>arg2(1,2);
>
>...turns into this em code:
>
>cal $arg0
>loc 1
>cal $arg1
>loc 2
>loc 1
>cal $arg2
>
>I don't think the backend knows how many parameters a call takes, and
>therefore it doesn't know how many to load into registers... that said, the
>ncg manual *does* have an example rule that ensures that top-of-stack always
>lives in a register; it might be possible to extend this concept. But it may
>not be possible to make ACK code conform to a register-centric ABI without
>passing through interface layers.

I'll definately take a look at the ncg document.

My thought regarding altering em to be register-centric would be along the
lines of r0 being reserved, volatile registers be r1 -> r` (where ` stands
for infinity), and -r1 -> -r` (negative numbered registers) are
non-volatile.  If the backend isn't forced to know the number of parameters
(which are passed in volatile registers), this would leave open a path for
function overloading.  The particular function gets tied after the message
tokens are delivered.  Functions that call additional functions save its
local variables to non-volatile registers, which are always restored before
returning from a function call (by the called function).  Always assume the
volatile registers are destroyed after a function call and the non-volatile
registers are intact.  Non-volatile registers are saved to a stack and
restored from it, just like in the real world.  The assembler would then be
responsible for tying the virtual registers to real ones.

It might be necessary to define "real" registers for stack pointer, link
register, count register, and perhaps a few others, but that's not
difficult.

tim


Gregory T. (tim) Kelly
Owner
Dialectronics.com

P.O. Box 606
Newberry, SC 29108

"Anything war can do, peace can do better."  -- Bishop Desmond Tutu

Re: [Tack-devel] ACK compiles on NetBSD-macppc, sort of...

[Gregory T. (t. K. <gt...@di...>]

Ugh, that should have been a new thread.  I am reposting.

tim

Gregory T. (tim) Kelly
Owner
Dialectronics.com

P.O. Box 606
Newberry, SC 29108

"Anything war can do, peace can do better."  -- Bishop Desmond Tutu

Re: [Tack-devel] ACK compiles on NetBSD-macppc, sort of...

[Gregory T. (t. K. <gt...@di...>]

Hi All,

(After a brief discussion with David I decided to remain with the list.)  I
can confirm that ACK builds on NetBSD/macppc.  I'm going to do a build on
OpenBSD/macppc as soon as possible.

If I'm still displaying long lines, I apologize.  When I do word/line
wrapping it comes out badly on other email clients that automatically
handle wrapping.

>[...]
>> It's occured to me that the em virtual machine needs to be rewritten, too,
>> to use an infinite number of registers (as per W. L. Van der Poel's dictum
>> referenced in the em documentation).  This would make RISC assemblers much
>> easier to write and schedule with full consideration of pipelining.
>
>You may have trouble. em doesn't use registers at all --- it's entirely stack
>based. The optimisers can provide register hinting, but it's the backend
>itself that does all the register allocation.

Yes, I understand this.  If instead of em using an infinitely deep stack it
used an infinite number of registers for local variables and parameters, I
theorize that em's operational outcomes would be the same with only minor
changes to the logic.

Consider the em output of the count function.  One of the steps necessary
for the stack based operation was to restore the value of max to the stack
each pass, since comparing the current value of i to max consumed both i
and max.  The ARM code reproduced this behavior, when it should have stuck
max into a register and then kept comparing that.  I recognize that some of
ARM's output comes from the the assembler being coded to a stack instead of
registers, but it does hint towards deeper problems getting an assembler to
order code efficiently when building line by line.

>I suspect that it may be easiest to implement the pipelining system as a
>separate stage after code generation; top, the target optimiser, already
>features a peephole optimisation system. It may be possible to extend this to
>reorder instructions to take account of pipelining.

The scheduling in the assembler will have to handle the pipelining, in
order to be most accurate.  Originally I thought getting pipelining hints
from em would be very helpful, but now I realize that it should be treated
like a one cycle-one instruction RISC engine and let a global optimizer in
the target assembler deal with pipelining.  The biggest help would be for
em to use registers instead of a stack so that the target assembler could
"look ahead" to see how soon the register or branch conditional is needed.
It seems to me that looking ahead on a stack based single line code rule
structure is difficult.

>I also have a nasty feeling that the ACK really, really likes to pass
>parameters on the stack, rather than in registers, and I'm not sure if this
>can be changed. Certainly, this C code:
>
>arg0();
>arg1(1);
>arg2(1,2);
>
>...turns into this em code:
>
>cal $arg0
>loc 1
>cal $arg1
>loc 2
>loc 1
>cal $arg2
>
>I don't think the backend knows how many parameters a call takes, and
>therefore it doesn't know how many to load into registers... that said, the
>ncg manual *does* have an example rule that ensures that top-of-stack always
>lives in a register; it might be possible to extend this concept. But it may
>not be possible to make ACK code conform to a register-centric ABI without
>passing through interface layers.

I'll definately take a look at the ncg document.

My thought regarding altering em to be register-centric would be along the
lines of r0 being reserved, volatile registers be r1 -> r` (where ` stands
for infinity), and -r1 -> -r` (negative numbered registers) are
non-volatile.  If the backend isn't forced to know the number of parameters
(which are passed in volatile registers), this would leave open a path for
function overloading.  The particular function gets tied after the message
tokens are delivered.  Functions that call additional functions save its
local variables to non-volatile registers, which are always restored before
returning from a function call (by the called function).  Always assume the
volatile registers are destroyed after a function call and the non-volatile
registers are intact.  Non-volatile registers are saved to a stack and
restored from it, just like in the real world.  The assembler would then be
responsible for tying the virtual registers to real ones.

It might be necessary to define "real" registers for stack pointer, link
register, count register, and perhaps a few others, but that's not
difficult.

tim


Gregory T. (tim) Kelly
Owner
Dialectronics.com

P.O. Box 606
Newberry, SC 29108

"Anything war can do, peace can do better."  -- Bishop Desmond Tutu

[Tack-devel] ack.out vs. ELF

[David G. <dg...@co...>]

I've been reading the discussion of file formats with a certain amount of=

interest, but I suspect people are reading more into it than there actual=
ly is...

It's worth remembering that the ACK isn't just a toolchain, it's a *platf=
orm*.
The em specification dictates a certain (flexible) program layout; there'=
s
even an interpreter that implements the VM. An em program communicates wi=
th
the outside world via system calls (the MON em opcode; the model is very
similar to the way V7 Unix worked, even down to the syscall numbering). T=
he
ACK comes with its own portable libc that uses these syscalls to do all t=
he
work; I suspect a lot of the build confusion comes from mistaking headers=

intended to be used by programs compiled *by* the ACK with headers used b=
y the
ACK itself.

As a result of this, the ACK knows absolutely nothing about shared librar=
ies.

However, there's no reason why we have to use the ACK's platform. It's
perfectly possible to compile a program referring to headers in /usr/incl=
ude,
link it with led into a partially linked object file (because led can't l=
ink
to shared libraries, of course), exporting it to ELF (using a program yet=
 to
be written), and then linking the result to shared libraries using the
platform linker.

If you do this then you'll lose the portability across platforms that the=
 ACK
was originally aiming for, but frankly, that's not particularly important=

these days.

I am planning to write two ack.out converters at some point, one for ELF =
and
one for a simple binary image; however, I don't know a lot about ELF and =
so
this would require a lot of reading up on... it may be easier instead to
persuade someone who knows about such things to add ack.out support to th=
e GNU
binutils toolchain instead, so that ld can read them in directly. ack.out=

doesn't look very complicated.

--=20
+- David Given --McQ-+ "Gaping from its single obling socket was
|  dg...@co...    | scintillating, many fauceted scarlet emerald..."
| (dg...@ta...) | --- Jim Theis, _The Eye of Argon_ (spelling
+- www.cowlark.com --+ original)

[Tack-devel] CVS update

[David G. <dg...@co...>]

Well, the mailing list seems to have come to life again --- not sure what=

happened; I think SF's servers went mad. I suspect that the current discu=
ssion
will continue to dribble in a fairly disjointed manner for some hours yet=
=2E

I'd just like to say that, enthusiasm renewed, I've managed to get the ne=
w
build system up and running in head of CVS. (I did actually post a few da=
ys
ago with an earlier update, but it seems to have gotten lost.) What you g=
et is:

* The ANSI & K&R C compilers
* The Pascal compiler
* Backends for most platforms (except Sparc related)
* All the optimisers (global, peephole, target)
* A reasonable collection of utilities, including the linker and libraria=
n

What you *don't* get are any run-time libraries, so you can't actually us=
e it
for generated real code yet, but it's perfectly suitable for experimentin=
g
with the code generators. I'll reemphasise that this is not a release; it=
's
just a point where head of CVS happens to build.

Plus, I got it all building on NetBSD as well as Linux (although I'm havi=
ng to
do 'ulimit -n 256' on SF's build farm to make it build).

If you want to use this, then:

* You will need to install LLgen 1.0.2 off the website (this is now built=

separately).
* Check out head of CVS via anonymous CVS.
* Do: ./pm configure
* Do: ./pm
* Wait.

pm is Prime Mover, which has its own project on pm.sf.net (currently just=
 an
SVN repository). It still has bugs and it's undocumented, but it's workin=
g
rather well, if I say so myself, and is *far* more straightforward to use=
 than
the ACK's old build system. Its steering files are called 'pmfile'.

It's currently set up to build everything into /tmp/ack-temp/staging.
Eventually, this will get copied out to a destination install directory, =
but
for now the ACK is configured to run from there. So, you can compile a C
program with:

cat > test.c <<EOF
extern int printf(const char* format, ...);
int main(int argc, char* argv[])
{
	printf("Hello, world!\n");
	return 0;
}
EOF
/tmp/ack-temp/staging/bin/ack -v -O6 -ansi test.c -mi386 -c.s

=2E..and a Pascal program with:

cat > test.p <<EOF
program test(output);
begin
writeln('Hello, world!')
end.
EOF
/tmp/ack-temp/staging/bin/ack -v -O6 test.p -mi386 -c.s

(If you get errors when trying different architectures, such as ARM, it's=

probably because noone's gotten round to writing a global optimiser
description file yet (see util/ego/descr). Try -O1.)

Maybe I should try Occam next...

--=20
+- David Given --McQ-+ "Gaping from its single obling socket was
|  dg...@co...    | scintillating, many fauceted scarlet emerald..."
| (dg...@ta...) | --- Jim Theis, _The Eye of Argon_ (spelling
+- www.cowlark.com --+ original)

[Tack-devel] test posting

[Timo S. <tim...@ri...>]

please ignore

Re: [Tack-devel] ACK compiles on NetBSD-macppc, sort of...

[Gregory T. (t. K. <gt...@di...>]

At 10:51 AM -0400 7/22/06, Perry E. Metzger wrote:
>Anyway, I'm bowing out of this discussion. I don't think I'm adding
>anything any more.

On the contrary, I believe you are a greater asset to David's efforts than I=
 am, so it is I that will take leave of the forum.  My goals are not the=
 same as NetBSD's, and you obviously have a much better chance of=
 successfully porting ACK to NetBSD than I do.  Rather than tax the forum=
 with my noise and chase off valuable resources, I will withdraw.

Good luck to all.

sincerely,
tim

Gregory T. (tim) Kelly
Owner
Dialectronics.com

P.O. Box 606
Newberry, SC 29108

"Anything war can do, peace can do better."  -- Bishop Desmond Tutu

Re: [Tack-devel] ACK compiles on NetBSD-macppc, sort of...

[Perry E. M. <pe...@pi...>]

"Gregory T. (tim) Kelly" <gt...@di...> writes:
> At 4:13 PM -0400 7/21/06, Perry E. Metzger wrote:
>>>>Well, this is an assembler issue, right? It should be straightforward
>>>>to generate ELF instead of other formats. You pretty much *need* ELF
>>>>on all modern platforms -- Linux and all the rest use ELF too. You
>>>>probably also need to be able to properly embed debugging symbols...
>>>
>>> I can't answer that at this time.
>>
>>What is there to answer? :)
>
> Whether or not it is better to break the structure and generate ELF
> natively, or keep the ACK structure and convert ack.out to ELF.

In the end the two are equivalent provided you can pass directives
through all the way to the code that produces the ultimate ELF
executable.

>>> It is probably important to consider that Tanenbaum's group went
>>> with multiple address space segmentation for a virtual memory model,
>>> while the rest of the world went with singe address space paging.
>>
>>Er, huh????
>
> It would seem to me that the virtual memory model used by an
> operating system would have some impact on choices of file formats.

Not much.

>>All modern operating systems are
>>demand paged. There isn't even support for segmentation on PPC -- the
>>x86 alone of modern processors still sort of has it but it is a rather
>>lame holdover in the architecture.
>
> There's a difference between operating systems lacking support for
> PowerPC's segmentation model and PowerPC lacking support for
> segmentation.  While almost all operating systems lack support for
> PPC's segmentation model, PPC paged segmentation itself is quite
> robust (see Section 7.5 of the Programming Environment Manual, in
> particular the discussion on virtual segment IDs).

I stand partially corrected -- PPC has segments, but they are still
not segments in the old fashioned sense from the days when OSes
swapped entire processes in and out at once.

In any case, none of this makes any difference to whether you are
going to run on Linux or *BSD or anything else if you don't support
ELF.

Anyway, I'm bowing out of this discussion. I don't think I'm adding
anything any more.

Perry

Re: [Tack-devel] back-end tables, assemblers, and front-ends

[Gregory T. (t. K. <gt...@di...>]

At 7:46 PM -0400 7/21/06, David Given wrote:
>[...]
>> LDR R7,[R13,#8]
>
>...is accessing function parameters, which seem to be passed on the stack. =
The
>reason why you're not seeing the arguments in r0-r4 is that they're just no=
t
>there!

Then what does the ARM assembler code look like prior to calling count()? =
 Does it set up the values for min and max on the stack or somewhere else,=
 like r0 and r1?  That code wasn't posted, so I wasn't able to determine=
 where the parameters were in reality compared to where they were expected=
 to be.

tim

Gregory T. (tim) Kelly
Owner
Dialectronics.com

P.O. Box 606
Newberry, SC 29108

"Anything war can do, peace can do better."  -- Bishop Desmond Tutu

Re: [Tack-devel] back-end tables, assemblers, and front-ends

[David G. <dg...@co...>]

Gregory T. (tim) Kelly wrote:
[...]
> STMFD R12<,{R13,R14}
> MOV R13,R12

I think this is using a radically different ABI than most systems use. Th=
is
looks like it's using R12 as the stack pointer and R13 as the frame point=
er.
This means that after this:

[...]
> SUB R12,R12,#8
> STMFD R12<,{R7,R6}

=2E..it's indexing four locals off R12 (two uninitialised variables and t=
he old
values of r6 and r7, and this:

[...]
> LDR R7,[R13,#8]

=2E..is accessing function parameters, which seem to be passed on the sta=
ck. The
reason why you're not seeing the arguments in r0-r4 is that they're just =
not
there!

The stack layout they're using seems to look like this (assuming sp is as=
 it
is after the function prologue):

sp+0C max
sp+08 min
sp+04 old fp  \ return frame
sp+00 old pc  /
fp+0C unused local
fp+08 unused local
fp+04 old r6
fp+00 old r7

This is a completely alien ABI to the entire rest of the ARM world, inclu=
ding
RiscOS, which the port was done *for*... I think they can get away with i=
t
because RiscOS executables are standalone binary blobs that communicate w=
ith
the outside world via system calls, which switch stacks. So binary
compatibility is unnecessary.

It does seem to be credited to "an anonymous student at the Vrije Univers=
iteit"...

--=20
+- David Given --McQ-+ "Gaping from its single obling socket was
|  dg...@co...    | scintillating, many fauceted scarlet emerald..."
| (dg...@ta...) | --- Jim Theis, _The Eye of Argon_ (spelling
+- www.cowlark.com --+ original)

Re: [Tack-devel] ACK compiles on NetBSD-macppc, sort of...

[Gregory T. (t. K. <gt...@di...>]

At 4:13 PM -0400 7/21/06, Perry E. Metzger wrote:
>>>Well, this is an assembler issue, right? It should be straightforward
>>>to generate ELF instead of other formats. You pretty much *need* ELF
>>>on all modern platforms -- Linux and all the rest use ELF too. You
>>>probably also need to be able to properly embed debugging symbols...
>>
>> I can't answer that at this time.
>
>What is there to answer? :)

Whether or not it is better to break the structure and generate ELF=
 natively, or keep the ACK structure and convert ack.out to ELF.

>> It is probably important to consider that Tanenbaum's group went
>> with multiple address space segmentation for a virtual memory model,
>> while the rest of the world went with singe address space paging.
>
>Er, huh????

It would seem to me that the virtual memory model used by an operating=
 system would have some impact on choices of file formats.  Beyond what is=
 needed to get ACK running on NetBSD, I am not particularly concerned with=
 ensuring ELF is the native file format.

>> Rather than having to go through some of the very twisted hoops (and
>> at least on PPC 32-bit ELF, broken hoops) to achieve various memory
>> protections for each section, Minix just puts each section in its
>> own address space (segment).=20
>
>Er, the original Minix didn't have a choice, and modern Minix is
>demand paged, not segment swapped.=20

That statement would be in direct contradiction to "Operating Systems:=
 Design and Implementation" (Tanenbaum, Woodhull) page 420:

"Memory management in Minix 3 is simple: paging is not used at all.  Minx 3=
 memory management as we will discuss it here does not include swapping eith=
er."

Additionally, on page 422:

"In Minix 3, however, the default is to compile programs to use separate I=
 and D space."

>All modern operating systems are
>demand paged. There isn't even support for segmentation on PPC -- the
>x86 alone of modern processors still sort of has it but it is a rather
>lame holdover in the architecture.

There's a difference between operating systems lacking support for PowerPC's=
 segmentation model and PowerPC lacking support for segmentation.  While=
 almost all operating systems lack support for PPC's segmentation model, PPC=
 paged segmentation itself is quite robust (see Section 7.5 of the=
 Programming Environment Manual, in particular the discussion on virtual=
 segment IDs).  While I don't have any direct experience confirming this,=
 there are plenty of discussions on the web indicating that 32-bit PowerPC=
 ELF breaks a lot of PPC's possibilities.  There is a reason why almost all=
 PPC vendors use file formats other than ELF (PEF, IBM's modified xcoff,=
 Mach-O for all its warts), and there are reasons I am reluctant to spend=
 any time more than absolutely necessary making ACK work with ELF.

In NetBSD/macppc the default is just to map the segment registers in 256M=
 increments for the full 4G/32 bit address space.  That is only one possible=
 use of the segment registers, but expected in a single address space paged=
 operating system.  There are 16 segment registers, each capable of=
 containing a 24 bit segment id and protection state for that 256M space. =
 That's a lot of support for segmentation.  Or at least, that's my=
 understanding of the paged segmentation mode and that it is widely underuti=
lized.

tim

Gregory T. (tim) Kelly
Owner
Dialectronics.com

P.O. Box 606
Newberry, SC 29108

"Anything war can do, peace can do better."  -- Bishop Desmond Tutu

Re: [Tack-devel] ACK compiles on NetBSD-macppc, sort of...

[Gregory T. (t. K. <gt...@di...>]

At 4:13 PM -0400 7/21/06, Perry E. Metzger wrote:
>> Unfortunately off_t is defined in several of the ACK headers.
>
>Then remove it. That is POSIXly incorrect. Instead, any header that
>needs off_t defined should include it.

David,

What are your thoughts on this?  If you sign off or indicate it is already changed in your tree, I'll delete it at my end.

tim

Gregory T. (tim) Kelly
Owner
Dialectronics.com

P.O. Box 606
Newberry, SC 29108

"Anything war can do, peace can do better."  -- Bishop Desmond Tutu

Re: [Tack-devel] ACK compiles on NetBSD-macppc, sort of...

[Perry E. M. <pe...@pi...>]

"Gregory T. (tim) Kelly" <gt...@di...> writes:
> At 8:53 AM -0400 7/21/06, Perry E. Metzger wrote:
>>You should structure things so that ACK doesn't know what the type of
>>off_t is -- different platforms have it different lengths. This is
>>pretty straightforward to do, and POSIX conforming code does not need
>>to know what length it is in general.
>
> Unfortunately off_t is defined in several of the ACK headers.

Then remove it. That is POSIXly incorrect. Instead, any header that
needs off_t defined should include it.

> I've generally defeated this with directives:
>
> #if   !defined(BSD)
> #define long off_t
> #endif

Don't bother. It is wrong. Dike it out, and include the proper include
file, or you won't have portable code.

Remember ACK predates modern systems by quite some years. Much of what
it does made sense 15 or 18 years ago, but not now.

>>> and a deprecated function call (ftime, replaced by gettimeofday),
>>
>>It is in libcompat, but in general, you should make things stick to
>>the lowest common denominator of modern POSIX. getttimeofday is
>>available on just about everything.
>
> The final version of moncalls.c can be changed to reflect this.  I
> left ftime in place for pre-BSD4.2 systems.

If you can find me a single pre-BSD4.2 system that's operating in the
wild, I'd be interested to hear about it. Indeed, I'd be interested in
hearing of any 4.2 system that is still booted. I suspect you would be
hard pressed even to find an operating 4.3 system, though there are
one or two vaxes that I know of that are sometimes booted into 4.3 for
kicks.

>>> I'm still trying to put the pieces together, but I think one step
>>> that will initially prevent usability on BSD is going to be the
>>> lack of an ack.out -> elf linker/converter.  It took me a day or
>>> two to realize ack.out is not a.out, but then again I'm a little
>>> slow on the uptake.
>>
>>Well, this is an assembler issue, right? It should be straightforward
>>to generate ELF instead of other formats. You pretty much *need* ELF
>>on all modern platforms -- Linux and all the rest use ELF too. You
>>probably also need to be able to properly embed debugging symbols...
>
> I can't answer that at this time.

What is there to answer? :)

> The platform specific converter (which I believe is in step 8) would
> handle the ack.out->elf conversion.  Going to ELF natively would
> require breaking with the structure, if I understand the structure
> correctly.

You may not have much of a choice...

> It is probably important to consider that Tanenbaum's group went
> with multiple address space segmentation for a virtual memory model,
> while the rest of the world went with singe address space paging.

Er, huh????

> Rather than having to go through some of the very twisted hoops (and
> at least on PPC 32-bit ELF, broken hoops) to achieve various memory
> protections for each section, Minix just puts each section in its
> own address space (segment). 

Er, the original Minix didn't have a choice, and modern Minix is
demand paged, not segment swapped. All modern operating systems are
demand paged. There isn't even support for segmentation on PPC -- the
x86 alone of modern processors still sort of has it but it is a rather
lame holdover in the architecture.

Perry