[CZT-Devel] Reducing memory usage: Signatures are the biggest culprit?

[Andrius V. <and...@ne...>]

Hi,

Since I had a profiler on CZT running for catching some leftover debugging
code, I thought I would give a quick look into the CZT memory usage as well.

I have discovered that most of the large amount of ZNames that are created
are referenced from typechecker Signatures (via NameTypePairs). This is the
case because when creating type signatures during typechecking, the
Signatures are duplicated in a lot of places.

A prominent example of this is in
net.sourceforge.czt.typechecker.z.ExprChecker:94 (method visitRefExpr).
This bit of code calculates the type of a RefExpr. If a reference is to a
Schema, it duplicates the Schema signature (assigns new IDs to all its
names) and then uses the new signature within the power type.

As a quick test, I tried removing this duplication and instead reuse the
original schema signature (with original ZName ids): `Signature sig =
signature;`

This single change reduced the memory consumption of typechecked `spec.tex`
by about 60%!

This lead me to thinking - do signatures really need to be duplicated
everywhere? I imagine that the signature of schema A would be the same
everywhere? Can we reuse the signature, or is it important to duplicate and
assign new IDs to the name?

The massive space saving becomes quite obvious when you think about it. If
we have schema references, every RefExpr would duplicate the whole schema
definition when typechecked, hence creating a lot of new ZName instances
and consuming all this memory.

Leo advised that Tim would be the person to ask about the Signature
duplication in typechecker? Is it really necessary, or can we reuse the
objects?

Best regards,
Andrius