Re: [CZT-Devel] Reducing memory usage: Signatures are the biggest culprit?

[Tim M. <tm...@un...>]

Oh, and another thought: the typechecker has an option "use name IDs". I 
don't remember what this did, but presumably there was some notation 
that name IDs were important for name equality for some packages (I 
think Petra added it for the Z rules package). We could change the 
typechecker such that if this was turned off, we simply didn't record 
name IDs, which would mean new signatures would not need to be created.

Tim

On 13/04/13 00:00, Andrius Velykis wrote:
> Hi,
>
> Since I had a profiler on CZT running for catching some leftover
> debugging code, I thought I would give a quick look into the CZT memory
> usage as well.
>
> I have discovered that most of the large amount of ZNames that are
> created are referenced from typechecker Signatures (via NameTypePairs).
> This is the case because when creating type signatures during
> typechecking, the Signatures are duplicated in a lot of places.
>
> A prominent example of this is in
> net.sourceforge.czt.typechecker.z.ExprChecker:94 (method visitRefExpr).
> This bit of code calculates the type of a RefExpr. If a reference is to
> a Schema, it duplicates the Schema signature (assigns new IDs to all its
> names) and then uses the new signature within the power type.
>
> As a quick test, I tried removing this duplication and instead reuse the
> original schema signature (with original ZName ids): `Signature sig =
> signature;`
>
> This single change reduced the memory consumption of typechecked
> `spec.tex` by about 60%!
>
> This lead me to thinking - do signatures really need to be duplicated
> everywhere? I imagine that the signature of schema A would be the same
> everywhere? Can we reuse the signature, or is it important to duplicate
> and assign new IDs to the name?
>
> The massive space saving becomes quite obvious when you think about it.
> If we have schema references, every RefExpr would duplicate the whole
> schema definition when typechecked, hence creating a lot of new ZName
> instances and consuming all this memory.
>
> Leo advised that Tim would be the person to ask about the Signature
> duplication in typechecker? Is it really necessary, or can we reuse the
> objects?
>
> Best regards,
> Andrius
>
>
>
> ------------------------------------------------------------------------------
> Precog is a next-generation analytics platform capable of advanced
> analytics on semi-structured data. The platform includes APIs for building
> apps and a phenomenal toolset for data science. Developers can use
> our toolset for easy data analysis & visualization. Get a free account!
> http://www2.precog.com/precogplatform/slashdotnewsletter
>
>
>
> _______________________________________________
> CZT-Devel mailing list
> CZT...@li...
> https://lists.sourceforge.net/lists/listinfo/czt-devel
>