czt-devel — For programmers using the CZT code

Re: [CZT-Devel] Issues Building CZT in Windows

[Andrius V. <an...@ve...>]

Dear Jon Lockhart,

Thanks for reporting your experiences building CZT. Could you send
more details about the error that is occurring for you? The last bits
of Maven build output would be good, or if tests are failing, try
attaching surefire test reports with the failures.

Also, there have been some changes since the last release, so the
installation instructions on the website may not be fully up-to-date.

The SVN repository is no longer updated (currently read-only and will
be removed soon). All new code goes to the Git repository. There are
more up-to-date instructions on building CZT available in the Git
repository: http://czt.git.sourceforge.net/git/gitweb.cgi?p=czt/czt;a=blob_plain;f=INSTALL.md

Finally, we have quite nice Eclipse plug-ins in the latest version for
CZT as an alternative to jEdit - the update site is built together
with the whole CZT.

I am looking into setting up a continuous integration (CI) system for
CZT soon. Hopefully then we will have "nightly" downloads and there
will be less need to build everything manually.

Best regards,
~Andrius Velykis


On Tue, Jul 17, 2012 at 9:00 PM, Jon Lockhart <ja...@bu...> wrote:
> Dear CZT Developers,
>
> I am having an issue building versions of the CZT tool set in Windows, and I
> am wondering if anyone has had a similar issue recently or in the past and
> has a suggestion for handling this problems, or a solution, which would be
> more beneficial in my case.
>
> I first started with following the installation instructions on the site,
> and downloaded an available copy of the svn version of 1.5, git bash, maven,
> and jedit. While trying to build the czt files, after got through all the
> installation instructions for the various tool sets, I ran into a dependency
> error on typechecker. Now I know that in the install it says that dependency
> errors will arise and that I just continue to try reinstalling but after
> several attempts all resulting in the same error, I decided that I should
> try getting the latest version of CZT from the git repository, seeing that
> it was a newer version and maybe some of these dependency errors would be
> resolved. Unfortunately this was not the case and in fact I also ran into an
> error in one of the tests for typechecker. So this time around I had a test
> error occur every time for type checker, and then I am always running into
> the same dependency error with the rules with the latest version from git.
>
> I have tried downloading version 1.5 and 1.6 again multiple times thinking
> that there was possibly an error in the download I had, I have tried
> increasing the java stack in my maven variables to 512 from 256, and I have
> even tried looking through the mail repository to see if anyone else has had
> an issue with compiling these more recent versions on Windows, but I didn't
> see any.
>
> As I said I am very open to fixes, solutions, or a work around for getting
> CZT compiled on a Windows box, so that I can import the necessary jar files
> into jEdit so I can start using that for my writing of Zed specifications.
>
> Thanks,
> Jon Lockhart
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> CZT-Devel mailing list
> CZT...@li...
> https://lists.sourceforge.net/lists/listinfo/czt-devel
>

[CZT-Devel] Issues Building CZT in Windows

[Jon L. <ja...@bu...>]

Dear CZT Developers,

I am having an issue building versions of the CZT tool set in Windows, and
I am wondering if anyone has had a similar issue recently or in the past
and has a suggestion for handling this problems, or a solution, which would
be more beneficial in my case.

I first started with following the installation instructions on the site,
and downloaded an available copy of the svn version of 1.5, git bash,
maven, and jedit. While trying to build the czt files, after got through
all the installation instructions for the various tool sets, I ran into a
dependency error on typechecker. Now I know that in the install it says
that dependency errors will arise and that I just continue to try
reinstalling but after several attempts all resulting in the same error, I
decided that I should try getting the latest version of CZT from the git
repository, seeing that it was a newer version and maybe some of these
dependency errors would be resolved. Unfortunately this was not the case
and in fact I also ran into an error in one of the tests for typechecker.
So this time around I had a test error occur every time for type checker,
and then I am always running into the same dependency error with the rules
with the latest version from git.

I have tried downloading version 1.5 and 1.6 again multiple times thinking
that there was possibly an error in the download I had, I have tried
increasing the java stack in my maven variables to 512 from 256, and I have
even tried looking through the mail repository to see if anyone else has
had an issue with compiling these more recent versions on Windows, but I
didn't see any.

As I said I am very open to fixes, solutions, or a work around for getting
CZT compiled on a Windows box, so that I can import the necessary jar files
into jEdit so I can start using that for my writing of Zed specifications.

Thanks,
Jon Lockhart

[CZT-Devel] CZT build infrastructure updates

[Andrius V. <an...@ve...>]

Hi everyone,

I want to share some updates regarding recent CZT build infrastructure updates.

The main change is that I have moved CZT Eclipse plug-in build to
Maven using Eclipse Tycho (http://www.eclipse.org/tycho/). It allows
building Eclipse plug-ins using Maven and fetches necessary
dependencies in Maven style. So now we can build CZT Eclipse from
command line, without actually having Eclipse running.

Due to this move, several other things have been affected:
- First, Maven requirement got bumped to Maven 3 (3.0.4 in particular
is recommended).
- Also, there is additional activity in the build job where Tycho
looks up dependencies for building CZT Eclipse (look for 'p2'
messages). This check happens on every build - if you want to avoid it
(e.g. when doing multiple rebuilds in a short period of time), you can
run Maven in "offline" mode with --offline argument.
- To build Eclipse plugins, run 'clean install' on the whole repo -
CZT Eclipse 'library' plug-in requires dependency jars fully built (I
am still looking into the best way for building this).
- Renamed Eclipse plugin directories according to Eclipse conventions,
e.g. to 'net.sourceforge.czt.eclipse'
- CZT Eclipse update site is now built automatically as well. After
build, you can point your Eclipse 'install new software' to your newly
built update site at
'file://<CZT_HOME>/eclipse/net.sourceforge.czt.eclipse.repository/target/repository'


In addition to that, I have done some steps to support building CZT
from within Eclipse IDE (my IDE of choice). One of the ways how
Eclipse supports Maven is using m2e (http://www.eclipse.org/m2e/).
However, it requires explicit 'lifecycle mapping' support. I have
added basic plugins to support CZT build - they are available at
<CZT_HOME>/eclipse/m2e and in the final update site. The full support
for m2e is not there yet - but CZT projects in Eclipse IDE are now
usable.


With the change to Git for version control and these build updates, I
have also updated INSTALL file to reflect this. It also has some
information on using Eclipse IDE to build CZT. I have written the
INSTALL.md file using Markdown
(http://daringfireball.net/projects/markdown/) - it is a popular
format for such documents (e.g. in GitHub, etc) and recent Maven Sites
support it as well. So it should not pose problems to regenerate CZT
website for the next release.


Finally, there have been some other small build changes. For one, I
renamed some of CZT Maven plugins, e.g. maven-gnast-plugin to
gnast-maven-plugin. The new Maven versions reserve 'maven-***-plugin'
for official Maven plugins and ask to change the name to
'***-maven-plugin'. Check the Git logs for details of any other
changes.

Thanks,
~Andrius

[CZT-Devel] CZT source code migrated to Git

[Andrius V. <an...@ve...>]

Hi everyone,

Thanks for the comments :) It looks like at least the majority of administrators prefer to go with Git. So since there has not been very recent activity in SVN commits, I thought I may just migrate it.

I have added the Git repository to the project and migrated SVN contents to it. If anyone is interested, I used SubGit (http://subgit.com/) to perform the migration, because it has nicer way of handling file ignores and tags than the default git-svn. Afterwards there was some manual tinkering with the tags (converting from lightweight to annotated), but in general the process is quite straightforward.

If you go to https://sourceforge.net/projects/czt/develop there are new links to clone the Git repository. There is a read-only link for public access, as well as developer access via ssh when you log-in.

From there on it looks like there are no specific requirements to use Git within SourceForge. The guidelines and some hints are available in their Git documentation: https://sourceforge.net/apps/trac/sourceforge/wiki/Git . They recommend using the SourceForge e-mail when committing (possibly they would have links to authors in the new platform) - it can be set up in your repository configuration after cloning:

git config user.name "YOUR NAME"
git config user.email "USE...@us..."

I used this convention during migration - everyone's commits are attributed to their SourceForge e-mail.

So I guess from now on Git repository should be the place to commit new code to CZT. To avoid accidental commits to SVN, I have disabled write access to that repository. It is still available as read only though - should we remove it altogether? Or provide some 'transitional' time window as a 'backup' - e.g. remove it when everyone is more or less comfortable with the Git repository?

Git repository write access is currently only available to project administrators. Should it be enabled for all developers? Looking at repository logs a lot of them have not contributed anything for a long time.. Otherwise there may be less incentive to give full write access to occasional contributors. Since Git is distributed version control system, I think the preferred way is for everyone to fork the main repository and then their changes are pulled back by the main repository maintainers..

Thanks for the support! This is the first step - now we can start looking into merging the main branches and releasing a new version sometime. Please report if there are any problems with the migrated repository..

~Andrius

Re: [CZT-Devel] Moving CZT source code repository to DVCS (Git vs Mercurial poll)

[Leo F. <leo...@ne...>]

Hi andrius, 

Thanks for this. I've used a bit of both, and am happy to go for git as well... The mac IDE for it is great too :-)

So, git 4 x 0 mercurial...

Leo



On 13 Jun 2012, at 00:36, "Tim Miller" <tm...@un...> wrote:

> Hi Andrius,
> 
> Thanks for the descriptions. I'm another vote for Git, mainly because I 
> already use it on other projects so won't need to hunt around for 
> tutorials when I check out the CZT repository next! Also, the local 
> branching is great (not sure what Mercurial does there).
> 
> So it's Git 3 - 0 Mercurial so far.
> 
> Cheers,
> Tim
> 
> On 13/06/12 06:44, Mark Utting wrote:
>> Andrius,
>> 
>> Thanks for giving such a good summary of the pros and cons of each system.
>> 
>> Yes, I'm in favour of moving CZT to a distributed version control
>> system, and would vote for GIT as well. (I haven't used it a lot yet,
>> but have several other projects that I'd like to move over, so I'd like
>> to get more experience with it).
>> 
>> Git: +1 = 2
>> Mercurial = 0.
>> 
>> Cheers
>> Mark
>> 
>> 
>> On 13 June 2012 04:31, Andrius Velykis <an...@ve...
>> <mailto:an...@ve...>> wrote:
>> 
>>    Hi everyone,
>> 
>>    I work with Leo in Newcastle and have a proposal to move CZT
>>    repository to a distributed version control system (DVCS), e.g. Git or
>>    Mercurial. Would like your comments about this :)
>> 
>>    Over the last year we have added some new features to CZT, including a
>>    transactional SectionManager, updated Eclipse plug-ins and support for
>>    Z/EVES theorem prover. It would be nice to release it all in a new CZT
>>    version.
>> 
>>    However, currently we have two branches in the repository. When we
>>    developed transactional SectionManager (which tracks key
>>    dependencies), we branched off the main SVN trunk. Now we need to
>>    merge back to release the code. From what I gather, SVN merging is
>>    trickier than in modern distributed version control systems (DVCS) and
>>    I am a bit afraid that if we move to a DVCS later, the merge history
>>    may get lost.
>> 
>>    Leo and I proposed to move to DVCS in an earlier email (and it was
>>    supported by Tim Miller). So what if we move CZT repository to DVCS
>>    (Git or Mercurial) now and merge afterwards, followed by a release of
>>    new CZT version.
>> 
>>    There are many benefits of using a DVCS and there are many "why"
>>    explanations on the web, e.g. here:
>>    http://blogs.atlassian.com/2012/02/version-control-centralized-dvcs/.
>>    SourceForge supports both Git and Mercurial so it is our choice which
>>    way CZT should go for. Both systems are comparable in functionality,
>>    so eventually it is the developer choice which we prefer to use.
>>    Again, there are MANY Git vs. Mercurial comparisons online, e.g. here
>>    (with links to more detailed arguments):
>>    http://www.atlassian.com/dvcs/overview/dvcs-options-git-or-mercurial
>> 
>>    I would like to start a developer poll about this :) Here is is my
>>    choice with some arguments following:
>> 
>>    - Git: +1 (Andrius)
>>    - Mercurial: 0
>> 
>>    I choose Git over Mercurial because it suits my style better. I have
>>    used both for some time but Git emerged as my DVCS of choice. I am a
>>    fan of the "staging area" and partial commits. Sometimes I develop a
>>    couple things at the same time and may only want to commit parts of
>>    the changes as well-isolated commits. Together with nice UIs (I am
>>    using SourceTree on Mac) I am able to pick specific lines in files to
>>    commit.
>> 
>>    Furthermore, I like the lightweight branches and have benefitted
>>    frequently from the 'rebasing' and history rewriting when needed. When
>>    using Mercurial, similar things can be achieved with 'bookmarks' and
>>    'mercurial queues', but the Git counterparts feel more like
>>    first-class citizens. I will not go far into details here.
>> 
>>    The worry about git's "history-destructive" changes can be alleviated
>>    with a change in working model. For example, student contributions
>>    could be achieved by them forking the main repository and then doing
>>    'pull requests' ('merge requests' in SourceForge) thus avoiding giving
>>    write-access to the main repository. Furthermore, by default
>>    SourceForge Git repositories have the 'denyFastForwards' flag set,
>>    which disallows 'force' pushes that are destructive to the history. So
>>    history is safe in this case (also there should be a "policy" among
>>    developers to have no 'force' pushes).
>> 
>>    Finally, Git seems to have the edge in market share at the moment.
>>    Considering CZT toolchain, here is a quick glance: Maven - SVN with
>>    Git mirrors, CUP - SVN, JFlex - SVN, Eclipse - Git, jEdit - SVN/Git.
>>    So it looks like the ones which are moving to DVCS, are choosing Git.
>> 
>>    In the end, I am not 'allergic' to any of the DVCS choices and will be
>>    glad to go with the public choice.
>> 
>>    When decided, I could handle the repository move from SVN to the DVCS
>>    of choice in SourceForge. While on the similar topic, we could also
>>    migrate to the new SourceForge platform (named Allura). This would
>>    require admin rights though.. if everyone is ok with that.
>> 
>>    What do you think about the move? Which system would you prefer? Or
>>    maybe you are against DVCS and would like to stay with SVN?
>> 
>>    Best regards,
>>    ~Andrius Velykis
>> 
>>    ------------------------------------------------------------------------------
>>    Live Security Virtual Conference
>>    Exclusive live event will cover all the ways today's security and
>>    threat landscape has changed and how IT managers can respond.
>>    Discussions
>>    will include endpoint security, mobile security and the latest in
>>    malware
>>    threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>>    _______________________________________________
>>    CZT-Devel mailing list
>>    CZT...@li... <mailto:CZT...@li...>
>>    https://lists.sourceforge.net/lists/listinfo/czt-devel
>> 
>> 
>> 
>> 
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> 
>> 
>> 
>> _______________________________________________
>> CZT-Devel mailing list
>> CZT...@li...
>> https://lists.sourceforge.net/lists/listinfo/czt-devel
> 
> 
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and 
> threat landscape has changed and how IT managers can respond. Discussions 
> will include endpoint security, mobile security and the latest in malware 
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> CZT-Devel mailing list
> CZT...@li...
> https://lists.sourceforge.net/lists/listinfo/czt-devel

Re: [CZT-Devel] Moving CZT source code repository to DVCS (Git vs Mercurial poll)

[Tim M. <tm...@un...>]

Hi Andrius,

Thanks for the descriptions. I'm another vote for Git, mainly because I 
already use it on other projects so won't need to hunt around for 
tutorials when I check out the CZT repository next! Also, the local 
branching is great (not sure what Mercurial does there).

So it's Git 3 - 0 Mercurial so far.

Cheers,
Tim

On 13/06/12 06:44, Mark Utting wrote:
> Andrius,
>
> Thanks for giving such a good summary of the pros and cons of each system.
>
> Yes, I'm in favour of moving CZT to a distributed version control
> system, and would vote for GIT as well. (I haven't used it a lot yet,
> but have several other projects that I'd like to move over, so I'd like
> to get more experience with it).
>
> Git: +1 = 2
> Mercurial = 0.
>
> Cheers
> Mark
>
>
> On 13 June 2012 04:31, Andrius Velykis <an...@ve...
> <mailto:an...@ve...>> wrote:
>
>     Hi everyone,
>
>     I work with Leo in Newcastle and have a proposal to move CZT
>     repository to a distributed version control system (DVCS), e.g. Git or
>     Mercurial. Would like your comments about this :)
>
>     Over the last year we have added some new features to CZT, including a
>     transactional SectionManager, updated Eclipse plug-ins and support for
>     Z/EVES theorem prover. It would be nice to release it all in a new CZT
>     version.
>
>     However, currently we have two branches in the repository. When we
>     developed transactional SectionManager (which tracks key
>     dependencies), we branched off the main SVN trunk. Now we need to
>     merge back to release the code. From what I gather, SVN merging is
>     trickier than in modern distributed version control systems (DVCS) and
>     I am a bit afraid that if we move to a DVCS later, the merge history
>     may get lost.
>
>     Leo and I proposed to move to DVCS in an earlier email (and it was
>     supported by Tim Miller). So what if we move CZT repository to DVCS
>     (Git or Mercurial) now and merge afterwards, followed by a release of
>     new CZT version.
>
>     There are many benefits of using a DVCS and there are many "why"
>     explanations on the web, e.g. here:
>     http://blogs.atlassian.com/2012/02/version-control-centralized-dvcs/.
>     SourceForge supports both Git and Mercurial so it is our choice which
>     way CZT should go for. Both systems are comparable in functionality,
>     so eventually it is the developer choice which we prefer to use.
>     Again, there are MANY Git vs. Mercurial comparisons online, e.g. here
>     (with links to more detailed arguments):
>     http://www.atlassian.com/dvcs/overview/dvcs-options-git-or-mercurial
>
>     I would like to start a developer poll about this :) Here is is my
>     choice with some arguments following:
>
>     - Git: +1 (Andrius)
>     - Mercurial: 0
>
>     I choose Git over Mercurial because it suits my style better. I have
>     used both for some time but Git emerged as my DVCS of choice. I am a
>     fan of the "staging area" and partial commits. Sometimes I develop a
>     couple things at the same time and may only want to commit parts of
>     the changes as well-isolated commits. Together with nice UIs (I am
>     using SourceTree on Mac) I am able to pick specific lines in files to
>     commit.
>
>     Furthermore, I like the lightweight branches and have benefitted
>     frequently from the 'rebasing' and history rewriting when needed. When
>     using Mercurial, similar things can be achieved with 'bookmarks' and
>     'mercurial queues', but the Git counterparts feel more like
>     first-class citizens. I will not go far into details here.
>
>     The worry about git's "history-destructive" changes can be alleviated
>     with a change in working model. For example, student contributions
>     could be achieved by them forking the main repository and then doing
>     'pull requests' ('merge requests' in SourceForge) thus avoiding giving
>     write-access to the main repository. Furthermore, by default
>     SourceForge Git repositories have the 'denyFastForwards' flag set,
>     which disallows 'force' pushes that are destructive to the history. So
>     history is safe in this case (also there should be a "policy" among
>     developers to have no 'force' pushes).
>
>     Finally, Git seems to have the edge in market share at the moment.
>     Considering CZT toolchain, here is a quick glance: Maven - SVN with
>     Git mirrors, CUP - SVN, JFlex - SVN, Eclipse - Git, jEdit - SVN/Git.
>     So it looks like the ones which are moving to DVCS, are choosing Git.
>
>     In the end, I am not 'allergic' to any of the DVCS choices and will be
>     glad to go with the public choice.
>
>     When decided, I could handle the repository move from SVN to the DVCS
>     of choice in SourceForge. While on the similar topic, we could also
>     migrate to the new SourceForge platform (named Allura). This would
>     require admin rights though.. if everyone is ok with that.
>
>     What do you think about the move? Which system would you prefer? Or
>     maybe you are against DVCS and would like to stay with SVN?
>
>     Best regards,
>     ~Andrius Velykis
>
>     ------------------------------------------------------------------------------
>     Live Security Virtual Conference
>     Exclusive live event will cover all the ways today's security and
>     threat landscape has changed and how IT managers can respond.
>     Discussions
>     will include endpoint security, mobile security and the latest in
>     malware
>     threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>     _______________________________________________
>     CZT-Devel mailing list
>     CZT...@li... <mailto:CZT...@li...>
>     https://lists.sourceforge.net/lists/listinfo/czt-devel
>
>
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>
>
>
> _______________________________________________
> CZT-Devel mailing list
> CZT...@li...
> https://lists.sourceforge.net/lists/listinfo/czt-devel

Re: [CZT-Devel] Moving CZT source code repository to DVCS (Git vs Mercurial poll)

[Mark U. <ma...@cs...>]

Andrius,

Thanks for giving such a good summary of the pros and cons of each system.

Yes, I'm in favour of moving CZT to a distributed version control system,
and would vote for GIT as well.  (I haven't used it a lot yet, but have
several other projects that I'd like to move over, so I'd like to get more
experience with it).

Git: +1 = 2
Mercurial = 0.

Cheers
Mark


On 13 June 2012 04:31, Andrius Velykis <an...@ve...> wrote:

> Hi everyone,
>
> I work with Leo in Newcastle and have a proposal to move CZT
> repository to a distributed version control system (DVCS), e.g. Git or
> Mercurial. Would like your comments about this :)
>
> Over the last year we have added some new features to CZT, including a
> transactional SectionManager, updated Eclipse plug-ins and support for
> Z/EVES theorem prover. It would be nice to release it all in a new CZT
> version.
>
> However, currently we have two branches in the repository. When we
> developed transactional SectionManager (which tracks key
> dependencies), we branched off the main SVN trunk. Now we need to
> merge back to release the code. From what I gather, SVN merging is
> trickier than in modern distributed version control systems (DVCS) and
> I am a bit afraid that if we move to a DVCS later, the merge history
> may get lost.
>
> Leo and I proposed to move to DVCS in an earlier email (and it was
> supported by Tim Miller). So what if we move CZT repository to DVCS
> (Git or Mercurial) now and merge afterwards, followed by a release of
> new CZT version.
>
> There are many benefits of using a DVCS and there are many "why"
> explanations on the web, e.g. here:
> http://blogs.atlassian.com/2012/02/version-control-centralized-dvcs/.
> SourceForge supports both Git and Mercurial so it is our choice which
> way CZT should go for. Both systems are comparable in functionality,
> so eventually it is the developer choice which we prefer to use.
> Again, there are MANY Git vs. Mercurial comparisons online, e.g. here
> (with links to more detailed arguments):
> http://www.atlassian.com/dvcs/overview/dvcs-options-git-or-mercurial
>
> I would like to start a developer poll about this :) Here is is my
> choice with some arguments following:
>
> - Git: +1 (Andrius)
> - Mercurial: 0
>
> I choose Git over Mercurial because it suits my style better. I have
> used both for some time but Git emerged as my DVCS of choice. I am a
> fan of the "staging area" and partial commits. Sometimes I develop a
> couple things at the same time and may only want to commit parts of
> the changes as well-isolated commits. Together with nice UIs (I am
> using SourceTree on Mac) I am able to pick specific lines in files to
> commit.
>
> Furthermore, I like the lightweight branches and have benefitted
> frequently from the 'rebasing' and history rewriting when needed. When
> using Mercurial, similar things can be achieved with 'bookmarks' and
> 'mercurial queues', but the Git counterparts feel more like
> first-class citizens. I will not go far into details here.
>
> The worry about git's "history-destructive" changes can be alleviated
> with a change in working model. For example, student contributions
> could be achieved by them forking the main repository and then doing
> 'pull requests' ('merge requests' in SourceForge) thus avoiding giving
> write-access to the main repository. Furthermore, by default
> SourceForge Git repositories have the 'denyFastForwards' flag set,
> which disallows 'force' pushes that are destructive to the history. So
> history is safe in this case (also there should be a "policy" among
> developers to have no 'force' pushes).
>
> Finally, Git seems to have the edge in market share at the moment.
> Considering CZT toolchain, here is a quick glance: Maven - SVN with
> Git mirrors, CUP - SVN, JFlex - SVN, Eclipse - Git, jEdit - SVN/Git.
> So it looks like the ones which are moving to DVCS, are choosing Git.
>
> In the end, I am not 'allergic' to any of the DVCS choices and will be
> glad to go with the public choice.
>
> When decided, I could handle the repository move from SVN to the DVCS
> of choice in SourceForge. While on the similar topic, we could also
> migrate to the new SourceForge platform (named Allura). This would
> require admin rights though.. if everyone is ok with that.
>
> What do you think about the move? Which system would you prefer? Or
> maybe you are against DVCS and would like to stay with SVN?
>
> Best regards,
> ~Andrius Velykis
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> CZT-Devel mailing list
> CZT...@li...
> https://lists.sourceforge.net/lists/listinfo/czt-devel
>

[CZT-Devel] Moving CZT source code repository to DVCS (Git vs Mercurial poll)

[Andrius V. <an...@ve...>]

Hi everyone,

I work with Leo in Newcastle and have a proposal to move CZT
repository to a distributed version control system (DVCS), e.g. Git or
Mercurial. Would like your comments about this :)

Over the last year we have added some new features to CZT, including a
transactional SectionManager, updated Eclipse plug-ins and support for
Z/EVES theorem prover. It would be nice to release it all in a new CZT
version.

However, currently we have two branches in the repository. When we
developed transactional SectionManager (which tracks key
dependencies), we branched off the main SVN trunk. Now we need to
merge back to release the code. From what I gather, SVN merging is
trickier than in modern distributed version control systems (DVCS) and
I am a bit afraid that if we move to a DVCS later, the merge history
may get lost.

Leo and I proposed to move to DVCS in an earlier email (and it was
supported by Tim Miller). So what if we move CZT repository to DVCS
(Git or Mercurial) now and merge afterwards, followed by a release of
new CZT version.

There are many benefits of using a DVCS and there are many "why"
explanations on the web, e.g. here:
http://blogs.atlassian.com/2012/02/version-control-centralized-dvcs/.
SourceForge supports both Git and Mercurial so it is our choice which
way CZT should go for. Both systems are comparable in functionality,
so eventually it is the developer choice which we prefer to use.
Again, there are MANY Git vs. Mercurial comparisons online, e.g. here
(with links to more detailed arguments):
http://www.atlassian.com/dvcs/overview/dvcs-options-git-or-mercurial

I would like to start a developer poll about this :) Here is is my
choice with some arguments following:

- Git: +1 (Andrius)
- Mercurial: 0

I choose Git over Mercurial because it suits my style better. I have
used both for some time but Git emerged as my DVCS of choice. I am a
fan of the "staging area" and partial commits. Sometimes I develop a
couple things at the same time and may only want to commit parts of
the changes as well-isolated commits. Together with nice UIs (I am
using SourceTree on Mac) I am able to pick specific lines in files to
commit.

Furthermore, I like the lightweight branches and have benefitted
frequently from the 'rebasing' and history rewriting when needed. When
using Mercurial, similar things can be achieved with 'bookmarks' and
'mercurial queues', but the Git counterparts feel more like
first-class citizens. I will not go far into details here.

The worry about git's "history-destructive" changes can be alleviated
with a change in working model. For example, student contributions
could be achieved by them forking the main repository and then doing
'pull requests' ('merge requests' in SourceForge) thus avoiding giving
write-access to the main repository. Furthermore, by default
SourceForge Git repositories have the 'denyFastForwards' flag set,
which disallows 'force' pushes that are destructive to the history. So
history is safe in this case (also there should be a "policy" among
developers to have no 'force' pushes).

Finally, Git seems to have the edge in market share at the moment.
Considering CZT toolchain, here is a quick glance: Maven - SVN with
Git mirrors, CUP - SVN, JFlex - SVN, Eclipse - Git, jEdit - SVN/Git.
So it looks like the ones which are moving to DVCS, are choosing Git.

In the end, I am not 'allergic' to any of the DVCS choices and will be
glad to go with the public choice.

When decided, I could handle the repository move from SVN to the DVCS
of choice in SourceForge. While on the similar topic, we could also
migrate to the new SourceForge platform (named Allura). This would
require admin rights though.. if everyone is ok with that.

What do you think about the move? Which system would you prefer? Or
maybe you are against DVCS and would like to stay with SVN?

Best regards,
~Andrius Velykis

Re: [CZT-Devel] Error rebuilding CZT.

[Frank Z. <fra...@yo...>]

Hi Tim and Leo,

An unfashionably late thanks for the reply to my problem.

In fact, I managed to solve it along the lines of what Tim suggested; 
that was deleting

   zeves/src/test/resources/tests/chain.tex

It compiles fine now, something might have become inconsistent in my 
local copy.

All the best
Frank

On 05/01/12 09:38, Leo Freitas wrote:
> Hi Frank,
>
> Hum... that shouldn't be happening, mostly because we (at Newcastle) haven't touched on the main trunk (after build 8139)
> for a couple of months now (E.g. we are developing on a branch, which by the way is to become the main trunk soon).
>
> One possible (yet not ideal) solution would be for you to use the TSM branch or to delete the chain.tex from your test build,
> until we figure out where is it coming from.
>
> I don't get this on my builds, but then that's because I am on the branch (tsm = for transactional section manager).
> I have on the spare machine the actual trunk and I will try and do a run through to see.
>
> Could you tell me what revision are you on and if with the tsm branch or the main trunk?
>
> Tim, thanks for the suggestion as well :-)...
>
> PS:
> could you please file it as a bug report on sourceforge?
>
> Best,
> Leo
>
> On 1 May 2012, at 06:56, Tim Miller wrote:
>
>> Hi Frank,
>>
>> I do not get this with a fresh update and install. Note however that it
>> is a test that is failing. You can build CZT without running the tests
>> using:
>>
>>      mvn -DskipTests install
>>
>> The error message you provided looks like a cascading failure from one
>> file (trunk/zeves/target/test-classes/tests/chain.tex). Are you sure
>> that file is a fresh one? Perhaps try deleting the local copy and doing
>> another svn update. Other than that, I can be of little use!
>>
>> Cheers,
>> Tim
>>
>> On 01/05/12 03:38, Frank Zeyda wrote:
>>> Dear CZT'lers,
>>>
>>> I am getting the following error after just updating from the repository
>>> and doing a fresh install.
>>>
>>> -------------------------------------------------------
>>>    T E S T S
>>> -------------------------------------------------------
>>> Running net.sourceforge.czt.zeves.CZT2ZEvesPrintingTest
>>> Exception thrown during testing of
>>> Unexpected exception
>>> 	File     :
>>> file:/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex
>>> 	Exception: net.sourceforge.czt.session.CommandException:
>>> net.sourceforge.czt.parser.util.ParseException:
>>> line 2165 column 0 in
>>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>>> Unknown latex command \end
>>> line 2165 column 5 in
>>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>>> Syntax error at symbol sche10dir8083https
>>> line 2211 column 46 in
>>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>>> Syntax error at symbol COMMA
>>> line 2234 column 116 in
>>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>>> Expression expected; found predicate
>>> line 2238 column 302 in
>>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>>> Expression expected; found predicate
>>> 	Cause    : net.sourceforge.czt.parser.util.ParseException:
>>> line 2165 column 0 in
>>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>>> Unknown latex command \end
>>> line 2165 column 5 in
>>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>>> Syntax error at symbol sche10dir8083https
>>> line 2211 column 46 in
>>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>>> Syntax error at symbol COMMA
>>> line 2234 column 116 in
>>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>>> Expression expected; found predicate
>>> line 2238 column 302 in
>>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>>> Expression expected; found predicate -
>>> net.sourceforge.czt.session.CommandException:
>>> net.sourceforge.czt.parser.util.ParseException:
>>> line 2165 column 0 in
>>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>>> Unknown latex command \end
>>> line 2165 column 5 in
>>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>>> Syntax error at symbol sche10dir8083https
>>> line 2211 column 46 in
>>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>>> Syntax error at symbol COMMA
>>> line 2234 column 116 in
>>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>>> Expression expected; found predicate
>>> line 2238 column 302 in
>>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>>> Expression expected; found predicate
>>> net.sourceforge.czt.session.CommandException:
>>> net.sourceforge.czt.parser.util.ParseException:
>>> line 2165 column 0 in
>>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>>> Unknown latex command \end
>>> line 2165 column 5 in
>>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>>> Syntax error at symbol sche10dir8083https
>>> line 2211 column 46 in
>>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>>> Syntax error at symbol COMMA
>>> line 2234 column 116 in
>>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>>> Expression expected; found predicate
>>> line 2238 column 302 in
>>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>>> Expression expected; found predicate
>>> 	at
>>> net.sourceforge.czt.parser.zeves.ParseUtils.doCompute(ParseUtils.java:536)
>>> 	at
>>> net.sourceforge.czt.session.AbstractCommand.compute(AbstractCommand.java:52)
>>> 	at net.sourceforge.czt.session.SectionManager.get(SectionManager.java:877)
>>> 	at
>>> net.sourceforge.czt.parser.util.CztManagedTest.parse(CztManagedTest.java:208)
>>> 	at
>>> net.sourceforge.czt.parser.util.CztManagedTest.parse(CztManagedTest.java:199)
>>> 	at
>>> net.sourceforge.czt.parser.util.CztManagedTest$AbstractManagedTest.runTest(CztManagedTest.java:394)
>>> 	at junit.framework.TestCase.runBare(TestCase.java:134)
>>> 	at junit.framework.TestResult$1.protect(TestResult.java:110)
>>> 	at junit.framework.TestResult.runProtected(TestResult.java:128)
>>> 	at junit.framework.TestResult.run(TestResult.java:113)
>>> 	at junit.framework.TestCase.run(TestCase.java:124)
>>> 	at junit.framework.TestSuite.runTest(TestSuite.java:232)
>>> 	at junit.framework.TestSuite.run(TestSuite.java:227)
>>> 	at
>>> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:79)
>>> 	at
>>> org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:59)
>>> 	at
>>> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:120)
>>> 	at
>>> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:103)
>>> 	at org.apache.maven.surefire.Surefire.run(Surefire.java:169)
>>> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> 	at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>> 	at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> 	at java.lang.reflect.Method.invoke(Method.java:601)
>>> 	at
>>> org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:350)
>>> 	at
>>> org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1021)
>>> Caused by: net.sourceforge.czt.parser.util.ParseException:
>>> line 2165 column 0 in
>>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>>> Unknown latex command \end
>>> line 2165 column 5 in
>>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>>> Syntax error at symbol sche10dir8083https
>>> line 2211 column 46 in
>>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>>> Syntax error at symbol COMMA
>>> line 2234 column 116 in
>>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>>> Expression expected; found predicate
>>> line 2238 column 302 in
>>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>>> Expression expected; found predicate
>>> 	at
>>> net.sourceforge.czt.parser.util.ParseExceptionCommand.compute(ParseExceptionCommand.java:36)
>>> 	at net.sourceforge.czt.session.SectionManager.get(SectionManager.java:877)
>>> 	at net.sourceforge.czt.parser.zeves.Parser.<init>(Parser.java:3100)
>>> 	at net.sourceforge.czt.parser.zeves.LatexParser.<init>(LatexParser.java:55)
>>> 	at net.sourceforge.czt.parser.zeves.ParseUtils.parse(ParseUtils.java:60)
>>> 	at
>>> net.sourceforge.czt.parser.zeves.ParseUtils.doCompute(ParseUtils.java:524)
>>> 	... 23 more
>>> Tests run: 36, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 25.103
>>> sec<<<   FAILURE!
>>>
>>> Results :
>>>
>>> Failed tests:
>>>     null(net.sourceforge.czt.parser.util.CztManagedTest$TestNormal)
>>>
>>> Tests run: 36, Failures: 1, Errors: 0, Skipped: 0
>>>
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [ERROR] BUILD FAILURE
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] There are test failures.
>>>
>>> Can anyone reproduce this problem?
>>>
>>> Cheers
>>> Frank
>>>
>>
>>
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> CZT-Devel mailing list
>> CZT...@li...
>> https://lists.sourceforge.net/lists/listinfo/czt-devel
>

Re: [CZT-Devel] Error rebuilding CZT.

[Leo F. <leo...@ne...>]

Hi Frank,

Hum... that shouldn't be happening, mostly because we (at Newcastle) haven't touched on the main trunk (after build 8139)
for a couple of months now (E.g. we are developing on a branch, which by the way is to become the main trunk soon).

One possible (yet not ideal) solution would be for you to use the TSM branch or to delete the chain.tex from your test build,
until we figure out where is it coming from. 

I don't get this on my builds, but then that's because I am on the branch (tsm = for transactional section manager).
I have on the spare machine the actual trunk and I will try and do a run through to see.

Could you tell me what revision are you on and if with the tsm branch or the main trunk?

Tim, thanks for the suggestion as well :-)... 

PS:
could you please file it as a bug report on sourceforge?

Best,
Leo

On 1 May 2012, at 06:56, Tim Miller wrote:

> Hi Frank,
> 
> I do not get this with a fresh update and install. Note however that it 
> is a test that is failing. You can build CZT without running the tests 
> using:
> 
>     mvn -DskipTests install
> 
> The error message you provided looks like a cascading failure from one 
> file (trunk/zeves/target/test-classes/tests/chain.tex). Are you sure 
> that file is a fresh one? Perhaps try deleting the local copy and doing 
> another svn update. Other than that, I can be of little use!
> 
> Cheers,
> Tim
> 
> On 01/05/12 03:38, Frank Zeyda wrote:
>> Dear CZT'lers,
>> 
>> I am getting the following error after just updating from the repository
>> and doing a fresh install.
>> 
>> -------------------------------------------------------
>>   T E S T S
>> -------------------------------------------------------
>> Running net.sourceforge.czt.zeves.CZT2ZEvesPrintingTest
>> Exception thrown during testing of
>> Unexpected exception
>> 	File     :
>> file:/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex
>> 	Exception: net.sourceforge.czt.session.CommandException:
>> net.sourceforge.czt.parser.util.ParseException:
>> line 2165 column 0 in
>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>> Unknown latex command \end
>> line 2165 column 5 in
>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>> Syntax error at symbol sche10dir8083https
>> line 2211 column 46 in
>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>> Syntax error at symbol COMMA
>> line 2234 column 116 in
>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>> Expression expected; found predicate
>> line 2238 column 302 in
>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>> Expression expected; found predicate
>> 	Cause    : net.sourceforge.czt.parser.util.ParseException:
>> line 2165 column 0 in
>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>> Unknown latex command \end
>> line 2165 column 5 in
>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>> Syntax error at symbol sche10dir8083https
>> line 2211 column 46 in
>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>> Syntax error at symbol COMMA
>> line 2234 column 116 in
>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>> Expression expected; found predicate
>> line 2238 column 302 in
>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>> Expression expected; found predicate -
>> net.sourceforge.czt.session.CommandException:
>> net.sourceforge.czt.parser.util.ParseException:
>> line 2165 column 0 in
>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>> Unknown latex command \end
>> line 2165 column 5 in
>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>> Syntax error at symbol sche10dir8083https
>> line 2211 column 46 in
>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>> Syntax error at symbol COMMA
>> line 2234 column 116 in
>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>> Expression expected; found predicate
>> line 2238 column 302 in
>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>> Expression expected; found predicate
>> net.sourceforge.czt.session.CommandException:
>> net.sourceforge.czt.parser.util.ParseException:
>> line 2165 column 0 in
>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>> Unknown latex command \end
>> line 2165 column 5 in
>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>> Syntax error at symbol sche10dir8083https
>> line 2211 column 46 in
>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>> Syntax error at symbol COMMA
>> line 2234 column 116 in
>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>> Expression expected; found predicate
>> line 2238 column 302 in
>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>> Expression expected; found predicate
>> 	at
>> net.sourceforge.czt.parser.zeves.ParseUtils.doCompute(ParseUtils.java:536)
>> 	at
>> net.sourceforge.czt.session.AbstractCommand.compute(AbstractCommand.java:52)
>> 	at net.sourceforge.czt.session.SectionManager.get(SectionManager.java:877)
>> 	at
>> net.sourceforge.czt.parser.util.CztManagedTest.parse(CztManagedTest.java:208)
>> 	at
>> net.sourceforge.czt.parser.util.CztManagedTest.parse(CztManagedTest.java:199)
>> 	at
>> net.sourceforge.czt.parser.util.CztManagedTest$AbstractManagedTest.runTest(CztManagedTest.java:394)
>> 	at junit.framework.TestCase.runBare(TestCase.java:134)
>> 	at junit.framework.TestResult$1.protect(TestResult.java:110)
>> 	at junit.framework.TestResult.runProtected(TestResult.java:128)
>> 	at junit.framework.TestResult.run(TestResult.java:113)
>> 	at junit.framework.TestCase.run(TestCase.java:124)
>> 	at junit.framework.TestSuite.runTest(TestSuite.java:232)
>> 	at junit.framework.TestSuite.run(TestSuite.java:227)
>> 	at
>> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:79)
>> 	at
>> org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:59)
>> 	at
>> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:120)
>> 	at
>> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:103)
>> 	at org.apache.maven.surefire.Surefire.run(Surefire.java:169)
>> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> 	at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> 	at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> 	at java.lang.reflect.Method.invoke(Method.java:601)
>> 	at
>> org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:350)
>> 	at
>> org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1021)
>> Caused by: net.sourceforge.czt.parser.util.ParseException:
>> line 2165 column 0 in
>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>> Unknown latex command \end
>> line 2165 column 5 in
>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>> Syntax error at symbol sche10dir8083https
>> line 2211 column 46 in
>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>> Syntax error at symbol COMMA
>> line 2234 column 116 in
>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>> Expression expected; found predicate
>> line 2238 column 302 in
>> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
>> Expression expected; found predicate
>> 	at
>> net.sourceforge.czt.parser.util.ParseExceptionCommand.compute(ParseExceptionCommand.java:36)
>> 	at net.sourceforge.czt.session.SectionManager.get(SectionManager.java:877)
>> 	at net.sourceforge.czt.parser.zeves.Parser.<init>(Parser.java:3100)
>> 	at net.sourceforge.czt.parser.zeves.LatexParser.<init>(LatexParser.java:55)
>> 	at net.sourceforge.czt.parser.zeves.ParseUtils.parse(ParseUtils.java:60)
>> 	at
>> net.sourceforge.czt.parser.zeves.ParseUtils.doCompute(ParseUtils.java:524)
>> 	... 23 more
>> Tests run: 36, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 25.103
>> sec<<<  FAILURE!
>> 
>> Results :
>> 
>> Failed tests:
>>    null(net.sourceforge.czt.parser.util.CztManagedTest$TestNormal)
>> 
>> Tests run: 36, Failures: 1, Errors: 0, Skipped: 0
>> 
>> [INFO]
>> ------------------------------------------------------------------------
>> [ERROR] BUILD FAILURE
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] There are test failures.
>> 
>> Can anyone reproduce this problem?
>> 
>> Cheers
>> Frank
>> 
> 
> 
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and 
> threat landscape has changed and how IT managers can respond. Discussions 
> will include endpoint security, mobile security and the latest in malware 
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> CZT-Devel mailing list
> CZT...@li...
> https://lists.sourceforge.net/lists/listinfo/czt-devel

Re: [CZT-Devel] Error rebuilding CZT.

[Tim M. <tm...@un...>]

Hi Frank,

I do not get this with a fresh update and install. Note however that it 
is a test that is failing. You can build CZT without running the tests 
using:

     mvn -DskipTests install

The error message you provided looks like a cascading failure from one 
file (trunk/zeves/target/test-classes/tests/chain.tex). Are you sure 
that file is a fresh one? Perhaps try deleting the local copy and doing 
another svn update. Other than that, I can be of little use!

Cheers,
Tim

On 01/05/12 03:38, Frank Zeyda wrote:
> Dear CZT'lers,
>
> I am getting the following error after just updating from the repository
> and doing a fresh install.
>
> -------------------------------------------------------
>    T E S T S
> -------------------------------------------------------
> Running net.sourceforge.czt.zeves.CZT2ZEvesPrintingTest
> Exception thrown during testing of
> Unexpected exception
> 	File     :
> file:/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex
> 	Exception: net.sourceforge.czt.session.CommandException:
> net.sourceforge.czt.parser.util.ParseException:
> line 2165 column 0 in
> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
> Unknown latex command \end
> line 2165 column 5 in
> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
> Syntax error at symbol sche10dir8083https
> line 2211 column 46 in
> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
> Syntax error at symbol COMMA
> line 2234 column 116 in
> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
> Expression expected; found predicate
> line 2238 column 302 in
> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
> Expression expected; found predicate
> 	Cause    : net.sourceforge.czt.parser.util.ParseException:
> line 2165 column 0 in
> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
> Unknown latex command \end
> line 2165 column 5 in
> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
> Syntax error at symbol sche10dir8083https
> line 2211 column 46 in
> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
> Syntax error at symbol COMMA
> line 2234 column 116 in
> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
> Expression expected; found predicate
> line 2238 column 302 in
> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
> Expression expected; found predicate -
> net.sourceforge.czt.session.CommandException:
> net.sourceforge.czt.parser.util.ParseException:
> line 2165 column 0 in
> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
> Unknown latex command \end
> line 2165 column 5 in
> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
> Syntax error at symbol sche10dir8083https
> line 2211 column 46 in
> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
> Syntax error at symbol COMMA
> line 2234 column 116 in
> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
> Expression expected; found predicate
> line 2238 column 302 in
> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
> Expression expected; found predicate
> net.sourceforge.czt.session.CommandException:
> net.sourceforge.czt.parser.util.ParseException:
> line 2165 column 0 in
> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
> Unknown latex command \end
> line 2165 column 5 in
> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
> Syntax error at symbol sche10dir8083https
> line 2211 column 46 in
> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
> Syntax error at symbol COMMA
> line 2234 column 116 in
> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
> Expression expected; found predicate
> line 2238 column 302 in
> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
> Expression expected; found predicate
> 	at
> net.sourceforge.czt.parser.zeves.ParseUtils.doCompute(ParseUtils.java:536)
> 	at
> net.sourceforge.czt.session.AbstractCommand.compute(AbstractCommand.java:52)
> 	at net.sourceforge.czt.session.SectionManager.get(SectionManager.java:877)
> 	at
> net.sourceforge.czt.parser.util.CztManagedTest.parse(CztManagedTest.java:208)
> 	at
> net.sourceforge.czt.parser.util.CztManagedTest.parse(CztManagedTest.java:199)
> 	at
> net.sourceforge.czt.parser.util.CztManagedTest$AbstractManagedTest.runTest(CztManagedTest.java:394)
> 	at junit.framework.TestCase.runBare(TestCase.java:134)
> 	at junit.framework.TestResult$1.protect(TestResult.java:110)
> 	at junit.framework.TestResult.runProtected(TestResult.java:128)
> 	at junit.framework.TestResult.run(TestResult.java:113)
> 	at junit.framework.TestCase.run(TestCase.java:124)
> 	at junit.framework.TestSuite.runTest(TestSuite.java:232)
> 	at junit.framework.TestSuite.run(TestSuite.java:227)
> 	at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:79)
> 	at
> org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:59)
> 	at
> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:120)
> 	at
> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:103)
> 	at org.apache.maven.surefire.Surefire.run(Surefire.java:169)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:601)
> 	at
> org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:350)
> 	at
> org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1021)
> Caused by: net.sourceforge.czt.parser.util.ParseException:
> line 2165 column 0 in
> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
> Unknown latex command \end
> line 2165 column 5 in
> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
> Syntax error at symbol sche10dir8083https
> line 2211 column 46 in
> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
> Syntax error at symbol COMMA
> line 2234 column 116 in
> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
> Expression expected; found predicate
> line 2238 column 302 in
> "/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex":
> Expression expected; found predicate
> 	at
> net.sourceforge.czt.parser.util.ParseExceptionCommand.compute(ParseExceptionCommand.java:36)
> 	at net.sourceforge.czt.session.SectionManager.get(SectionManager.java:877)
> 	at net.sourceforge.czt.parser.zeves.Parser.<init>(Parser.java:3100)
> 	at net.sourceforge.czt.parser.zeves.LatexParser.<init>(LatexParser.java:55)
> 	at net.sourceforge.czt.parser.zeves.ParseUtils.parse(ParseUtils.java:60)
> 	at
> net.sourceforge.czt.parser.zeves.ParseUtils.doCompute(ParseUtils.java:524)
> 	... 23 more
> Tests run: 36, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 25.103
> sec<<<  FAILURE!
>
> Results :
>
> Failed tests:
>     null(net.sourceforge.czt.parser.util.CztManagedTest$TestNormal)
>
> Tests run: 36, Failures: 1, Errors: 0, Skipped: 0
>
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] BUILD FAILURE
> [INFO]
> ------------------------------------------------------------------------
> [INFO] There are test failures.
>
> Can anyone reproduce this problem?
>
> Cheers
> Frank
>

[CZT-Devel] Error rebuilding CZT.

[Frank Z. <fra...@yo...>]

Dear CZT'lers,

I am getting the following error after just updating from the repository 
and doing a fresh install.

-------------------------------------------------------
  T E S T S
-------------------------------------------------------
Running net.sourceforge.czt.zeves.CZT2ZEvesPrintingTest
Exception thrown during testing of
Unexpected exception
	File     : 
file:/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex
	Exception: net.sourceforge.czt.session.CommandException: 
net.sourceforge.czt.parser.util.ParseException:
line 2165 column 0 in 
"/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex": 
Unknown latex command \end
line 2165 column 5 in 
"/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex": 
Syntax error at symbol sche10dir8083https
line 2211 column 46 in 
"/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex": 
Syntax error at symbol COMMA
line 2234 column 116 in 
"/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex": 
Expression expected; found predicate
line 2238 column 302 in 
"/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex": 
Expression expected; found predicate
	Cause    : net.sourceforge.czt.parser.util.ParseException:
line 2165 column 0 in 
"/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex": 
Unknown latex command \end
line 2165 column 5 in 
"/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex": 
Syntax error at symbol sche10dir8083https
line 2211 column 46 in 
"/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex": 
Syntax error at symbol COMMA
line 2234 column 116 in 
"/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex": 
Expression expected; found predicate
line 2238 column 302 in 
"/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex": 
Expression expected; found predicate - 
net.sourceforge.czt.session.CommandException: 
net.sourceforge.czt.parser.util.ParseException:
line 2165 column 0 in 
"/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex": 
Unknown latex command \end
line 2165 column 5 in 
"/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex": 
Syntax error at symbol sche10dir8083https
line 2211 column 46 in 
"/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex": 
Syntax error at symbol COMMA
line 2234 column 116 in 
"/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex": 
Expression expected; found predicate
line 2238 column 302 in 
"/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex": 
Expression expected; found predicate
net.sourceforge.czt.session.CommandException: 
net.sourceforge.czt.parser.util.ParseException:
line 2165 column 0 in 
"/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex": 
Unknown latex command \end
line 2165 column 5 in 
"/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex": 
Syntax error at symbol sche10dir8083https
line 2211 column 46 in 
"/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex": 
Syntax error at symbol COMMA
line 2234 column 116 in 
"/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex": 
Expression expected; found predicate
line 2238 column 302 in 
"/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex": 
Expression expected; found predicate
	at 
net.sourceforge.czt.parser.zeves.ParseUtils.doCompute(ParseUtils.java:536)
	at 
net.sourceforge.czt.session.AbstractCommand.compute(AbstractCommand.java:52)
	at net.sourceforge.czt.session.SectionManager.get(SectionManager.java:877)
	at 
net.sourceforge.czt.parser.util.CztManagedTest.parse(CztManagedTest.java:208)
	at 
net.sourceforge.czt.parser.util.CztManagedTest.parse(CztManagedTest.java:199)
	at 
net.sourceforge.czt.parser.util.CztManagedTest$AbstractManagedTest.runTest(CztManagedTest.java:394)
	at junit.framework.TestCase.runBare(TestCase.java:134)
	at junit.framework.TestResult$1.protect(TestResult.java:110)
	at junit.framework.TestResult.runProtected(TestResult.java:128)
	at junit.framework.TestResult.run(TestResult.java:113)
	at junit.framework.TestCase.run(TestCase.java:124)
	at junit.framework.TestSuite.runTest(TestSuite.java:232)
	at junit.framework.TestSuite.run(TestSuite.java:227)
	at 
org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:79)
	at 
org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:59)
	at 
org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:120)
	at 
org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:103)
	at org.apache.maven.surefire.Surefire.run(Surefire.java:169)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at 
org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:350)
	at 
org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1021)
Caused by: net.sourceforge.czt.parser.util.ParseException:
line 2165 column 0 in 
"/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex": 
Unknown latex command \end
line 2165 column 5 in 
"/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex": 
Syntax error at symbol sche10dir8083https
line 2211 column 46 in 
"/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex": 
Syntax error at symbol COMMA
line 2234 column 116 in 
"/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex": 
Expression expected; found predicate
line 2238 column 302 in 
"/local/d0p6/zeyda/repo/czt/trunk/zeves/target/test-classes/tests/chain.tex": 
Expression expected; found predicate
	at 
net.sourceforge.czt.parser.util.ParseExceptionCommand.compute(ParseExceptionCommand.java:36)
	at net.sourceforge.czt.session.SectionManager.get(SectionManager.java:877)
	at net.sourceforge.czt.parser.zeves.Parser.<init>(Parser.java:3100)
	at net.sourceforge.czt.parser.zeves.LatexParser.<init>(LatexParser.java:55)
	at net.sourceforge.czt.parser.zeves.ParseUtils.parse(ParseUtils.java:60)
	at 
net.sourceforge.czt.parser.zeves.ParseUtils.doCompute(ParseUtils.java:524)
	... 23 more
Tests run: 36, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 25.103 
sec <<< FAILURE!

Results :

Failed tests:
   null(net.sourceforge.czt.parser.util.CztManagedTest$TestNormal)

Tests run: 36, Failures: 1, Errors: 0, Skipped: 0

[INFO] 
------------------------------------------------------------------------
[ERROR] BUILD FAILURE
[INFO] 
------------------------------------------------------------------------
[INFO] There are test failures.

Can anyone reproduce this problem?

Cheers
Frank

-- 
  Frank Zeyda, BSc Dipl.-Inform. PhD
  Research Associate
  High Integrity Systems Engineering Group
  Department of Computer Science
  University of York (UK)

  Email: ze...@cs...
  Phone: +44-(0)1904-325434
  WWW: http://www-users.cs.york.ac.uk/~zeyda/

  EMAIL DISCLAIMER: http://www.york.ac.uk/docs/disclaimer/email.htm

Re: [CZT-Devel] some admin issues / decisions

[Tim M. <tm...@un...>]

Hi Leo,

> a) performance issues = ZName?
>
> comments / thoughts please?
>
I think we need to look into this. CPU performance in CZT is ok, but 
memory is horrible. I'm happy to do some experimentation with the 
typechecker, as you and I discussed via email, but I won't get to it 
just yet.

Longer term, we need to think about our use of IDs, which creates a lot 
of new ZName instances.

> =======================
> b) release = do one now?
>
> It's been a while without a release. I thought to do another one. I haven't done this before in maven / source forge.
> any hints / tips / do's / donut's you know of?
>
Yes, a new release would be great. Petra has always taken care of these, 
so I can offer no useful suggestions.

> =======================
> c) version control choice(s) = add mercurial?
>
> a user talked to me about using Mercurial alongside SVN in order to be able to have (experimental) extensions under version control,
> without the hassle of branching/merging, where the source forge copy would be the master rep still…
>
> any thoughts / suggestions / things-against this?  If none, I was tempted to try and use mercurial ….
>
Alongside SVN? I wasn't aware such a thing was possible. I've no problem 
with having distributed version control -- it certainly makes it easier 
to do branching (I create my own local branches in CZT due to the 
overhead of creating a new branch in SVN).

> =======================
> d) licensing issue = GPL x EPL
>
> It came to my attention that CZT main license (GPL) has some conflicts with Eclipse (EPL) licence
>
> http://mmilinkov.wordpress.com/2010/04/06/epl-gpl-commentary/
>
> Given our Eclipse integration, I thought if there could be any issues with such conflict(s)?
>
Without giving it too much thought, I wouldn't worry about it. There 
must be a number of GPL'd Eclipse plug ins, and our work is non-profit, 
so nobody is going to chase us down. As long as people abide by the 
spirit, I don't think the Eclipse folks will care.

Cheers,
Tim

[CZT-Devel] some admin issues / decisions

[Leo F. <leo...@ne...>]

Hi guys,

I would like to ask you about a few admin issues/decisions… I will keep it in some order of importance. please comment :-)

a) performance issues 		= ZName?
b) release 					= do one now?
c) version control choice(s)	= add mercurial?
d) licensing issue				= GPL x EPL

=======================
a) performance issues = ZName?

Some users were asking about problems when loading large specs in CZT leading to memory blowouts.
After some profiling and a few small changes we got great improvements. 
Some problems still remain, mostly with ZName and its use of Ids. 

This brings back the use of Flyweight patterns for some constructs (e.g., AST sharing), probably ZName most of all should do it.
I am looking into that together with Tim to see whether we will have any real gain or not…. We will see….

Suggestions on how to make the Flyweight for names (considering Ids) are welcome. For instance, 

\begin{zsection}
  \SECTION Test 
\end{zsection}
\begin{zed}
  variable == 1
\end{zed}

has 130 ZName instances created with 66 remaining live in memory (i.e., we could try avoiding duplication of ZNames).
these names come mostly from prelude. Of these 66 names, we have 60 unique ids but just 14 unique name "strings"/word
(e.g., notice that "_+_", "+" are two different ZNames; and that gets multiplied by the number of ids each get).

With StdToolkit, these numbers are 3252 create; 2267 live; 1488 unique ids; 169 unique name strings….
So optimisation of names looks promising….

comments / thoughts please?

=======================
b) release = do one now?

It's been a while without a release. I thought to do another one. I haven't done this before in maven / source forge.
any hints / tips / do's / donut's you know of?

=======================
c) version control choice(s) = add mercurial?

a user talked to me about using Mercurial alongside SVN in order to be able to have (experimental) extensions under version control,
without the hassle of branching/merging, where the source forge copy would be the master rep still…

any thoughts / suggestions / things-against this?  If none, I was tempted to try and use mercurial ….

=======================
d) licensing issue = GPL x EPL

It came to my attention that CZT main license (GPL) has some conflicts with Eclipse (EPL) licence 

http://mmilinkov.wordpress.com/2010/04/06/epl-gpl-commentary/

Given our Eclipse integration, I thought if there could be any issues with such conflict(s)?

=========

Thanks.
Leo

[CZT-Devel] section manager dependencies (round 2)

[Leo F. <leo...@ne...>]

Hi guys,

We've almost finished with the dependencies stuff on the section manager. Now, on the branch, all tests pass
as well as all top-level tools. We are now updating the section manager usage within Eclipse; in particular when
failures happen (E.g., transaction cancellation).

One interesting thing we notice is that in various top-level tools, the section manager is not quite rightly used
(e.g., TypeCheckUtils.run does use the command typeCheck / parse directly rather than the SM.get()). That's
okay, but it won't reflect the top-level spec deps, obviously.

On quick run all dependencies / dependants looked correct. More testing is needed to weed out the odd cases
(e.g., duplicated sections per specification on one file; duplicated sections on source of different markup; section
names on files with different name; etc).

Soon we will merge the branch tsm into the main trunk as well….

Best,
Leo

[CZT-Devel] new section manager - comments?

[Leo F. <leo...@ne...>]

Hi guys,

Andrius and I have been working this week on the manager and we just created a branch. Soon we will merge changes 
to the main trunk. Here is an update on what's been happening :-)

---- 1) in general 

We are nearly there in the implementation of the new section manager: its main difference is that is calculates 
dependencies correctly keeping track of both dependants (downward) and dependencies (upward). 
Thanks for the comments - and Petra, thanks for the link to the cztsecman paper: it kept us in check on the facilities envisaged.

We are not finished yet, as some tests still need updating. And there is a small problem with LatexMarkupFunction: the most
difficult one of all to keep track of dependencies within the transaction. Parser tests pass, except for ZEves ones, as we have 
some unconventional cases (e.g., files with multiple sections or sections not in synch with the file), that need updating.

We branched the main trunk into 

http://czt.svn.sourceforge.net/viewvc/czt/branches/tsm/

Please have a look at SectionManager.java in session and in the commands giving your comments…
This is changing and not ready yet.

---- 1.1) using it

For the user,  we tried to keep the protocol just as before with get/puts. So, most of the code around is the same.
Command implementations must change, however. Most places of use do not need to change, unless they had
misused the section manager (e.g., taken advantage of duplicate keys; or used command exceptions to get keys; etc).
When keys are removed, all dependants/dependencies are removed as well.

For IDEs, we envisage having a section manager per project, where all files involved within a project are handled
by a single manager. This improves performance by reducing rework, and with the dependencies tracking keeps 
all in check for soundness. We are also going to use time stamps for consistency.

---- 1.3) the future - concurrency

The overall plan is to have concurrent section management (!!!) The plan is once we understand the dependencies graph well,
we would be able to predict what could be split into different threads so that we could speed up some processes, in particular
parsing/lexing, which takes most time. VC generation can be quite expensive as well, and would benefit from this.

This is just an idea for now, but the hope is to factor / extend the manager to allow concurrency as an option.

Best,
Leo & Andrius 

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
---- 2) in detail

We plan to document this in a small document, as well as in various comments along the implementation.
We will also have a bunch of examples of both usage and implementation.

---- 2.1) the dependencies

This is the tricky bit, and we are documenting each command/key and its dependencies soon. It will be part of
the documentation of each command - or perhaps even within the section manager itself.

Overall the protocol/implementation works well, is simple (enough), and conservative/strict. For instance, we do not allow:
	a) key duplication (e.g., two transactions trying to calculate the same key?); 
	b) key removal of ongoing transaction (e.g., interference? - removing a key removes dependants/dependencies);
	c) new transaction on cached keys (e.g., would lead to key duplication); 
	d) and there are other cases...

The only tricky case is LatexMarkupFunction (LMF), and somewhat the OpTable. 

Like in kernels, the main trouble lies within bootstrapping, and that is parsing. When parsing ZSect, its OpTable gets
created and triggers parsing of parent tables, which is fine. Trouble with LMF is that it is lexing, so during lexing of a 
section, we might start parsing of its parent. And when that involves Spec or ZSect with cycles, things get messy. 
Another messy case is the one with multiple sections in a file or even repeated/redeclared sections….. we are getting there.

The main trouble with redeclared sections is that it is caught as a type error (!!!) despite the fact it manifest itself as a lexing problem!
And since we are no longer allowing for an inconsistent section manager (e.g., key duplication), sorting this one out is getting hard. 

---- 2.2) the protocol

get(Key)
	startTransaction(key)
	try
	{
		compute(key)
	}
	catch (CmdExpt e)
	{
		cancel(key)
	}

put(Key, Value)
	startTransaction(key)
	endTransaction(key, value)

Command.compute
	if okay
		endTransaction(key, value)
	else
		cancelTransaction(key) or
		throw new CommandException().

Just like in some of the parser visitors, we compute implicit dependencies within get(Key).
We also allow explicit dependencies (e.g., those not existing dependencies that do not appear
in the get protocol; or top-level dependencies needed like Spec/ZSect; etc). 

The idea here is that more dependencies will make the manager more sensitive to change, which is okay.
Later we will fine tune those to see the best balance. We are keeping it very conservative in this calc.

There are various consistency checks to ensure the soundness of transactions and dependencies. 

--- 2.3) the implementation

 We added transactions to the manager in the follow way:

a) we keep a stack of transactions by (key, index) pair
b) we keep a list of dependency keys

The transaction stack contains the user given key plus a pointer to the dependency keys list. 
That key will depend on all elements in b) from the index to the size. The dependencies list
grows at get(Key) and shrinks at endTransaction when the last transaction finishes (e.g., it
contains the transitive dependencies of a key).

Upon endTransaction we store the mapping from key->value within the manager and calculate
the dependency maps in both directions. There are two methods getDependants(key) and getDependencies(key).

When removing a key (e.g., on reset()), we remove all dependants and dependencies. We ensure you
cannot remove dependants for a key of an ongoing transaction. On reset/cloning  we require that there are no ongoing transactions. 

A small addition is the method on keys to consider permanent (e.g., those that don't get removed on reset).
Originally, it was on all those involving prelude or toolkit in names. It remains this way, but we give the user the chance to
register if there are any more.

For debugging, there is a detailed tracing mechanism that dumps how the transactions and dependencies progress. 

-------- 

Phew…. please give us comments / suggestions / opinions…. 

as soon as we clear all the test cases, we will merge these changes to the main trunk.

Best,
Leo & Andrius

Re: [CZT-Devel] section manager dependencies

[Leo F. <leo...@ne...>]

Hi Mark,

Yep... I tried this in past quickly and realised it wasn't trivial. Given the need now with larger specs and memory problems,
we are having another go.... Thanks for the .html version; I checked it and the features there are more or less within the section manger's remit.
I saw it was on revision 3553, but coultn'd see from the SVN logs any file named

ToolkitSectionInfoRegistry
WrappedSectionInfoRegistry

etc.... By the way, we are also developing a concurrent section manager: one that has lock sets over keys and enables locking on write as well as read.
This shall be useful in the presence of parallel / asynchronous IDE editing of files (E.g., Eclipse does take advantange of concurrency if possible).

We come back to you on how we get on :-)

Thanks for the tips once again

Best,
Leo

On 29 Jan 2012, at 20:34, Mark Utting wrote:

Leo,

Good luck!
This is really tricky, for Z.

I remember that Petra and I had 3-4 different attempts at designing a section manager that kept track of versions of Z sections, or the dependencies between Z sections, so that you could update one section without reparsing the whole spec.  But none of the designs we tried were both sound and efficient, so we eventually put that problem in the too-hard basket, and did something that only cached the standard toolkits, which we assumed would not change within a session.  Some of this is documented in:

   trunk/session/src/main/java/net/sourceforge/czt/session/package.html

It refers to several classes that have presumably since been deleted, but should be in the history of the SVN repository.

It would have been more useful if we'd recorded the use cases of the situations that are difficult to handle!

Cheers
Mark


On 28 January 2012 02:50, Leo Freitas <leo...@ne...<mailto:leo...@ne...>> wrote:
Hi guys,

we've been working in a way to start using the section manager dependencies in order to avoid duplicated work (e.g., as is, we have section manager per buffer in Eclipse and jEdit).
obviously, the trouble is that invalidation of "dirty" sections is crucial in order to trust that the manager is in a correct state. This is necessary mostly for large specs, given that rework
gets in your way (e.g., Mondex has about 20 sections; you would get 20 section managers like a onion-layer of duplicated information - quite a waste!).

we started adding the various dependency relationships between various parts of CZT. This is experimental and we will come to it soon. At the moment, since no tool is using such
dependency information, it will make no difference. Andi n the end, if there is any issue on trusting the dependency results, one can just return to the (inefficient) section manager
 per buffer approach.

We will summarise soon what think the dependencies for each element involve are. There is also the issue of which directions of dependency should we worry about (upwards to
parents or downwards to children or both; as well as mutual dependencies like ZSect and its OpTable).

In the past, I come across a Z spec of a similar database of information management, and ideally an implementation should have such a origin.
Are you aware of this spec? I couldn't find it.... Also, if you have any thoughts or comments, please let us know ...

Best,
Leo & Andrius
------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
_______________________________________________
CZT-Devel mailing list
CZT...@li...<mailto:CZT...@li...>
https://lists.sourceforge.net/lists/listinfo/czt-devel

Re: [CZT-Devel] section manager dependencies

[Mark U. <ma...@cs...>]

Leo,

Good luck!
This is really tricky, for Z.

I remember that Petra and I had 3-4 different attempts at designing a
section manager that kept track of versions of Z sections, or the
dependencies between Z sections, so that you could update one section
without reparsing the whole spec.  But none of the designs we tried were
both sound and efficient, so we eventually put that problem in the too-hard
basket, and did something that only cached the standard toolkits, which we
assumed would not change within a session.  Some of this is documented in:

   trunk/session/src/main/java/net/sourceforge/czt/session/package.html

It refers to several classes that have presumably since been deleted, but
should be in the history of the SVN repository.

It would have been more useful if we'd recorded the use cases of the
situations that are difficult to handle!

Cheers
Mark


On 28 January 2012 02:50, Leo Freitas <leo...@ne...> wrote:

> Hi guys,
>
> we've been working in a way to start using the section manager
> dependencies in order to avoid duplicated work (e.g., as is, we have
> section manager per buffer in Eclipse and jEdit).
> obviously, the trouble is that invalidation of "dirty" sections is crucial
> in order to trust that the manager is in a correct state. This is necessary
> mostly for large specs, given that rework
> gets in your way (e.g., Mondex has about 20 sections; you would get 20
> section managers like a onion-layer of duplicated information - quite a
> waste!).
>
> we started adding the various dependency relationships between various
> parts of CZT. This is experimental and we will come to it soon. At the
> moment, since no tool is using such
> dependency information, it will make no difference. Andi n the end, if
> there is any issue on trusting the dependency results, one can just return
> to the (inefficient) section manager
>  per buffer approach.
>
> We will summarise soon what think the dependencies for each element
> involve are. There is also the issue of which directions of dependency
> should we worry about (upwards to
> parents or downwards to children or both; as well as mutual dependencies
> like ZSect and its OpTable).
>
> In the past, I come across a Z spec of a similar database of information
> management, and ideally an implementation should have such a origin.
> Are you aware of this spec? I couldn't find it.... Also, if you have any
> thoughts or comments, please let us know ...
>
> Best,
> Leo & Andrius
>
> ------------------------------------------------------------------------------
> Try before you buy = See our experts in action!
> The most comprehensive online learning library for Microsoft developers
> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
> Metro Style Apps, more. Free future releases when you subscribe now!
> http://p.sf.net/sfu/learndevnow-dev2
> _______________________________________________
> CZT-Devel mailing list
> CZT...@li...
> https://lists.sourceforge.net/lists/listinfo/czt-devel
>

[CZT-Devel] section manager dependencies

[Leo F. <leo...@ne...>]

Hi guys,

we've been working in a way to start using the section manager dependencies in order to avoid duplicated work (e.g., as is, we have section manager per buffer in Eclipse and jEdit).
obviously, the trouble is that invalidation of "dirty" sections is crucial in order to trust that the manager is in a correct state. This is necessary mostly for large specs, given that rework 
gets in your way (e.g., Mondex has about 20 sections; you would get 20 section managers like a onion-layer of duplicated information - quite a waste!).

we started adding the various dependency relationships between various parts of CZT. This is experimental and we will come to it soon. At the moment, since no tool is using such 
dependency information, it will make no difference. Andi n the end, if there is any issue on trusting the dependency results, one can just return to the (inefficient) section manager
 per buffer approach.

We will summarise soon what think the dependencies for each element involve are. There is also the issue of which directions of dependency should we worry about (upwards to 
parents or downwards to children or both; as well as mutual dependencies like ZSect and its OpTable).

In the past, I come across a Z spec of a similar database of information management, and ideally an implementation should have such a origin. 
Are you aware of this spec? I couldn't find it.... Also, if you have any thoughts or comments, please let us know ...

Best,
Leo & Andrius

Re: [CZT-Devel] cycles in section parents: parsing or typing error?

[Andrius V. <an...@ve...>]

Hello,

Thanks for the comments. We have added the support for cyclic parents
- CZT parsing no longer crashes with StackOverflowError. Code is in
SVN.

As Leo has mentioned, the problem used to occur with LatexMarkupParser
- it requires parent LatexMarkupFunctions to be built before current
section's LatexMarkupFunction can be produced. This requires parent
sections to be parsed, and with cyclic parents gets into a loop. The
same issue happens when creating InfoTables (e.g. OpTable, ThmTable,
etc.) during parsing - they require parent InfoTables to be built.

The implementation adds a cycle blocker (see
net.sourceforge.czt.parser.util.CyclicParseManager). It keeps the
"active" section stack, and an already "active" section can no longer
be visited. For example, say we are parsing a cycle A -> B -> C -> A.
After parent calls from A and B, we get a situation when C is being
parsed, with "active" sections A and B. Now when C is querying for
parents to continue, A is excluded as being active, so C is parsed as
if having no parent at all. This allows all sections to be parsed - if
C did have some dependencies on A, e.g. Latex Markup definitions,
parsing of C would fail. Note that the cyclic relationships are
reported as parse warnings. So in the above case, the user would have
some idea about why C parsing has failed - it would indicate that
there is a parent loop.

The typechecker also checks for parent cycles, and if found, reports
them as typechecker errors. There is a utility class to resolve parent
relationships for parsed specifications -
net.sourceforge.czt.parser.util.SectParentResolver.

We have also added several test cases of cyclic parents, which are run
for all parsers and typecheckers.

Best regards,
Andrius Velykis

On Wed, Jan 18, 2012 at 17:19, Leo Freitas <leo...@ne...> wrote:
> Hi guys,
>
> Many thanks for this. Our implementation is already in line with this, so will commit soon.
>
> One technicality though: because LatexMarkupFunction is calculated earlier (e.g., as a lexer),
> and teh information tables (e.g., optable, joker, thm, proofs, definition, etc) occur later on unicode
> stream, and we want a solution that is markup independent, the treatment in the parser is to some
> extent repeated for these two parts of the process - I don't think there is any performance penalty
> for this (e.g., the process is already quite like that anyway - LMF first; tables later).
>
> Best,
> Leo
>
>
> On 17 Jan 2012, at 21:38, Tim Miller wrote:
>
>> It definitely belongs in the typechecker. There is a note in the
>> "inheriting section" section of the Z standard that says:
>>
>> "NOTE 1 Ancestors need not be immediate parents, and a section cannot be
>> amongst its own ancestors (no cycles in the parent relation)."
>>
>> It is not part of the type rule though, so I must have missed it when I
>> implemented the type rules.
>>
>> Cheers,
>> Tim
>>
>> On 18/01/12 07:06, Mark Utting wrote:
>>> Leo
>>>
>>> Doing it in the typechecker sounds best to me.
>>>
>>> 1. You might be able to give better error messages there.
>>> 2. It will/should work on unicode input, not just latex input.
>>>
>>> Cheers
>>> Mark
>>>
>>> On 18 January 2012 02:07, Leo Freitas <leo...@ne...
>>> <mailto:leo...@ne...>> wrote:
>>>
>>>    Hi,
>>>
>>>    We are trying to solve the problem (e.g., stack overflow) involving
>>>    cyclic parents of sections.
>>>    The error occurs when calculating the latex markup function of
>>>    parents during parsing.
>>>
>>>    Our change/solution avoids the error, and enables sections with
>>>    cyclic parents to be parsed.
>>>    Next, the typechecker ensures that the cycle is an error (e.g., note
>>>    1, p. 5, 13.2.2.1).
>>>
>>>    So the question is, should the error be flagged by the parser or the
>>>    typechecker?
>>>    Looking at the standard, this we couldn't find where it would be. At
>>>    the moment,
>>>    we have it at the typechecker.
>>>
>>>    Comments?
>>>
>>>    Leo & Andrius
>>>    ------------------------------------------------------------------------------
>>>    Keep Your Developer Skills Current with LearnDevNow!
>>>    The most comprehensive online learning library for Microsoft developers
>>>    is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
>>>    Metro Style Apps, more. Free future releases when you subscribe now!
>>>    http://p.sf.net/sfu/learndevnow-d2d
>>>    _______________________________________________
>>>    CZT-Devel mailing list
>>>    CZT...@li... <mailto:CZT...@li...>
>>>    https://lists.sourceforge.net/lists/listinfo/czt-devel
>>>
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Keep Your Developer Skills Current with LearnDevNow!
>>> The most comprehensive online learning library for Microsoft developers
>>> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
>>> Metro Style Apps, more. Free future releases when you subscribe now!
>>> http://p.sf.net/sfu/learndevnow-d2d
>>>
>>>
>>>
>>> _______________________________________________
>>> CZT-Devel mailing list
>>> CZT...@li...
>>> https://lists.sourceforge.net/lists/listinfo/czt-devel
>>
>>
>> ------------------------------------------------------------------------------
>> Keep Your Developer Skills Current with LearnDevNow!
>> The most comprehensive online learning library for Microsoft developers
>> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
>> Metro Style Apps, more. Free future releases when you subscribe now!
>> http://p.sf.net/sfu/learndevnow-d2d
>> _______________________________________________
>> CZT-Devel mailing list
>> CZT...@li...
>> https://lists.sourceforge.net/lists/listinfo/czt-devel
>
>
> ------------------------------------------------------------------------------
> Keep Your Developer Skills Current with LearnDevNow!
> The most comprehensive online learning library for Microsoft developers
> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
> Metro Style Apps, more. Free future releases when you subscribe now!
> http://p.sf.net/sfu/learndevnow-d2d
> _______________________________________________
> CZT-Devel mailing list
> CZT...@li...
> https://lists.sourceforge.net/lists/listinfo/czt-devel

Re: [CZT-Devel] cycles in section parents: parsing or typing error?

[Andrius V. <and...@ne...>]

Hello,

Thanks for the comments. We have added the support for cyclic parents - CZT parsing no longer crashes with StackOverflowError. Code is in SVN.

As Leo has mentioned, the problem used to occur with LatexMarkupParser - it requires parent LatexMarkupFunctions to be built before current section's LatexMarkupFunction can be produced. This requires parent sections to be parsed, and with cyclic parents gets into a loop. The same issue happens when creating InfoTables (e.g. OpTable, ThmTable, etc.) during parsing - they require parent InfoTables to be built.

The implementation adds a cycle blocker (see net.sourceforge.czt.parser.util.CyclicParseManager). It keeps the "active" section stack, and an already "active" section can no longer be visited. For example, say we are parsing a cycle A -> B -> C -> A. After parent calls from A and B, we get a situation when C is being parsed, with "active" sections A and B. Now when C is querying for parents to continue, A is excluded as being active, so C is parsed as if having no parent at all. This allows all sections to be parsed - if C did have some dependencies on A, e.g. Latex Markup definitions, parsing of C would fail. Note that the cyclic relationships are reported as parse warnings. So in the above case, the user would have some idea about why C parsing has failed - it would indicate that there is a parent loop.

The typechecker also checks for parent cycles, and if found, reports them as typechecker errors. There is a utility class to resolve parent relationships for parsed specifications - net.sourceforge.czt.parser.util.SectParentResolver.

We have also added several test cases of cyclic parents, which are run for all parsers and typecheckers.

Best regards,
Andrius Velykis

On 18 Jan 2012, at 17:19, Leo Freitas wrote:

> Hi guys,
> 
> Many thanks for this. Our implementation is already in line with this, so will commit soon.
> 
> One technicality though: because LatexMarkupFunction is calculated earlier (e.g., as a lexer),
> and teh information tables (e.g., optable, joker, thm, proofs, definition, etc) occur later on unicode
> stream, and we want a solution that is markup independent, the treatment in the parser is to some
> extent repeated for these two parts of the process - I don't think there is any performance penalty
> for this (e.g., the process is already quite like that anyway - LMF first; tables later).
> 
> Best,
> Leo
> 
> 
> On 17 Jan 2012, at 21:38, Tim Miller wrote:
> 
>> It definitely belongs in the typechecker. There is a note in the 
>> "inheriting section" section of the Z standard that says:
>> 
>> "NOTE 1 Ancestors need not be immediate parents, and a section cannot be 
>> amongst its own ancestors (no cycles in the parent relation)."
>> 
>> It is not part of the type rule though, so I must have missed it when I 
>> implemented the type rules.
>> 
>> Cheers,
>> Tim
>> 
>> On 18/01/12 07:06, Mark Utting wrote:
>>> Leo
>>> 
>>> Doing it in the typechecker sounds best to me.
>>> 
>>> 1. You might be able to give better error messages there.
>>> 2. It will/should work on unicode input, not just latex input.
>>> 
>>> Cheers
>>> Mark
>>> 
>>> On 18 January 2012 02:07, Leo Freitas <leo...@ne...
>>> <mailto:leo...@ne...>> wrote:
>>> 
>>>   Hi,
>>> 
>>>   We are trying to solve the problem (e.g., stack overflow) involving
>>>   cyclic parents of sections.
>>>   The error occurs when calculating the latex markup function of
>>>   parents during parsing.
>>> 
>>>   Our change/solution avoids the error, and enables sections with
>>>   cyclic parents to be parsed.
>>>   Next, the typechecker ensures that the cycle is an error (e.g., note
>>>   1, p. 5, 13.2.2.1).
>>> 
>>>   So the question is, should the error be flagged by the parser or the
>>>   typechecker?
>>>   Looking at the standard, this we couldn't find where it would be. At
>>>   the moment,
>>>   we have it at the typechecker.
>>> 
>>>   Comments?
>>> 
>>>   Leo & Andrius
>>>   ------------------------------------------------------------------------------
>>>   Keep Your Developer Skills Current with LearnDevNow!
>>>   The most comprehensive online learning library for Microsoft developers
>>>   is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
>>>   Metro Style Apps, more. Free future releases when you subscribe now!
>>>   http://p.sf.net/sfu/learndevnow-d2d
>>>   _______________________________________________
>>>   CZT-Devel mailing list
>>>   CZT...@li... <mailto:CZT...@li...>
>>>   https://lists.sourceforge.net/lists/listinfo/czt-devel
>>> 
>>> 
>>> 
>>> 
>>> ------------------------------------------------------------------------------
>>> Keep Your Developer Skills Current with LearnDevNow!
>>> The most comprehensive online learning library for Microsoft developers
>>> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
>>> Metro Style Apps, more. Free future releases when you subscribe now!
>>> http://p.sf.net/sfu/learndevnow-d2d
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> CZT-Devel mailing list
>>> CZT...@li...
>>> https://lists.sourceforge.net/lists/listinfo/czt-devel
>> 
>> 
>> ------------------------------------------------------------------------------
>> Keep Your Developer Skills Current with LearnDevNow!
>> The most comprehensive online learning library for Microsoft developers
>> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
>> Metro Style Apps, more. Free future releases when you subscribe now!
>> http://p.sf.net/sfu/learndevnow-d2d
>> _______________________________________________
>> CZT-Devel mailing list
>> CZT...@li...
>> https://lists.sourceforge.net/lists/listinfo/czt-devel
> 
> 
> ------------------------------------------------------------------------------
> Keep Your Developer Skills Current with LearnDevNow!
> The most comprehensive online learning library for Microsoft developers
> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
> Metro Style Apps, more. Free future releases when you subscribe now!
> http://p.sf.net/sfu/learndevnow-d2d
> _______________________________________________
> CZT-Devel mailing list
> CZT...@li...
> https://lists.sourceforge.net/lists/listinfo/czt-devel

Re: [CZT-Devel] memory optimisation - comments please...

[Leo F. <leo...@ne...>]

Hi Tim,

The Netbeans profiler is actually quite powerful and easy to use. There is even a OO-query (SQL like) language to analyse the heap dump (!!!)

Thanks for the suggestion - you were right: I managed to reduce the ArrayList/Object[] memory consumption from 24% to 16%
(e.g., in real terms from 105MB to 80MB - some good results as well).

And the changes weren't that significant: mostly at TermImpl and GNast templates for corejava + a few spots here and there. 

I am quite happy with the results, and the usability within Eclipse is now much much better / smooth for some of the larger specs I am handling
(e.g., bare in mind I also had in memory the proof state and VCG data /info )!!! 

Thanks for suggesting this :-) 

And others, please let me know your experiences - if it improve / deteriorate  etc....

Best,
Leo

On 25 Jan 2012, at 09:00, Tim Miller wrote:

> Hi Leo,
> 
> Thanks for looking into all of this. It is some useful analysis.
> 
> I have tried to use the getAnns() method without creating new references 
> to it. I violate this only to improve readability. I don't think it is 
> causing to many problems though because the annotation lists persist in 
> memory as long as the AST exists, so references to these lists elsewhere 
> won't have much of an effect.
> 
> One possibility to help with reducing the annotation list memory 
> consumption is to modify the gnast code such that the annotation list 
> for each term is initialised to null, and then only created the first 
> time the getAnns() method is called. I'd wager that for a lot of terms, 
> it is never called.
> 
> And regarding your previous email, I definitely think a 2-3% increase in 
> CPU use is a good trade off for halving memory usage. In the parser and 
> typechecker, we can always wait a bit longer for the result, but once 
> we're out of memory, there's nowhere to go!
> 
> Cheers,
> Tim
> 
> On 25/01/12 17:47, Leo Freitas wrote:
>> Hi guys,
>> 
>> I think I found another source of some of the retained instances that seem to have weird behaviour.
>> 
>> Term has a List<Object>  getAnns() method, which exposes information like LocAnn and TypeAnn and others.
>> It's called in 1112 places across 17 CZT sub-projects: that exposes the ArrayList / Object[] for retention from those
>> that use the result or keep it in maps or the like.
>> 
>> Having said that, I couldn't find a place of misuse in the scattered references that I searched for (e.g., this might
>> be just another red herring or a key point for modification/memory-performance-improvement).....
>> 
>> thoughts / comments / suggestions ?
>> 
>> Leo
>> ------------------------------------------------------------------------------
>> Keep Your Developer Skills Current with LearnDevNow!
>> The most comprehensive online learning library for Microsoft developers
>> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
>> Metro Style Apps, more. Free future releases when you subscribe now!
>> http://p.sf.net/sfu/learndevnow-d2d
>> _______________________________________________
>> CZT-Devel mailing list
>> CZT...@li...
>> https://lists.sourceforge.net/lists/listinfo/czt-devel
>> 
> 
> 
> ------------------------------------------------------------------------------
> Keep Your Developer Skills Current with LearnDevNow!
> The most comprehensive online learning library for Microsoft developers
> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
> Metro Style Apps, more. Free future releases when you subscribe now!
> http://p.sf.net/sfu/learndevnow-d2d
> _______________________________________________
> CZT-Devel mailing list
> CZT...@li...
> https://lists.sourceforge.net/lists/listinfo/czt-devel

Re: [CZT-Devel] memory optimisation - comments please...

[Tim M. <tm...@un...>]

Hi Leo,

Thanks for looking into all of this. It is some useful analysis.

I have tried to use the getAnns() method without creating new references 
to it. I violate this only to improve readability. I don't think it is 
causing to many problems though because the annotation lists persist in 
memory as long as the AST exists, so references to these lists elsewhere 
won't have much of an effect.

One possibility to help with reducing the annotation list memory 
consumption is to modify the gnast code such that the annotation list 
for each term is initialised to null, and then only created the first 
time the getAnns() method is called. I'd wager that for a lot of terms, 
it is never called.

And regarding your previous email, I definitely think a 2-3% increase in 
CPU use is a good trade off for halving memory usage. In the parser and 
typechecker, we can always wait a bit longer for the result, but once 
we're out of memory, there's nowhere to go!

Cheers,
Tim

On 25/01/12 17:47, Leo Freitas wrote:
> Hi guys,
>
> I think I found another source of some of the retained instances that seem to have weird behaviour.
>
> Term has a List<Object>  getAnns() method, which exposes information like LocAnn and TypeAnn and others.
> It's called in 1112 places across 17 CZT sub-projects: that exposes the ArrayList / Object[] for retention from those
> that use the result or keep it in maps or the like.
>
> Having said that, I couldn't find a place of misuse in the scattered references that I searched for (e.g., this might
> be just another red herring or a key point for modification/memory-performance-improvement).....
>
> thoughts / comments / suggestions ?
>
> Leo
> ------------------------------------------------------------------------------
> Keep Your Developer Skills Current with LearnDevNow!
> The most comprehensive online learning library for Microsoft developers
> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
> Metro Style Apps, more. Free future releases when you subscribe now!
> http://p.sf.net/sfu/learndevnow-d2d
> _______________________________________________
> CZT-Devel mailing list
> CZT...@li...
> https://lists.sourceforge.net/lists/listinfo/czt-devel
>

[CZT-Devel] memory optimisation - comments please...

[Leo F. <leo...@ne...>]

Hi guys,

I think I found another source of some of the retained instances that seem to have weird behaviour.

Term has a List<Object> getAnns() method, which exposes information like LocAnn and TypeAnn and others.
It's called in 1112 places across 17 CZT sub-projects: that exposes the ArrayList / Object[] for retention from those
that use the result or keep it in maps or the like. 

Having said that, I couldn't find a place of misuse in the scattered references that I searched for (e.g., this might 
be just another red herring or a key point for modification/memory-performance-improvement)..... 

thoughts / comments / suggestions ?

Leo

Re: [CZT-Devel] Memory allocation with ArrayLists; arrays; etc.

[Leo F. <leo...@ne...>]

Hi Tim,

Yes, after parsing. And yes, I found that rather odd too. There is something funny going on 
in Garbage Collection somewhere. I guess the trouble is that there are certain structures
(e.g., TokenSequence iterators) that end up retaining the biggest part of the object memory.

Having said that, I was now trying to find the sources of the problem(s) using more intrusive 
snapshot points (AKA memory dumps at particular execution points).

As is, the only major problem with memory consumption is the duplication (e.g., ArrayList/Object[]).
The presence of various (4-6) BigIntegers in LocAnn is also an issue, but given most of them are shared
it's not such a big deal. 

I've tweaked the PerformanceSettings constant(s) a little bit more to keep arraylists/object[] capacity 0 at creation,
and that just about halved the memory used (!)... possibly at some speed expense given it will take some time to
increase capacity. 

At the profiling sessions done, this wasn't a problem: I've put initial capacity at 0, 1, and 10 (e.g., only very few go 
beyond that, mainly on type info), and the CPU increase was marginal (e,g., with 10 it was 300ms - 2.75% - quicker; with 1 100ms quicker), 
yet  the memory gain was significant (e.g., with 10 and 1 it was it was twice the heap at 100MB instead of 50MB for 0).

One nice thing would be to have this as a configuration file or a SectionManager option.... once all key bottleneck points are identified.
By default I will keep it at 0, as the performance penalty in CPU time seems marginal/acceptable (e.g., 2-3% penalty for halving memory needs).

Suggestions / comments?

Best,
Leo

On 24 Jan 2012, at 23:11, Tim Miller wrote:

> Nice work!
> 
> When you say you added "parser = null", I assume you mean after you do 
> the parsing? This would mean that, prior to your change, a reference to 
> the parser is being kept even after the variable scope of the variable 
> "parser" ends? That seems odd.
> 
> On 25/01/12 01:22, Leo Freitas wrote:
>> Tim,
>> 
>> I've forgot to mention below that I was talking about parsing and typechecking only.
>> 
>> I've just run the same profiling setup on the VCG for these larger examples and yes, the worst ones were
>> Object[] (24%), char[] (17.3%), ArrayList[] (9%) and BigInteger (2-3%).
>> 
>> Surprising ones were int[] (6.5%) and short[] (6%) arrays, since they are not explicitly created anywhere in CZT.
>> LocalAnn took 3%; Iterators were also a surprise: both iterator() and listIterator() cover about 10%, where more
>> obvious ones like String (0.9%) and ZName (1.7%) were quite low.
>> 
>> I've run the profiling sessions about 3-5 times on each example taking the average.
>> 
>> ----
>> 
>> Searching for potential sources of such unexpected types, I managed to find a few successful candidates:
>> 
>> a) ParseUtils
>> 
>> char[], short[], and int[] are mostly present in the Java CUP low-level classes to represent the internal parsing tables from the grammar.
>> I simply added an explicit "parser = null" to the main ParseUtils.parse method. These arrays combined account for about 30% of memory.
>> 
>> after the change, these arrays now account for 1.1% (!!!) that's quite an improvement.
>> 
>> b) ListIterator and Iterator
>> 
>> These appear in various places, but places that should influence all runs are only on SmartScanner (and possibly PrettyPrinter).
>> After this change, they went from a combined (ListIterator + Iterator) 49% footprint to 17%; another good improvement.
>> 
>> d) ArrayList and Object[]
>> 
>> There are various places and identifying where are the ones of most significance is tedious/time-consuming.
>> Instead, I've done a thorough search through various projects and changed default constructors to more sensible values, when possible,
>> or to a parameterised default when not (e.g., PerformanceSettings interface in util project).
>> 
>> This led to a decrease in the memory footprint of these objects of....
>> 
>> e) BigInteger
>> 
>> Why do we need big integers within LocAnn and other places? I guess because long would be potentially small?
>> But would there really be something bigger than 2^32 or 2^64 in number of lines of source in a file say? Hum...
>> 
>> In terms of memory footprint, it varied between 0.7% to 3%, depending on the example. I won't change this one for now.
>> 
>> f) Garbage collection time
>> 
>> Firstly it was about 16-20% now it was about 46.1% of time taken. Don't know if that's related to changes, but looks like it.
>> 
>> ======
>> 
>> Changes a) and b) led to a drastic memory footprint decrease from 8MB to 0.91MB;
>> Other changes led to a minor improvement in comparison (i.e., 0.80MB).
>> 
>> That's despite the fact that changes in d) were the most numerous - they didn't amount to much improvement, but some.
>> 
>> Hopefully this will enable much better performance on larger specs. I am committing it now to see how it goes.
>> 
>> Best,
>> Leo
>> 
>> On 23 Jan 2012, at 14:00, Leo Freitas wrote:
>> 
>>> Hi Tim,
>>> 
>>> That's interesting. I';ve been doing similar (profiling) tests over specs of some size (e.g., Mondex, Tokeneer, Xenon, IEEE float point unit);
>>> although I guess they are smaller than iFACTS - apart from Xenon, which is quite large.
>>> 
>>> On the profiling sessions, the worst culprit was "char[]" arrays, mostly from the java_cup lexer. The Object[] and ArrayList[] were about 2-3% each,
>>> for what the char[] was 90% (!)... Similarly, on profiling CPU, it was the IO operations on zzRefill within the java_cup scanner that took the largest chunk
>>> of the time (27%). Smart scanning (e.g., lookahead) has taken only about 2-3%.
>>> 
>>> I wonder... what was the profilling setup that you used to get to the creation of Object[] ArrayList as the main problem?
>>> Although the change you refer to below shouldn't be relatively simple to change, as Petra pointed out.
>>> 
>>> I just want to get the right picture to tackle such performance problem for larger specs.
>>> 
>>> Best,
>>> Leo
>>> 
>>> On 5 Jan 2012, at 23:51, Tim Miller wrote:
>>> 
>>>> Hi everyone,
>>>> 
>>>> Anthony Hall and I have been discussing some memory problems that CZT
>>>> has when parsing large specifications. Anthony has been trying to
>>>> typecheck the iFacts specification, but without much luck due to the
>>>> large memory resources.
>>>> 
>>>> We've each been playing around with VisualVM, and Anthony pointed out is
>>>> that the two largest memory hogs are Object[] and ArrayList, taking up
>>>> around 23% and 10% of the heap respectively.
>>>> Most of the object arrays and ArrayLists contain exactly 10 items, and
>>>> almost all items in these lists are null.
>>>> 
>>>> After some poking around, I discovered that when ArrayList is created
>>>> using the default constructor, it allocates 10 items initially. This
>>>> appears to be where the 10 items come from in each case. I suspect this
>>>> may also be contributing to some of the memory problems, considering
>>>> that CZT has so many empty annotation lists, etc. Creating ArrayLists
>>>> with an initial capacity of 0 or 1 (using the constructor ArrayList(int
>>>> initialCapacity)) may give us some substantial space savings
>>>> 
>>>> It appears that most ArrayLists are created in the gnast-generated code.
>>>> Petra, how difficult would it be to create these lists using this other
>>>> constructor?
>>>> 
>>>> Regards,
>>>> Tim
>>>> 
>>>> ------------------------------------------------------------------------------
>>>> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
>>>> infrastructure or vast IT resources to deliver seamless, secure access to
>>>> virtual desktops. With this all-in-one solution, easily deploy virtual
>>>> desktops for less than the cost of PCs and save 60% on VDI infrastructure
>>>> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
>>>> _______________________________________________
>>>> CZT-Devel mailing list
>>>> CZT...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/czt-devel
>>> 
>>> 
>>> ------------------------------------------------------------------------------
>>> Try before you buy = See our experts in action!
>>> The most comprehensive online learning library for Microsoft developers
>>> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
>>> Metro Style Apps, more. Free future releases when you subscribe now!
>>> http://p.sf.net/sfu/learndevnow-dev2
>>> _______________________________________________
>>> CZT-Devel mailing list
>>> CZT...@li...
>>> https://lists.sourceforge.net/lists/listinfo/czt-devel
>> 
>> 
> 
> 
> ------------------------------------------------------------------------------
> Keep Your Developer Skills Current with LearnDevNow!
> The most comprehensive online learning library for Microsoft developers
> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
> Metro Style Apps, more. Free future releases when you subscribe now!
> http://p.sf.net/sfu/learndevnow-d2d
> _______________________________________________
> CZT-Devel mailing list
> CZT...@li...
> https://lists.sourceforge.net/lists/listinfo/czt-devel