CEDET

David Engster writes:
> I've checked in the new Fortran90 support for Semantic, which comes in
> the form of a (very basic) grammar f90.by and semantic-f90.el.

Eric, I'm not sure what to do with the Makefiles. I added the files to
the Projects; should I now simply regenerate the Makefiles with EDE and
check them in? Or is there some other workflow for this?

-David

On 10/03/2010 04:16 PM, David Engster wrote:
> David Engster writes:
>> I've checked in the new Fortran90 support for Semantic, which comes in
>> the form of a (very basic) grammar f90.by and semantic-f90.el.

Huzzah!

> Eric, I'm not sure what to do with the Makefiles. I added the files to
> the Projects; should I now simply regenerate the Makefiles with EDE and
> check them in? Or is there some other workflow for this?

When you load in a .el or .by file, it should have asked to add it to 
some target.  In this case, I assume it is the "by" target and the 
"languages" target.

Then do:

M-x ede-proj-regenerate

to recreate the Makefile.

After running the top-level build and utest as a loose test, you would 
need to check in:

cedet/semantic/bovine/Project.ede
                       Makefile

and that's it.

Thanks!
Eric

On 10/03/2010 03:25 PM, David Engster wrote:
> I've checked in the new Fortran90 support for Semantic, which comes in
> the form of a (very basic) grammar f90.by and semantic-f90.el.
>
> There's still a lot of work to do, but I think it's in a state to be
> actually useful; please see the tests in semantic/tests/testf90.f90 to
> see which kind of completions the parser can currently provide. If there
> are some F90 programmers out there, I'd appreciate if they could do some
> testing with the code. Please note that while F77 is more or less a
> subset of F90, it is not really supported by the parser; this is also
> why the parser is only activated for f90-mode, but not for fortran-mode,
> since it relies on reasonably 'modern' Fotran90/95 code.

Hi David,

I've been looking in your code you checked in.  Looks like some good stuff.

In the test files I noticed that you used the C++/Java comment character 
"//" to mark unit test locations.

Curious, I looked in the unit test harness and was sad to see that the 
regexp used has the // in it directly.

I'll take a wild guess that this should be replaced with 
`comment-start-skip' or something equivalent so you can use sane fortran 
comments in you test files.  It appears that regexp will work for both 
these modes.

Thanks!
Eric

Eric M. Ludlam writes:
> In the test files I noticed that you used the C++/Java comment character 
> "//" to mark unit test locations.
>
> Curious, I looked in the unit test harness and was sad to see that the 
> regexp used has the // in it directly.
>
> I'll take a wild guess that this should be replaced with 
> `comment-start-skip' or something equivalent so you can use sane fortran 
> comments in you test files.  It appears that regexp will work for both 
> these modes.

Yes. I committed a change which replaces the '//' with
`comment-start-skip' in semantic-ia-utest.

> After running the top-level build and utest as a loose test, you would 
> need to check in:

> cedet/semantic/bovine/Project.ede
>                        Makefile

> and that's it.

I had added the files to Project.ede, but wasn't sure if the Makefile
would somehow get regenerated automagically. I did it now like you said
and it seems to work fine.

-David

On 10/10/2010 04:36 PM, David Engster wrote:
> Eric M. Ludlam writes:
>> In the test files I noticed that you used the C++/Java comment character
>> "//" to mark unit test locations.
>>
>> Curious, I looked in the unit test harness and was sad to see that the
>> regexp used has the // in it directly.
>>
>> I'll take a wild guess that this should be replaced with
>> `comment-start-skip' or something equivalent so you can use sane fortran
>> comments in you test files.  It appears that regexp will work for both
>> these modes.
>
> Yes. I committed a change which replaces the '//' with
> `comment-start-skip' in semantic-ia-utest.

Thanks!

>> After running the top-level build and utest as a loose test, you would
>> need to check in:
>
>> cedet/semantic/bovine/Project.ede
>>                         Makefile
>
>> and that's it.
>
> I had added the files to Project.ede, but wasn't sure if the Makefile
> would somehow get regenerated automagically. I did it now like you said
> and it seems to work fine.

Great!  Thanks for all that.

I looks like you may need to do the same to the Makefile in 
semantic/tests for your Fortran test file.

This would only be caught by doing an install test.  See:

cedet/testdist.sh

which works like this:

make
make dist
./testdist.sh emacs

(or xemacs, or whichever emacs you want to use.)

In looking at buildbot, it does not run the integration test either 
which you can do with:

make itest

This currently fails right away for me right now.  Probably because it 
depended on the vagaries of the old `semantic-tag-similar-p'.  I checked 
in a small fix for that, but it is failing in the texinfo test, but I'm 
not sure why yet.

If we can get the itest running, then doing a dist test may be a good 
thing for build-bot to use, possibly as a secondary build test.

Eric

Eric M. Ludlam writes:
>> I had added the files to Project.ede, but wasn't sure if the Makefile
>> would somehow get regenerated automagically. I did it now like you said
>> and it seems to work fine.
>
> Great!  Thanks for all that.
>
> I looks like you may need to do the same to the Makefile in 
> semantic/tests for your Fortran test file.

I had to add the file to semantic-ua-test. I think the Makefile doesn't
really do anything there?

> This would only be caught by doing an install test.  See:
>
> cedet/testdist.sh
>
> which works like this:
>
> make
> make dist
> ./testdist.sh emacs

This doesn't work for me, because I can't generate ChangeLogs, and
rcs2log won't work with bzr. You should either simply drop Changelogs or
we'll have to search for another tool to generate them.

> (or xemacs, or whichever emacs you want to use.)

CEDET doesn't build under xemacs at the moment.

> In looking at buildbot, it does not run the integration test either 
> which you can do with:
>
> make itest
>
> This currently fails right away for me right now.  Probably because it 
> depended on the vagaries of the old `semantic-tag-similar-p'.  I checked 
> in a small fix for that, but it is failing in the texinfo test, but I'm 
> not sure why yet.

I think I fixed it; it was a problem similar to the one you already
encountered. I had to add :members to the ignorable attributes when the
'extra' tags are tested, since tag-similar-p now also compares sublists
of tags.

> If we can get the itest running, then doing a dist test may be a good 
> thing for build-bot to use, possibly as a secondary build test.

It would have to run non-interactively for buildbot, of course. I
changed cit-test.sh accordingly so that it can also run in batch
mode. I'll see that I add it as another test, but first I have to make
sure it runs on the other platforms (the Windows buildslave currently
has a stupid failure; will be fixed tomorrow).

-David

On 10/19/2010 12:40 PM, David Engster wrote:
> Eric M. Ludlam writes:
>>> I had added the files to Project.ede, but wasn't sure if the Makefile
>>> would somehow get regenerated automagically. I did it now like you said
>>> and it seems to work fine.
>>
>> Great!  Thanks for all that.
>>
>> I looks like you may need to do the same to the Makefile in
>> semantic/tests for your Fortran test file.
>
> I had to add the file to semantic-ua-test. I think the Makefile doesn't
> really do anything there?

The Makefile is in cedet/semantic/tests where it would be included in 
the distribution.  That's why I was suggesting the dist test below.

Sorry that wasn't clear.

>> This would only be caught by doing an install test.  See:
>>
>> cedet/testdist.sh
>>
>> which works like this:
>>
>> make
>> make dist
>> ./testdist.sh emacs
>
> This doesn't work for me, because I can't generate ChangeLogs, and
> rcs2log won't work with bzr. You should either simply drop Changelogs or
> we'll have to search for another tool to generate them.

Hmmm, I was under the impression there was an rcs2log look-alike program 
for bazaar, but I can't find one by poking around in google.

>> (or xemacs, or whichever emacs you want to use.)
>
> CEDET doesn't build under xemacs at the moment.

Indeed.  Something broke along the way and I was never able to figure it 
out. :(

>> In looking at buildbot, it does not run the integration test either
>> which you can do with:
>>
>> make itest
>>
>> This currently fails right away for me right now.  Probably because it
>> depended on the vagaries of the old `semantic-tag-similar-p'.  I checked
>> in a small fix for that, but it is failing in the texinfo test, but I'm
>> not sure why yet.
>
> I think I fixed it; it was a problem similar to the one you already
> encountered. I had to add :members to the ignorable attributes when the
> 'extra' tags are tested, since tag-similar-p now also compares sublists
> of tags.

Great!  Thanks for looking into that.

>> If we can get the itest running, then doing a dist test may be a good
>> thing for build-bot to use, possibly as a secondary build test.
>
> It would have to run non-interactively for buildbot, of course. I

Is that because there is no display to connect to?  I don't have to 
"interact" with Emacs to run the test, but it does need a display.

> changed cit-test.sh accordingly so that it can also run in batch

That's probably a good idea.  It runs interactively since debugging is 
that much easier for the complexities that the integration tests are 
trying to create.  It also exercises the display engine for COGRE.

> mode. I'll see that I add it as another test, but first I have to make
> sure it runs on the other platforms (the Windows buildslave currently
> has a stupid failure; will be fixed tomorrow).

Nifty.  Good Luck with that.  When these tests suites first came out, 
lots of folks helped identify testing issues on a wide range of 
platforms.  Hopefully it will work fine.

Thanks!
Eric

"Eric M. Ludlam" <eric@...> writes:

> On 10/19/2010 12:40 PM, David Engster wrote:
>> Eric M. Ludlam writes:
>> This doesn't work for me, because I can't generate ChangeLogs, and
>> rcs2log won't work with bzr. You should either simply drop Changelogs or
>> we'll have to search for another tool to generate them.
>
> Hmmm, I was under the impression there was an rcs2log look-alike program 
> for bazaar, but I can't find one by poking around in google.

I avoid Bzr as much as I can, but AFAIK there is a --gnu-changelog
option to `bzr log' (dunno since which version) which outputs the log in
the GNU ChangeLog format.

HTH,

Štěpán

Eric M. Ludlam writes:
> On 10/19/2010 12:40 PM, David Engster wrote:
>> Eric M. Ludlam writes:
>>>> I had added the files to Project.ede, but wasn't sure if the Makefile
>>>> would somehow get regenerated automagically. I did it now like you said
>>>> and it seems to work fine.
>>>
>>> Great!  Thanks for all that.
>>>
>>> I looks like you may need to do the same to the Makefile in
>>> semantic/tests for your Fortran test file.
>>
>> I had to add the file to semantic-ua-test. I think the Makefile doesn't
>> really do anything there?
>
> The Makefile is in cedet/semantic/tests where it would be included in 
> the distribution.  That's why I was suggesting the dist test below.
>
> Sorry that wasn't clear.

Thanks for clarifying. I've regenerated the Makefile.

>>> This would only be caught by doing an install test.  See:
>>>
>>> cedet/testdist.sh
>>>
>>> which works like this:
>>>
>>> make
>>> make dist
>>> ./testdist.sh emacs
>>
>> This doesn't work for me, because I can't generate ChangeLogs, and
>> rcs2log won't work with bzr. You should either simply drop Changelogs or
>> we'll have to search for another tool to generate them.
>
> Hmmm, I was under the impression there was an rcs2log look-alike program 
> for bazaar, but I can't find one by poking around in google.

Yep, there is. It's called bzr. ;-)

bzr log --gnu-changelog

Didn't know that one, either...

>>> (or xemacs, or whichever emacs you want to use.)
>>
>> CEDET doesn't build under xemacs at the moment.
>
> Indeed.  Something broke along the way and I was never able to figure it 
> out. :(

Yes, I wasn't able to figure out why it fails to compile the
grammar during the build. It seems like some hook isn't running in batch
mode or something.

>>> If we can get the itest running, then doing a dist test may be a good
>>> thing for build-bot to use, possibly as a secondary build test.
>>
>> It would have to run non-interactively for buildbot, of course. I
>
> Is that because there is no display to connect to?  I don't have to 
> "interact" with Emacs to run the test, but it does need a display.

Well, you have to interact if there's an error. :-) Otherwise, the test
just hangs since Emacs doesn't exit. When running from buildbot, there's
only stdout/stderr. In batch mode, Emacs will log everything from
'message' to stderr, so you have a nice log from the test.

>> changed cit-test.sh accordingly so that it can also run in batch
>
> That's probably a good idea.  It runs interactively since debugging is 
> that much easier for the complexities that the integration tests are 
> trying to create.  It also exercises the display engine for COGRE.

>From what I see in the logs, it looks to me that Emacs is running the
full integration test in batch mode. I'm guessing it also happily draws
the UML graphs, you just don't see them. ;-)

In case of an error, you'll see the backtrace in the logs. You are right
though that the backtrace often won't really help, since so many
external programs are involved, but I can then run it interactively on
the failing buildslave and provide more information.

>> mode. I'll see that I add it as another test, but first I have to make
>> sure it runs on the other platforms (the Windows buildslave currently
>> has a stupid failure; will be fixed tomorrow).
>
> Nifty.  Good Luck with that.  When these tests suites first came out, 
> lots of folks helped identify testing issues on a wide range of 
> platforms.  Hopefully it will work fine.

They do work, also on Windows. However, in Windows, the Automake itest
is running unbearably slow - it takes 52 minutes(!!!). I mean, cygwin
was never particularly fast, but that's excessive. I'm guessing it's due
to the heavy forking/exec from the autotools, which is a *very* costly
operation on Windows, but I don't remember it being that slow when I
tested it last time under XP; maybe it's a new Win7 thingy. Anyway, the
automake itest is working, but I'm disabling it for now for the Windows
build.

Another problem is that Emacs22 won't run the integration tests in batch
mode; it just hangs after a short time, I'm guessing when some external
process is fired up. Maybe Emacs22 simply doesn't support that in batch
mode, I don't know, but I had to disable the integration tests for
Emacs22.

Also, the csope test is skipped, because cedet-cscope-min-version is set
to "16", but the latest version is 15.7...

Apart from those problems, everything runs fine now.

-David

On 10/20/2010 03:49 PM, David Engster wrote:
> Eric M. Ludlam writes:
>> On 10/19/2010 12:40 PM, David Engster wrote:
>>> Eric M. Ludlam writes:
>>>> This would only be caught by doing an install test.  See:
>>>>
>>>> cedet/testdist.sh
>>>>
>>>> which works like this:
>>>>
>>>> make
>>>> make dist
>>>> ./testdist.sh emacs
>>>
>>> This doesn't work for me, because I can't generate ChangeLogs, and
>>> rcs2log won't work with bzr. You should either simply drop Changelogs or
>>> we'll have to search for another tool to generate them.
>>
>> Hmmm, I was under the impression there was an rcs2log look-alike program
>> for bazaar, but I can't find one by poking around in google.
>
> Yep, there is. It's called bzr. ;-)
>
> bzr log --gnu-changelog
>
> Didn't know that one, either...

Ah, great.  That should enable the changelog generation then, and allow 
making distributions.

I'll give it a try next time I've got a few moments.

>>>> (or xemacs, or whichever emacs you want to use.)
>>>
>>> CEDET doesn't build under xemacs at the moment.
>>
>> Indeed.  Something broke along the way and I was never able to figure it
>> out. :(
>
> Yes, I wasn't able to figure out why it fails to compile the
> grammar during the build. It seems like some hook isn't running in batch
> mode or something.

On the XEmacs mailing list someone chased this down to something similar 
broken in mode-local, like the major-mode-changed hook or something.

>>> mode. I'll see that I add it as another test, but first I have to make
>>> sure it runs on the other platforms (the Windows buildslave currently
>>> has a stupid failure; will be fixed tomorrow).
>>
>> Nifty.  Good Luck with that.  When these tests suites first came out,
>> lots of folks helped identify testing issues on a wide range of
>> platforms.  Hopefully it will work fine.
>
> They do work, also on Windows. However, in Windows, the Automake itest
> is running unbearably slow - it takes 52 minutes(!!!). I mean, cygwin
> was never particularly fast, but that's excessive. I'm guessing it's due
> to the heavy forking/exec from the autotools, which is a *very* costly
> operation on Windows, but I don't remember it being that slow when I
> tested it last time under XP; maybe it's a new Win7 thingy. Anyway, the
> automake itest is working, but I'm disabling it for now for the Windows
> build.

I've looked at the output in buildbot.  It is great you've been able to 
get all this stuff running there.  I'm sure that will be a big help.

> Another problem is that Emacs22 won't run the integration tests in batch
> mode; it just hangs after a short time, I'm guessing when some external
> process is fired up. Maybe Emacs22 simply doesn't support that in batch
> mode, I don't know, but I had to disable the integration tests for
> Emacs22.

I remember having trouble getting Emacs 22 to run everything too, but 
batch mode for the integration test was not something I tried.  It is 
missing a few Eamcs 23 features CEDET became dependent on. :(

> Also, the csope test is skipped, because cedet-cscope-min-version is set
> to "16", but the latest version is 15.7...

This came up on a different forum around the time you first sent this. 
I somehow ended up with 16a (an alpha?) on my system.  If cscope works 
with 15.7 (or earlier, I don't really know), we can just change the 
minimum needed version.

Eric

Eric M. Ludlam writes:
> On 10/20/2010 03:49 PM, David Engster wrote:
>> bzr log --gnu-changelog
>>
>> Didn't know that one, either...
>
> Ah, great.  That should enable the changelog generation then, and allow 
> making distributions.

Yes, but it's a bit strange that bzr includes merge commits in the
ChangeLog. This is why there is this

2010-10-16  David Engster  <dengste@...>

        Extended `semantic-tag-similar-p' to better compare arguments
        and types.

which obviously doesn't say much, because the 'real' changes are in the
commits I did in the branch (which are also visible in the ChangeLog,
though). BTW, you can get a very nice overview by installing a graphical
plugin like qbzr and then just do 'bzr qlog'.

> I remember having trouble getting Emacs 22 to run everything too, but 
> batch mode for the integration test was not something I tried.  It is 
> missing a few Eamcs 23 features CEDET became dependent on. :(

The integration test runs fine interactively with Emacs22, so it's
really just a problem with batch mode. I was actually quite surprised
that it worked without problems with Emacs 23 and later.

>> Also, the csope test is skipped, because cedet-cscope-min-version is set
>> to "16", but the latest version is 15.7...
>
> This came up on a different forum around the time you first sent this. 
> I somehow ended up with 16a (an alpha?) on my system.  If cscope works 
> with 15.7 (or earlier, I don't really know), we can just change the 
> minimum needed version.

It seems to work fine with 15.7, but not with 15.6.

-David

David Engster writes:

> Eric M. Ludlam writes:
>> On 10/20/2010 03:49 PM, David Engster wrote:
>>> bzr log --gnu-changelog
>>> 
>>> Didn't know that one, either...
>> 
>> Ah, great.  That should enable the changelog generation then, and allow 
>> making distributions.

> Yes, but it's a bit strange that bzr includes merge commits in the
> ChangeLog. This is why there is this

> 2010-10-16  David Engster  <dengste@...>

>         Extended `semantic-tag-similar-p' to better compare arguments
>         and types.

Well, it seems that the '--gnu-changelog' option is just a brain-dead
prettify of 'bzr log -n0'.

Problem here is that the merge commit could indeed include some changes
by itself, so it is neither an option to simply drop all merge commit
messages when doing a "--gnu-changelog" :)

Personally, I think that a ChangeLog file is not needed (while a NEWS
is, and that's where messages in merge commits can help), as all the
necessary information should be already encoded on the repository
history.

BTW, this reminds me that the synchronization changes I've been doing
are infinitely changelog unfriendly.

Lluis

-- 
 "And it's much the same thing with knowledge, for whenever you learn
 something new, the whole world becomes that much richer."
 -- The Princess of Pure Reason, as told by Norton Juster in The Phantom
 Tollbooth

On 10/25/2010 04:35 PM, David Engster wrote:
>>> >>  Also, the csope test is skipped, because cedet-cscope-min-version is set
>>> >>  to "16", but the latest version is 15.7...
>> >
>> >  This came up on a different forum around the time you first sent this.
>> >  I somehow ended up with 16a (an alpha?) on my system.  If cscope works
>> >  with 15.7 (or earlier, I don't really know), we can just change the
>> >  minimum needed version.
> It seems to work fine with 15.7, but not with 15.6.

I'll go with that then.  Thanks!

Eric