CEDET

Ok, I've written a more thorough guide for developing in CEDET. It covers the
work of developers.

What is not covered is how release managers should manage updates to release
branches. I have just never done that, so I'm offering myself to do this work so
I'll gather some experience and document it on a new version of the attached
document.

You will also see at the end of it that it would be nice to have some kind of
commit log message standard, so that unknowledgeable eyes will quickly see what
has happend at the high level just by peering through the log.

Lluis

On 08/25/2010 07:25 PM, Lluís wrote:
> Ok, I've written a more thorough guide for developing in CEDET. It covers the
> work of developers.
>
> What is not covered is how release managers should manage updates to release
> branches. I have just never done that, so I'm offering myself to do this work so
> I'll gather some experience and document it on a new version of the attached
> document.

Thanks Lluis,

This is an impressive bit of work, and will be invaluable to me as I 
haven't used bazaar yet.  I really appreciate the thought and time that 
has been put into your document.

> You will also see at the end of it that it would be nice to have some kind of
> commit log message standard, so that unknowledgeable eyes will quickly see what
> has happend at the high level just by peering through the log.

I have always used commit messages in ChangeLog format and used rcs2log 
to create ChangeLogs that are (in theory) compatible with the GNU 
standards.  Long ago I found that I was bad at keeping changelogs up to 
date, and was glad to find a way to automate the process. :)

In CVS, since a commit via the Emacs support is for one file, the format 
would be just

(function1): comment
(function2): comment

If bazaar via Emacs is the same, that would be easy.  You mention the 
Emacs mode for performing operations.  I'll guess that would be the 
standard way most of us will operate, so perhaps those keybindings can 
be added?

Thanks!
Eric

On 08/25/2010 07:25 PM, Lluís wrote:
> Ok, I've written a more thorough guide for developing in CEDET. It covers the
> work of developers.

Hi Lluis,

I've been working through the instructions.  They've worked well so far. 
  I created a branch, and made a .bzrignore file to ignore all the 
productions from the build.  I'm assuming this is a handy thing to have 
checked in?

While trying to check in the above, one of the things I noticed is that 
bzr recognizes that semantic/semantic-grammar-wy.el is modified, due to 
the build.  Following the instructions in your document I tried the 
merge from the mainline into my branch.  This ought to be a no-op. 
Unfortunately, it won't allow the merge to run unless I also commit the 
grammar-wy change.  Does anyone know how to work around this (in terms 
of process.)

Thanks
Eric

Eric M Ludlam writes:
> I've been working through the instructions.  They've worked well so far. 
>   I created a branch, and made a .bzrignore file to ignore all the 
> productions from the build.  I'm assuming this is a handy thing to have 
> checked in?

I assume you've committed this into your local branch. And yes, this would be
handy to have available on the trunk. In fact, this kind of change could be
nicely committed directly into the trunk, alltogether with the deletion of the
now-obsolete cvsignore files.


> While trying to check in the above, one of the things I noticed is that 
> bzr recognizes that semantic/semantic-grammar-wy.el is modified, due to 
> the build.

Aha... apart from files that should be ignored (e.g., .elc), the file you
mention is modified during build. This can be avoided in two ways:
- re-generating parser code only when parser definition is modified (this is a
  simple matter of makefile dependencies)
- avoid the modifications inserted by the re-generation step:
  - changes in spacing (simply re-commit with new spacing, and no new changes
    should arise)
  - "History" section is deleted from the .el file; I believe that's because
    there's no History section on the file?
  - X-RCS field is "resetted"; I've deleted all those fields from my local
    branch, as I think they're no longer necessary
  - "Author" and "Created" fields are set during build, but I think they should
    be maintained from the parser definition

Now, if you look into the results after a "make clean-all", you'll see that,
additionally, "semantic/bovine/semantic-skeleton-by.el" is deleted, but was also
present on the repository. I think the compiled skeleton parser should not be
present on the repository, or else not deleted during a clean action.

This brings me to the point that on my local branch, I cannot seem to regenerate
the Makefile files from their Project.ede definitions. Right now I'm rewriting
the make infrastructure to work with plain Makfiles, as right now I'm heavily
reorganizing the location of files.

The layout I want to have at the end is:

cedet
 |- <various top-level files>
 |- src    -- bovine/wisent parser definitions (maybe it should be called parsers)
 |- lisp   -- plain lisp code, will contain compiled parsers after build
 |   |- cedet
 |   |   |- cogre
 |   |   |- common
 |   |   |- contrib
 |   |   |- ede
 |   |   |- ede.el
 |   |   |- semantic
 |   |   |- semantic.el
 |   |   |- srecode
 |   |   `- srecode.el
 |   |- eieio
 |   `- speedbar -- is speedbar going to be developed by CEDET, or has it moved
 |               -- altogether to emacs? the code in emacs is not exactly the
 |               -- same as the one in cedet, but haven't really checked on the
 |               -- differences
 |- images -- image files from the various packages
 |   |- cogre
 |   |- common
 |   `- speedbar
 |- doc    -- documentation files
 `- tests  -- all test suites, organized per-package/subsystem
           -- (e.g., language parsing/completion, ede, ...)


> Following the instructions in your document I tried the 
> merge from the mainline into my branch.  This ought to be a no-op. 
> Unfortunately, it won't allow the merge to run unless I also commit the 
> grammar-wy change.  Does anyone know how to work around this (in terms 
> of process.)

Ok, so now for your question :)

You cannot merge if you have local non-committed changes, so you have 3 ways out
of here (to be executed before merging from the mainline):

- bzr revert; will loose all non-committed changes (makes sense in this case)
- bzr commit; has no real sense to commit those changes on the compiled grammar
- bzr shelve; I didn't document that one, but puts local changes aside, then you
              can merge and 'bzr unshelve' later on (see 'bzr help shelve')

Hope this helps.

Lluis

-- 
 "And it's much the same thing with knowledge, for whenever you learn
 something new, the whole world becomes that much richer."
 -- The Princess of Pure Reason, as told by Norton Juster in The Phantom
 Tollbooth

On 08/31/2010 09:42 AM, Lluís wrote:
> Eric M Ludlam writes:
>> I've been working through the instructions.  They've worked well so far.
>>    I created a branch, and made a .bzrignore file to ignore all the
>> productions from the build.  I'm assuming this is a handy thing to have
>> checked in?
>
> I assume you've committed this into your local branch. And yes, this would be
> handy to have available on the trunk. In fact, this kind of change could be
> nicely committed directly into the trunk, alltogether with the deletion of the
> now-obsolete cvsignore files.

I made a local branch for that to try out that development path, so I'll 
continue then with my innocuous test change.

>> While trying to check in the above, one of the things I noticed is that
>> bzr recognizes that semantic/semantic-grammar-wy.el is modified, due to
>> the build.
>
> Aha... apart from files that should be ignored (e.g., .elc), the file you
> mention is modified during build. This can be avoided in two ways:
> - re-generating parser code only when parser definition is modified (this is a
>    simple matter of makefile dependencies)

This is the default, but when you check stuff out the timestamps don't 
always follow nicely.

> - avoid the modifications inserted by the re-generation step:
>    - changes in spacing (simply re-commit with new spacing, and no new changes
>      should arise)
>    - "History" section is deleted from the .el file; I believe that's because
>      there's no History section on the file?
>    - X-RCS field is "resetted"; I've deleted all those fields from my local
>      branch, as I think they're no longer necessary
>    - "Author" and "Created" fields are set during build, but I think they should
>      be maintained from the parser definition

These seem like good ideas.  CVS somehow knew that nothing was changing, 
whereas bazaar detects many changes.  Does bazaar strip whitespace in 
some fancy way?

> Now, if you look into the results after a "make clean-all", you'll see that,
> additionally, "semantic/bovine/semantic-skeleton-by.el" is deleted, but was also
> present on the repository. I think the compiled skeleton parser should not be
> present on the repository, or else not deleted during a clean action.

This I do not recall.  It doesn't seem like it is necessary to me.

> This brings me to the point that on my local branch, I cannot seem to regenerate
> the Makefile files from their Project.ede definitions. Right now I'm rewriting
> the make infrastructure to work with plain Makfiles, as right now I'm heavily
> reorganizing the location of files.

The old Project.ede files are not going to work in this new world.  A 
new project, likely for all of CEDET should probably be built, and the 
old Project files abandoned.

Basically, delete all the old Project.ede files, and issue the command 
"M-x ede-new" under CEDET to create a Makefile project.  Then the same 
for every subdir, and add targets and files to targets as you go.

It has always been useful using EDE to manage CEDET to make sure that it 
is always well tested. :)

> The layout I want to have at the end is:
>
> cedet
>   |-<various top-level files>
>   |- src    -- bovine/wisent parser definitions (maybe it should be called parsers)

By this you mean the .by and .wy files?

There was a request via the email list to group code for a particular 
mode together.  Perhaps like:

   |- modes
   |   |- c (home of c.by, semantic-c.el, semanticdb-cscope.el)
   |   |- java (home of java.wy, java-tags.wy, ...)

Of course, then does SRecode templates for C or Java go in those dirs 
too?  I would guess this would be the way to get everything 
"intergrated" together more tightly, helping identify code for a 
particular language all in one place.

This would probably need to be run by the Emacs maintainers, since we 
are also trying to get the two trees to match.

>   |- lisp   -- plain lisp code, will contain compiled parsers after build
>   |   |- cedet
>   |   |   |- cogre
>   |   |   |- common
>   |   |   |- contrib
>   |   |   |- ede
>   |   |   |- ede.el
>   |   |   |- semantic
>   |   |   |- semantic.el
>   |   |   |- srecode
>   |   |   `- srecode.el
>   |   |- eieio
>   |   `- speedbar -- is speedbar going to be developed by CEDET, or has it moved
>   |               -- altogether to emacs? the code in emacs is not exactly the
>   |               -- same as the one in cedet, but haven't really checked on the
>   |               -- differences

Speedbar in CEDET is different only in that I've kept in code that 
allows it to run in ancient versions of Emacs.  I have not needed to 
enhance Speedbar in a very long time.

>   |- images -- image files from the various packages
>   |   |- cogre
>   |   |- common
>   |   `- speedbar
>   |- doc    -- documentation files
>   `- tests  -- all test suites, organized per-package/subsystem
>             -- (e.g., language parsing/completion, ede, ...)

There are two batches of tests.  Unit tests (close to the source) and 
integration tests (the cit-* stuff) which tests the packages from a 
pretty high level.  There are many tests in the same file as the sources 
it tests.  Those may be a challenge for you to find. ;)


>> Following the instructions in your document I tried the
>> merge from the mainline into my branch.  This ought to be a no-op.
>> Unfortunately, it won't allow the merge to run unless I also commit the
>> grammar-wy change.  Does anyone know how to work around this (in terms
>> of process.)
>
> Ok, so now for your question :)
>
> You cannot merge if you have local non-committed changes, so you have 3 ways out
> of here (to be executed before merging from the mainline):
>
> - bzr revert; will loose all non-committed changes (makes sense in this case)
> - bzr commit; has no real sense to commit those changes on the compiled grammar
> - bzr shelve; I didn't document that one, but puts local changes aside, then you
>                can merge and 'bzr unshelve' later on (see 'bzr help shelve')
>

Thanks.  I'll try this.

Eric

Eric M Ludlam writes:

[...]
>>> While trying to check in the above, one of the things I noticed is that
>>> bzr recognizes that semantic/semantic-grammar-wy.el is modified, due to
>>> the build.
>> 
>> Aha... apart from files that should be ignored (e.g., .elc), the file you
>> mention is modified during build. This can be avoided in two ways:
>> - re-generating parser code only when parser definition is modified (this is a
>> simple matter of makefile dependencies)

> This is the default, but when you check stuff out the timestamps don't 
> always follow nicely.

I see. It seems to have been reported already, and there is no definitive
solution to that:
  https://bugs.launchpad.net/bzr/+bug/245170


>> - avoid the modifications inserted by the re-generation step:
>> - changes in spacing (simply re-commit with new spacing, and no new changes
>> should arise)
>> - "History" section is deleted from the .el file; I believe that's because
>> there's no History section on the file?
>> - X-RCS field is "resetted"; I've deleted all those fields from my local
>> branch, as I think they're no longer necessary
>> - "Author" and "Created" fields are set during build, but I think they should
>> be maintained from the parser definition

> These seem like good ideas.  CVS somehow knew that nothing was changing, 
> whereas bazaar detects many changes.  Does bazaar strip whitespace in 
> some fancy way?

Not that I'm aware of. Is it possible that the current generator code is
producing files with different spacing?


>> Now, if you look into the results after a "make clean-all", you'll see that,
>> additionally, "semantic/bovine/semantic-skeleton-by.el" is deleted, but was also
>> present on the repository. I think the compiled skeleton parser should not be
>> present on the repository, or else not deleted during a clean action.

> This I do not recall.  It doesn't seem like it is necessary to me.

Well, I thought the compiled skeleton was there just let others quickly see how
a compiled parser looks like. But if you don't see a reason for it to be there,
I'll simply delete it.


>> This brings me to the point that on my local branch, I cannot seem to regenerate
>> the Makefile files from their Project.ede definitions. Right now I'm rewriting
>> the make infrastructure to work with plain Makfiles, as right now I'm heavily
>> reorganizing the location of files.

> The old Project.ede files are not going to work in this new world.  A 
> new project, likely for all of CEDET should probably be built, and the 
> old Project files abandoned.

> Basically, delete all the old Project.ede files, and issue the command 
> "M-x ede-new" under CEDET to create a Makefile project.  Then the same 
> for every subdir, and add targets and files to targets as you go.

> It has always been useful using EDE to manage CEDET to make sure that it 
> is always well tested. :)

Aaahhh... now it makes sense :)

Well, I'll start with makefiles from scratch, which I'm more comfortable with
right now, and a rewriting with EDE will follow once CEDET matches the interface
in current emacs (all the simple global-enabling functions and the like).

This reminds me of something else. There are a few lisp files on the root
directory, are all of them still alive? I'm specially talking about
cedet-build.el, as then I should rewrite it to follow the new directory
structure.


>> The layout I want to have at the end is:
>> 
>> cedet
>> |-<various top-level files>
>> |- src    -- bovine/wisent parser definitions (maybe it should be called parsers)

> By this you mean the .by and .wy files?

Yes. In part because emacs is only shipping the compiled parsers, not the
sources/definitions.


> There was a request via the email list to group code for a particular 
> mode together.  Perhaps like:

>    |- modes
>    |   |- c (home of c.by, semantic-c.el, semanticdb-cscope.el)
>    |   |- java (home of java.wy, java-tags.wy, ...)

> Of course, then does SRecode templates for C or Java go in those dirs 
> too?  I would guess this would be the way to get everything 
> "intergrated" together more tightly, helping identify code for a 
> particular language all in one place.

Right, this should ease language-specific development, but some pieces are
shared among different languages (e.g., cscope support mutiple languages).


> This would probably need to be run by the Emacs maintainers, since we 
> are also trying to get the two trees to match.

Well, first I'll try with the current reorganization, which is still big enough
to take some time :)

We can talk with the emacs people about this after the next cross-merge.


[...]
>> |   `- speedbar -- is speedbar going to be developed by CEDET, or has it moved
>> |               -- altogether to emacs? the code in emacs is not exactly the
>> |               -- same as the one in cedet, but haven't really checked on the
>> |               -- differences

> Speedbar in CEDET is different only in that I've kept in code that 
> allows it to run in ancient versions of Emacs.  I have not needed to 
> enhance Speedbar in a very long time.

Well, my question is if it makes sense to maintain it inside cedet, just like
eieio. If not, I'll simply delete the directory, but if it makes sense, it
should be synched with the emacs version.


[...]
>> `- tests  -- all test suites, organized per-package/subsystem
>> -- (e.g., language parsing/completion, ede, ...)

> There are two batches of tests.  Unit tests (close to the source) and 
> integration tests (the cit-* stuff) which tests the packages from a 
> pretty high level.  There are many tests in the same file as the sources 
> it tests.  Those may be a challenge for you to find. ;)

Like the 'modes' directory, I'll leave this for now.


Lluis
-- 
 "And it's much the same thing with knowledge, for whenever you learn
 something new, the whole world becomes that much richer."
 -- The Princess of Pure Reason, as told by Norton Juster in The Phantom
 Tollbooth

Hi Lluis,

Let me see if I can summarize the strategy from the bits and pieces so far:

1) Match CEDET to Emacs as exactly as possible, pulling in those bits 
that Emacs left behind.

2) Cross-merge between Emacs and CEDET.  Supposedly Emacs will be 
updated to CEDET 1.0, and CEDET will gain any remaining changes 
incompatible with XEmacs or Emacs 22.

3) Devise an updated structure to accommodate an improved development 
model, such as a dir-per-language.  Propose to Emacs and work out 
details.  Implement in CEDET.  Cross-merge up to Emacs.

4) Adapt updated file hierarchy to build to use EDE, obsolete packages 
out of CEDET, design external-to-emacs packaging and install techniques. 
  (possibly using elpa?)

If this is what you are implying, then I think it's a good idea. :)

In this light, I'd like to recommend that 'contrib' be outside of 
'lisp', as those items can't be in Emacs proper, and should be sorted as 
such.

I also think that src (the .wy and .by files) are derived Lisp files the 
way a yacc file has C or C++ in it, and should be near the support .el 
file, ie : c.by + semantic-c.el go together.

Thanks for all  your work on this!

Eric

On 09/01/2010 11:25 AM, Lluís wrote:
> Eric M Ludlam writes:
>
> [...]
>>>> While trying to check in the above, one of the things I noticed is that
>>>> bzr recognizes that semantic/semantic-grammar-wy.el is modified, due to
>>>> the build.
>>>
>>> Aha... apart from files that should be ignored (e.g., .elc), the file you
>>> mention is modified during build. This can be avoided in two ways:
>>> - re-generating parser code only when parser definition is modified (this is a
>>> simple matter of makefile dependencies)
>
>> This is the default, but when you check stuff out the timestamps don't
>> always follow nicely.
>
> I see. It seems to have been reported already, and there is no definitive
> solution to that:
>    https://bugs.launchpad.net/bzr/+bug/245170
>
>
>>> - avoid the modifications inserted by the re-generation step:
>>> - changes in spacing (simply re-commit with new spacing, and no new changes
>>> should arise)
>>> - "History" section is deleted from the .el file; I believe that's because
>>> there's no History section on the file?
>>> - X-RCS field is "resetted"; I've deleted all those fields from my local
>>> branch, as I think they're no longer necessary
>>> - "Author" and "Created" fields are set during build, but I think they should
>>> be maintained from the parser definition
>
>> These seem like good ideas.  CVS somehow knew that nothing was changing,
>> whereas bazaar detects many changes.  Does bazaar strip whitespace in
>> some fancy way?
>
> Not that I'm aware of. Is it possible that the current generator code is
> producing files with different spacing?
>
>
>>> Now, if you look into the results after a "make clean-all", you'll see that,
>>> additionally, "semantic/bovine/semantic-skeleton-by.el" is deleted, but was also
>>> present on the repository. I think the compiled skeleton parser should not be
>>> present on the repository, or else not deleted during a clean action.
>
>> This I do not recall.  It doesn't seem like it is necessary to me.
>
> Well, I thought the compiled skeleton was there just let others quickly see how
> a compiled parser looks like. But if you don't see a reason for it to be there,
> I'll simply delete it.
>
>
>>> This brings me to the point that on my local branch, I cannot seem to regenerate
>>> the Makefile files from their Project.ede definitions. Right now I'm rewriting
>>> the make infrastructure to work with plain Makfiles, as right now I'm heavily
>>> reorganizing the location of files.
>
>> The old Project.ede files are not going to work in this new world.  A
>> new project, likely for all of CEDET should probably be built, and the
>> old Project files abandoned.
>
>> Basically, delete all the old Project.ede files, and issue the command
>> "M-x ede-new" under CEDET to create a Makefile project.  Then the same
>> for every subdir, and add targets and files to targets as you go.
>
>> It has always been useful using EDE to manage CEDET to make sure that it
>> is always well tested. :)
>
> Aaahhh... now it makes sense :)
>
> Well, I'll start with makefiles from scratch, which I'm more comfortable with
> right now, and a rewriting with EDE will follow once CEDET matches the interface
> in current emacs (all the simple global-enabling functions and the like).
>
> This reminds me of something else. There are a few lisp files on the root
> directory, are all of them still alive? I'm specially talking about
> cedet-build.el, as then I should rewrite it to follow the new directory
> structure.
>
>
>>> The layout I want to have at the end is:
>>>
>>> cedet
>>> |-<various top-level files>
>>> |- src    -- bovine/wisent parser definitions (maybe it should be called parsers)
>
>> By this you mean the .by and .wy files?
>
> Yes. In part because emacs is only shipping the compiled parsers, not the
> sources/definitions.
>
>
>> There was a request via the email list to group code for a particular
>> mode together.  Perhaps like:
>
>>     |- modes
>>     |   |- c (home of c.by, semantic-c.el, semanticdb-cscope.el)
>>     |   |- java (home of java.wy, java-tags.wy, ...)
>
>> Of course, then does SRecode templates for C or Java go in those dirs
>> too?  I would guess this would be the way to get everything
>> "intergrated" together more tightly, helping identify code for a
>> particular language all in one place.
>
> Right, this should ease language-specific development, but some pieces are
> shared among different languages (e.g., cscope support mutiple languages).
>
>
>> This would probably need to be run by the Emacs maintainers, since we
>> are also trying to get the two trees to match.
>
> Well, first I'll try with the current reorganization, which is still big enough
> to take some time :)
>
> We can talk with the emacs people about this after the next cross-merge.
>
>
> [...]
>>> |   `- speedbar -- is speedbar going to be developed by CEDET, or has it moved
>>> |               -- altogether to emacs? the code in emacs is not exactly the
>>> |               -- same as the one in cedet, but haven't really checked on the
>>> |               -- differences
>
>> Speedbar in CEDET is different only in that I've kept in code that
>> allows it to run in ancient versions of Emacs.  I have not needed to
>> enhance Speedbar in a very long time.
>
> Well, my question is if it makes sense to maintain it inside cedet, just like
> eieio. If not, I'll simply delete the directory, but if it makes sense, it
> should be synched with the emacs version.
>
>
> [...]
>>> `- tests  -- all test suites, organized per-package/subsystem
>>> -- (e.g., language parsing/completion, ede, ...)
>
>> There are two batches of tests.  Unit tests (close to the source) and
>> integration tests (the cit-* stuff) which tests the packages from a
>> pretty high level.  There are many tests in the same file as the sources
>> it tests.  Those may be a challenge for you to find. ;)
>
> Like the 'modes' directory, I'll leave this for now.
>
>
> Lluis

Eric M Ludlam writes:
> 1) Match CEDET to Emacs as exactly as possible, pulling in those bits 
> that Emacs left behind.

> 2) Cross-merge between Emacs and CEDET.  Supposedly Emacs will be 
> updated to CEDET 1.0, and CEDET will gain any remaining changes 
> incompatible with XEmacs or Emacs 22.

> 3) Devise an updated structure to accommodate an improved development 
> model, such as a dir-per-language.  Propose to Emacs and work out 
> details.  Implement in CEDET.  Cross-merge up to Emacs.

That's it!


> 4) Adapt updated file hierarchy to build to use EDE, obsolete packages 
> out of CEDET, design external-to-emacs packaging and install techniques. 
>   (possibly using elpa?)

Elpa should be a nice way to do it, specially now that next emacs will integrate
an improved version of elpa, with support for multiple package repositories.


[...]
> In this light, I'd like to recommend that 'contrib' be outside of 
> 'lisp', as those items can't be in Emacs proper, and should be sorted as 
> such.

You're right. I'll leave it at './lisp/contrib' (where it is now).


> I also think that src (the .wy and .by files) are derived Lisp files the 
> way a yacc file has C or C++ in it, and should be near the support .el 
> file, ie : c.by + semantic-c.el go together.

Makes sense. Besides, I still haven't done this specific file movement.


BTW, what about the 'cedet-build.el' I told you about? Is it obsolete?


> Thanks for all  your work on this!

Well, thank you for cedet. In fact I want to finish this so I'll be able to use
an up-to-date cedet without having to completely modify my emacs config :)


Lluis

-- 
 "And it's much the same thing with knowledge, for whenever you learn
 something new, the whole world becomes that much richer."
 -- The Princess of Pure Reason, as told by Norton Juster in The Phantom
 Tollbooth

On 09/01/2010 02:49 PM, Lluís wrote:
> [...]
>> >  In this light, I'd like to recommend that 'contrib' be outside of
>> >  'lisp', as those items can't be in Emacs proper, and should be sorted as
>> >  such.
> You're right. I'll leave it at './lisp/contrib' (where it is now).

What?  I think your first and second sentences contradict.  I was trying 
to imply that contrib should not be in the lisp directory, such as in 
./contrib instead of ./lisp/contrib.


>> >  I also think that src (the .wy and .by files) are derived Lisp files the
>> >  way a yacc file has C or C++ in it, and should be near the support .el
>> >  file, ie : c.by + semantic-c.el go together.
> Makes sense. Besides, I still haven't done this specific file movement.
>
>
> BTW, what about the 'cedet-build.el' I told you about? Is it obsolete?

cedet-build will be useful when the EDE projects start working again. 
It allows windows users to build Emacs without having Make installed. 
It will need to be updated, but the essence of it (load order, and 
telling EDE to run compiles) will be the same.

Eric

Eric M Ludlam writes:

> On 09/01/2010 02:49 PM, Lluís wrote:
>> [...]
>>> >  In this light, I'd like to recommend that 'contrib' be outside of
>>> >  'lisp', as those items can't be in Emacs proper, and should be sorted as
>>> >  such.
>> You're right. I'll leave it at './lisp/contrib' (where it is now).

> What?  I think your first and second sentences contradict.  I was trying 
> to imply that contrib should not be in the lisp directory, such as in 
> ./contrib instead of ./lisp/contrib.

Ups. I misread it as saying that it should be out of the ./lisp/cedet directory.


>> BTW, what about the 'cedet-build.el' I told you about? Is it obsolete?

> cedet-build will be useful when the EDE projects start working again. 
> It allows windows users to build Emacs without having Make installed. 
> It will need to be updated, but the essence of it (load order, and 
> telling EDE to run compiles) will be the same.

Ok, then I won't delete it.


Lluis

-- 
 "And it's much the same thing with knowledge, for whenever you learn
 something new, the whole world becomes that much richer."
 -- The Princess of Pure Reason, as told by Norton Juster in The Phantom
 Tollbooth

On 08/31/2010 09:42 AM, Lluís wrote:
>> >  Following the instructions in your document I tried the
>> >  merge from the mainline into my branch.  This ought to be a no-op.
>> >  Unfortunately, it won't allow the merge to run unless I also commit the
>> >  grammar-wy change.  Does anyone know how to work around this (in terms
>> >  of process.)
> Ok, so now for your question:)
>
> You cannot merge if you have local non-committed changes, so you have 3 ways out
> of here (to be executed before merging from the mainline):
>
> - bzr revert; will loose all non-committed changes (makes sense in this case)
> - bzr commit; has no real sense to commit those changes on the compiled grammar
> - bzr shelve; I didn't document that one, but puts local changes aside, then you
>                can merge and 'bzr unshelve' later on (see 'bzr help shelve')
>
> Hope this helps.


Thanks for the help above.  Those commands along with improving the 
.bzrignore file got me to a point where I was merged, committed, and 
ready to push.

I then got this error, probably because of an add I messed up:

projectile:trunk> bzr push 
bzr+ssh://zappo@...>
bzr: ERROR: Server sent an unexpected error: ('error', 'Operation denied 
because it would change the main history, which is not permitted by the 
append_revisions_only setting on branch 
"chroot-47128132513360:///bzrroot/cedet/code/trunk/".')


I mv'd the branch out of the way, got a checkout of trunk, and was able 
to start committing stuff directly the way I used to do it for CVS. 
This is good, as I'm finally able to get those one file patches I'd been 
posting into the repository.

I will try the branchy way again, probably for the tag-similar-p thing I 
was discussing earlier.

Thanks
Eric

Hello Eric

You could try the DVC package, that provides unified interface for
different distributed version control systems, including bzr

P.S. I have an article about it
http://alexott.net/en/writings/emacs-vcs/EmacsDVC.html

Eric M. Ludlam  at "Thu, 23 Sep 2010 21:02:51 -0400" wrote:
 EML> On 08/31/2010 09:42 AM, Lluís wrote:
 >>> >  Following the instructions in your document I tried the
 >>> >  merge from the mainline into my branch.  This ought to be a no-op.
 >>> >  Unfortunately, it won't allow the merge to run unless I also commit the
 >>> >  grammar-wy change.  Does anyone know how to work around this (in terms
 >>> >  of process.)
 >> Ok, so now for your question:)
 >>
 >> You cannot merge if you have local non-committed changes, so you have 3 ways out
 >> of here (to be executed before merging from the mainline):
 >>
 >> - bzr revert; will loose all non-committed changes (makes sense in this case)
 >> - bzr commit; has no real sense to commit those changes on the compiled grammar
 >> - bzr shelve; I didn't document that one, but puts local changes aside, then you
 >>                can merge and 'bzr unshelve' later on (see 'bzr help shelve')
 >>
 >> Hope this helps.


 EML> Thanks for the help above.  Those commands along with improving the 
 EML> .bzrignore file got me to a point where I was merged, committed, and 
 EML> ready to push.

 EML> I then got this error, probably because of an add I messed up:

 EML> projectile:trunk> bzr push 
 EML> bzr+ssh://zappo@...>
 EML> bzr: ERROR: Server sent an unexpected error: ('error', 'Operation denied 
 EML> because it would change the main history, which is not permitted by the 
 EML> append_revisions_only setting on branch 
 EML> "chroot-47128132513360:///bzrroot/cedet/code/trunk/".')


 EML> I mv'd the branch out of the way, got a checkout of trunk, and was able 
 EML> to start committing stuff directly the way I used to do it for CVS. 
 EML> This is good, as I'm finally able to get those one file patches I'd been 
 EML> posting into the repository.

 EML> I will try the branchy way again, probably for the tag-similar-p thing I 
 EML> was discussing earlier.


-- 
With best wishes, Alex Ott, MBA
http://alexott.blogspot.com/           http://alexott.net
http://alexott-ru.blogspot.com/

On 09/24/2010 02:28 AM, Alex Ott wrote:
> Hello Eric
>
> You could try the DVC package, that provides unified interface for
> different distributed version control systems, including bzr
>
> P.S. I have an article about it
> http://alexott.net/en/writings/emacs-vcs/EmacsDVC.html

One of the comments says:

#dvc sucks all the fun out of both #Emacs and #Mercurial.

I don't know if I could handle Emacs not being fun.  ;)


I'll give it a try next time I start hacking.

Eric

On 2010-09-24 12:43 +0100, Eric M. Ludlam wrote:
> One of the comments says:
>
> #dvc sucks all the fun out of both #Emacs and #Mercurial.
>
> I don't know if I could handle Emacs not being fun.

With magit.el (for git), it is fun doing version control. I like the
feature that I can navigate to a chunk in diff, hitting s then to
another hitting s and commit only those chunks that I have selected. If
that granularity is not enough, I can use a region instead.

Leo

Leo  at "Fri, 24 Sep 2010 12:56:48 +0100" wrote:
 L> On 2010-09-24 12:43 +0100, Eric M. Ludlam wrote:
 >> One of the comments says:
 >>
 >> #dvc sucks all the fun out of both #Emacs and #Mercurial.
 >>
 >> I don't know if I could handle Emacs not being fun.

 L> With magit.el (for git), it is fun doing version control. I like the
 L> feature that I can navigate to a chunk in diff, hitting s then to
 L> another hitting s and commit only those chunks that I have selected. If
 L> that granularity is not enough, I can use a region instead.

Yep - magit is very good package - I always use it for my work. But dvc
allows me to not learn new shortcuts, when I'm switching to bzr/hg/darcs ;-)

-- 
With best wishes, Alex Ott, MBA
http://alexott.blogspot.com/           http://alexott.net
http://alexott-ru.blogspot.com/

Eric M Ludlam writes:

> Thanks for the help above.  Those commands along with improving the .bzrignore
> file got me to a point where I was merged, committed, and ready to push.

> I then got this error, probably because of an add I messed up:

> projectile:trunk> bzr push
> bzr+ssh://zappo@...>
> bzr: ERROR: Server sent an unexpected error: ('error', 'Operation denied because
> it would change the main history, which is not permitted by the
> append_revisions_only setting on branch
> "chroot-47128132513360:///bzrroot/cedet/code/trunk/".')


> I mv'd the branch out of the way, got a checkout of trunk, and was able to start
> committing stuff directly the way I used to do it for CVS. This is good, as I'm
> finally able to get those one file patches I'd been posting into the repository.

> I will try the branchy way again, probably for the tag-similar-p thing I was
> discussing earlier.

* Short answer

  This is probably due to the fact that someone pushed new revisions
  into mainline trunk between your "bzr commit" and "bzr push" commands,
  just after the merge.

* How to prevent it in the future

  I'have updated the Development.org file (attached) with a slightly
  modified workflow that will prevent such problems.

  For those of you who aleady have a 'trunk' branch and do not want to
  get a fresh copy, go inside the 'trunk' directory and run this:
      bzr bind bzr+ssh://USERNAME@...>

  This will transform your 'trunk' "branch" into a "checkout". The
  difference is that you must use "update" instead of "pull", and
  "commit" automatically performs a "push". The idea of a checkout is
  that it provides a similar workflow to that that of traditional
  centralized version control systems (see =bzr help branches= and =bzr
  help checkouts=).

* Longer answer

  Mirroring the configuration in the Emacs repository, I activated the
  'append_revisions_only' option on the sourceforge server.

  In bazaar, revisions are numbered sequentially (revno; one of the
  reasons is friendliness), but development can be performed off-line
  (I'll see changes only when others push). Thus, some combinations of
  operations can produce the effect of re-numbering commits.

  The aforementioned configuration option does not allow pushes to the
  server that re-number the history; aka new revisions can only be added
  on the top of the history.

  Some have been discussing on the bazaar list to activate this option
  by default, as it makes sense whenever developers reference certain
  commit revisions on commit messages or mailing lists.

Lluís  writes:
>   I'have updated the Development.org file (attached) with a slightly
>   modified workflow that will prevent such problems.

Sorry, I messed up with the attachment. Here it is again.

On 09/26/2010 06:14 PM, Lluís wrote:
> Lluís  writes:
>>    I'have updated the Development.org file (attached) with a slightly
>>    modified workflow that will prevent such problems.
>
> Sorry, I messed up with the attachment. Here it is again.

Hi Lluis,

Could you add your Develop.org into bzr.  In CVS, I used to have a 
PRERELEASE_CHECKLIST, and USING_CEDET_FROM_CVS with big bold names for 
people browsing the sources directly.

I think it would be helpful to do something similar with Develop.org. 
It will also make it easier for me to find when I need it. :)

Thanks!
Eric

Eric M Ludlam writes:

> Could you add your Develop.org into bzr.  In CVS, I used to have a 
> PRERELEASE_CHECKLIST, and USING_CEDET_FROM_CVS with big bold names for 
> people browsing the sources directly.

Hey! Sorry for the delay, but I'm very busy lately. I've just added the
file into "trunk".

Lluis

-- 
 "And it's much the same thing with knowledge, for whenever you learn
 something new, the whole world becomes that much richer."
 -- The Princess of Pure Reason, as told by Norton Juster in The Phantom
 Tollbooth

On 10/04/2010 04:59 PM, Lluís wrote:
> Eric M Ludlam writes:
>
>> Could you add your Develop.org into bzr.  In CVS, I used to have a
>> PRERELEASE_CHECKLIST, and USING_CEDET_FROM_CVS with big bold names for
>> people browsing the sources directly.
>
> Hey! Sorry for the delay, but I'm very busy lately. I've just added the
> file into "trunk".

Thanks!  And no worries here.  It's catapult season, and I can't keep up 
with all the mail either. :)

When mid November rolls around, if you still haven't had time for the 
file renames, I can probably start helping with the build process.

Last weekend we finally got our catapult's rope bundle 'pretension' 
system working, and we got only one of our two rope bundles tightened 
up.  It took 9 hours, going into the night.   Now to do the second rope 
bundle.

Eric

-- 
http://www.siege-engine.com

Eric M Ludlam writes:

> When mid November rolls around, if you still haven't had time for the 
> file renames, I can probably start helping with the build process.

If I were to work intensely, I think getting it done (except for fixing
bugs I might introduce) shouldn't take more than two weekends, so I hope
to have it done by the end of the month.


> Last weekend we finally got our catapult's rope bundle 'pretension' 
> system working, and we got only one of our two rope bundles tightened 
> up.  It took 9 hours, going into the night.   Now to do the second rope 
> bundle.

That really sounds like a great deal of fun :)

Lluis

-- 
 "And it's much the same thing with knowledge, for whenever you learn
 something new, the whole world becomes that much richer."
 -- The Princess of Pure Reason, as told by Norton Juster in The Phantom
 Tollbooth

Lluís writes:
> Eric M Ludlam writes:
>
> [...]
>>>> While trying to check in the above, one of the things I noticed is that
>>>> bzr recognizes that semantic/semantic-grammar-wy.el is modified, due to
>>>> the build.
>>> 
>>> Aha... apart from files that should be ignored (e.g., .elc), the file you
>>> mention is modified during build. This can be avoided in two ways:
>>> - re-generating parser code only when parser definition is modified (this is a
>>> simple matter of makefile dependencies)
>
>> This is the default, but when you check stuff out the timestamps don't 
>> always follow nicely.
>
> I see. It seems to have been reported already, and there is no definitive
> solution to that:
>   https://bugs.launchpad.net/bzr/+bug/245170

Well, the timestamps will usually be identical. Since you rebuild the
grammar *unless* the el-file is newer than the wy-file, a rebuild of the
wy-file will happen when the timestamps match.

So I'd suggest to apply this patch:

=== modified file 'semantic/semantic-grammar.el'
--- semantic/semantic-grammar.el	2010-08-19 23:28:10 +0000
+++ semantic/semantic-grammar.el	2010-10-08 14:12:19 +0000
@@ -828,9 +828,9 @@
          )
     (if (and (not force)
              (not (buffer-modified-p))
-             (file-newer-than-file-p
-              (buffer-file-name semantic--grammar-output-buffer)
-              (buffer-file-name semantic--grammar-input-buffer)))
+             (not (file-newer-than-file-p
+		   (buffer-file-name semantic--grammar-input-buffer)
+		   (buffer-file-name semantic--grammar-output-buffer))))
         (message "Package `%s' is up to date." package)
       ;; Create the package
       (set-buffer semantic--grammar-output-buffer)



Alternatively, if using timestamps is just not possible to do reliably
with bzr, we could rename the file to semantic-grammar-wy-default.el or
something like that, and use

(unless (require 'semantic-grammar-wy nil t)
  (require 'semantic-grammar-wy-default))

in semantic-grammar.el. This way, a generated semantic-grammar-wy.el
wouldn't be under version control.

>> These seem like good ideas.  CVS somehow knew that nothing was changing, 
>> whereas bazaar detects many changes.  Does bazaar strip whitespace in 
>> some fancy way?
>
> Not that I'm aware of. Is it possible that the current generator code is
> producing files with different spacing?

I see the following differences in semantic-grammar-wy.el:

- Changed Author/Created
- History section is included
- Tabs are used instead of spaces

so bzr is rightfully detecting it as changed.

-David

On 10/08/2010 11:53 AM, David Engster wrote:
> Lluís writes:
>> Eric M Ludlam writes:
>>
>> [...]
>>>>> While trying to check in the above, one of the things I noticed is that
>>>>> bzr recognizes that semantic/semantic-grammar-wy.el is modified, due to
>>>>> the build.
>>>>
>>>> Aha... apart from files that should be ignored (e.g., .elc), the file you
>>>> mention is modified during build. This can be avoided in two ways:
>>>> - re-generating parser code only when parser definition is modified (this is a
>>>> simple matter of makefile dependencies)
>>
>>> This is the default, but when you check stuff out the timestamps don't
>>> always follow nicely.
>>
>> I see. It seems to have been reported already, and there is no definitive
>> solution to that:
>>    https://bugs.launchpad.net/bzr/+bug/245170
>
> Well, the timestamps will usually be identical. Since you rebuild the
> grammar *unless* the el-file is newer than the wy-file, a rebuild of the
> wy-file will happen when the timestamps match.
>
> So I'd suggest to apply this patch:
>
> === modified file 'semantic/semantic-grammar.el'
> --- semantic/semantic-grammar.el	2010-08-19 23:28:10 +0000
> +++ semantic/semantic-grammar.el	2010-10-08 14:12:19 +0000
> @@ -828,9 +828,9 @@
>            )
>       (if (and (not force)
>                (not (buffer-modified-p))
> -             (file-newer-than-file-p
> -              (buffer-file-name semantic--grammar-output-buffer)
> -              (buffer-file-name semantic--grammar-input-buffer)))
> +             (not (file-newer-than-file-p
> +		   (buffer-file-name semantic--grammar-input-buffer)
> +		   (buffer-file-name semantic--grammar-output-buffer))))
>           (message "Package `%s' is up to date." package)
>         ;; Create the package
>         (set-buffer semantic--grammar-output-buffer)

That seems like a good idea.  Thanks.


> Alternatively, if using timestamps is just not possible to do reliably
> with bzr, we could rename the file to semantic-grammar-wy-default.el or
> something like that, and use
>
> (unless (require 'semantic-grammar-wy nil t)
>    (require 'semantic-grammar-wy-default))
>
> in semantic-grammar.el. This way, a generated semantic-grammar-wy.el
> wouldn't be under version control.

That looks like a good way to do it in Lluis's updated variant of the 
build system.  If that is a long ways from being completed, we can look 
into doing that in the current build variant.

>>> These seem like good ideas.  CVS somehow knew that nothing was changing,
>>> whereas bazaar detects many changes.  Does bazaar strip whitespace in
>>> some fancy way?
>>
>> Not that I'm aware of. Is it possible that the current generator code is
>> producing files with different spacing?
>
> I see the following differences in semantic-grammar-wy.el:
>
> - Changed Author/Created
> - History section is included
> - Tabs are used instead of spaces
>
> so bzr is rightfully detecting it as changed.

This seems like another good argument for using a -default variant.

Eric