CEDET

Hi,

The packaging task seems pretty useful, but I haven't had time to look 
into it much.

As for the cedet complexities, there is a project under the bzr file 
rename branch to get the CEDET file names and init to more closely match 
what is in Emacs 23/24.  I would assume that packaging from that set of 
files will be easier since autoloads and generated files may more 
closely match what is expected for Emacs.

Perhaps David or Lluis will have ideas about that.

Eric

On 06/11/2011 01:46 PM, Richard Kim wrote:
> I would like to share my experiments with emacs 24's new packaging
> system as it relates to cedet and ecb.  I would appreciate any
> feedback.
>
> At the risk of being redundant let me briefly summarize what the
> packaging system is all about in case it is new to some of you.  In
> short it is a way to create packages (not unlike Debian packages)
> consisting of one or more emacs-lisp files, documentation, etc. for
> easy installation and upgrading.  http://elpa.gnu.org/packages/ is the
> official repository.  I think the official goal is to update this
> repository so that one can get more frequent upgrades of standard
> elisp packages without having to wait for releases of emacs itself.
> The installation of a package involves untar'ing, creation of an
> autoload file per package, and byte compiling.  It appears that xemacs
> had its own packaging system for a long time, but I have no experience
> with it.
>
> My interest with the packaging is primarily in deploying and upgrading
> of third-party and my own elisp packages for sharing with my
> colleagues at work.  For example some of the files that can be found
> in my own ELPA (Emacs Lisp Package Archive) repository at
> ~/public_html/elpa are:
>
>      anything-20110410.tar	consists of a few files from emacswiki
>      bbdb-2.35.tar
>      cedet-trunk-20110530.tar	latest bzr files
>      doxymacs-1.8.0.tar
>      dvc-20101228.tar
>      ecb-20110527.tar		latest cvs files
>      elpa-utils-20110522.tar	tools I wrote to help create and manage elpa
>      emacs-w3m-20101228.tar
>      icicles-20110410.tar	consists of a few files form emacswiki
>      kimr-utils-20110410.tar	my own elisp files
>      mmm-mode-0.4.8.tar
>      python-mode-5.2.0.tar	from python.org
>      python-utils-20110410.tar	misc collection of python utilities
>
> where all are packages that I created by downloading (via bzr, svn,
> cvs or just tarballs) from Internet except elpa-utils and kimr-utils
> which are my own creations.  cedet-trunk package was created using the
> latest bzr code.  ecb was created using the latest cvs code.
>
> These packages are deployed for the latest emacs-23 (i.e.,
> http://bzr.savannah.gnu.org/r/emacs/emacs-23/) as well as latest
> emacs-24 (i.e., http://bzr.savannah.gnu.org/r/emacs/trunk/) which I
> compiled from the source code.  I would like to support xemacs as
> well, but I simply do not have the expertise or energy to do the work
> myself without help from others, but I am willing to help if someone
> can lend me a hand.
>
> The packaging system is part of only emacs-24; it is not part of
> emacs-23.  However it was easy to back port it by simply lifting these
> files from emacs-24 and installing them for emacs-23: help-mode.el,
> package.el, tabulated-list.el.
>
> Creating and installing cedet and ecb packages required some work.
> For cedet, I got things to work by advising package-unpack as detailed
> below. This is defined in package.el.  Aside from this, it was pretty
> straight forward.  For ecb, I think all that I needed to do was to add
> (setq ecb-version-check nil).
>
> Now if I want to deploy the latest version of cedet from bzr, then it
> is really easy to create the package and install it for both emacs-23
> and emacs-24, so that everyone at the site can enjoy it.
>
> The package-unpack advice is shown below which I wrote a hack.
> It would be nice if there was a cleaner way to create and install
> cedet package.  I welcome any ideas on this or any other issues.
>
> (defadvice package-unpack
>    (around customize-cedet-autoloads activate compile)
>    "The default behavior of this function is to untar the tarball, generate
> autoloads, then byte compile all *.el files in the top directory.
>
> Cedet installation is much more complex than that.  So we have to handle it in
> custom manner, i.e., call `cedet-build-in-default-emacs'."
>    (if (not (string= (symbol-name (ad-get-arg 0)) "cedet-trunk"))
>        ;; Use the default method for all packages except a package named
>        ;; "cedet-trunk" which is the name I use for latest cedet code on bzr
>        ;; trunk branch.
>        ad-do-it
>      (let* ((name    (ad-get-arg 0)) ;; "cedet-trunk"
>             (version (ad-get-arg 1)) ;; e.g., "20110501"
>             (dirname (concat (symbol-name name) "-" version))
>             (pkg-dir (expand-file-name dirname package-user-dir)))
>        (make-directory package-user-dir t)
>        (let* ((default-directory (file-name-as-directory package-user-dir)))
>          ;; Remove directory to prevent possible issues caused to old files.
>          ;; I think this should be done within default package-unpack.  However
>          ;; Stefan was understandably hesitant to make that the default behvior
>          ;; since it is possible to remove files that you really did not want to!
>          (when (and (file-directory-p pkg-dir)
>                     (y-or-n-p (format "Delete %s before installing a new one?" pkg-dir)))
>            (message "Deleting %s ..." pkg-dir)
>            (delete-directory pkg-dir 'recursive))
>          (package-untar-buffer dirname)
>          ;; tarball has been unpacked.
>          (let* ((auto-name (format "%s-autoloads.el" name))
>                 (generated-autoload-file (expand-file-name auto-name pkg-dir))
>                 (cedet-init-file (expand-file-name "common/cedet.el" pkg-dir))
>                 (cedet-build-file (expand-file-name "cedet-build.el" pkg-dir)))
>
>            ;; Manually create autoloads file named cedet-trunk-autoloads.el and
>            ;; add these two lines:
>            ;;     (load ".../common/cedet.el")
>            ;;     (provide 'cedet-autoloads)
>            (set-buffer (find-file-noselect generated-autoload-file))
>            (setq buffer-read-only nil)
>            (erase-buffer)
>            (insert (format "(load \"%s\")\n" cedet-init-file))
>            (insert "(provide 'cedet-autoloads)\n")
>            (save-buffer)
>
>            ;; Now run the official cedet build function
>            (load cedet-build-file)
>            (cedet-build-in-default-emacs)
>            )))))
>
> ------------------------------------------------------------------------------
> EditLive Enterprise is the world's most technically advanced content
> authoring tool. Experience the power of Track Changes, Inline Image
> Editing and ensure content is compliant with Accessibility Checking.
> http://p.sf.net/sfu/ephox-dev2dev
> _______________________________________________
> Cedet-devel mailing list
> Cedet-devel@...
> https://lists.sourceforge.net/lists/listinfo/cedet-devel
>

Eric M. Ludlam writes:
> As for the cedet complexities, there is a project under the bzr file 
> rename branch to get the CEDET file names and init to more closely match 
> what is in Emacs 23/24.  I would assume that packaging from that set of 
> files will be easier since autoloads and generated files may more 
> closely match what is expected for Emacs.
>
> Perhaps David or Lluis will have ideas about that.

Yes, I figure it would be easier to package that branch, but it isn't
usable at the moment.

> On 06/11/2011 01:46 PM, Richard Kim wrote:
>> It appears that xemacs had its own packaging system for a long time,
>> but I have no experience with it.

CEDET doesn't currently compile under XEmacs.

>> Aside from this, it was pretty straight forward.  For ecb, I think
>> all that I needed to do was to add (setq ecb-version-check nil).

Now that is interesting to hear. So you can confirm that ECB does work
with CEDET-bzr if you disable version checks? This has become something
like a FAQ recently, and ECB development seems to have stalled.

>> The package-unpack advice is shown below which I wrote a hack.
>> It would be nice if there was a cleaner way to create and install
>> cedet package.  I welcome any ideas on this or any other issues.

I haven't had time to look at ELPA so far. I agree that it would be neat
to have CEDET-bzr packaged like that. Unfortunately, the CEDET build
process is pretty involved because of all the grammar generation and
stuff. The version that ships with Emacs is cheating a bit because it
doesn't generate the grammars during build, but the maintainers just
generate them once and then check them in. We could do something similar
for ELPA, but at the moment, I'm afraid I don't have time to really deal
with this.

-David

Hi David,

I get cedet email in digest form, so please excuse my irregular quotations.

> Date: Tue, 14 Jun 2011 22:58:09 +0200
> From: David Engster <deng@...>
> Subject: Re: [CEDET-devel] packaging cedet and ecb
> To: cedet-devel@...
> Message-ID: <87oc206sgu.fsf@...>
> Content-Type: text/plain
> 
> >> Aside from this, it was pretty straight forward.  For ecb, I think
> >> all that I needed to do was to add (setq ecb-version-check nil).
> 
> Now that is interesting to hear. So you can confirm that ECB does work
> with CEDET-bzr if you disable version checks? This has become something
> like a FAQ recently, and ECB development seems to have stalled.

Yes ECB works with version check disabled, i.e., `M-x ecb-activate'
starts up fine.  I have not done much testing beyond that other than to
open a few files, so if there are other problems, I would not know.

I used these versions

- latest emacs-23 bzr branch code as of today 2011/06/25 compiled on
  latest 64-bit ubuntu linux.

- latest trunk bzr branch code (i.e., emacs 24) as of today 2011/06/25
  compiled on latest 64-bit ubuntu.

- latest cedet bzr trunk code as of today

- latest ecb cvs trunk code as of today

I was able to create ELPA package for both cedet and ecb, install them
to both emacs-23 and emacs-24, then have ecb-activate work fine for
both.  Aside from (setq ecb-version-check nil), I removed
ecb-advice-test.el file from ecb, because it got in the way of ELPA
package creation and/or installation.

Although some of the ELPA related code that I wrote is not specific to
cedet or ecb, perhaps it might be helpful if I check in the code
somewhere under cedet/contrib directory.  That way I can explain exactly
what steps are necessary to create and install ELPA packages for cedet
and ecb for both emacs23 and emacs24.  I have at least three files in
mind:

    elpa.el has code to help create and otherwise manage own's own ELPA
    packages and repository.

    elpa-pkg-cedet-trunk.el is cedet package specific code.

    elpa-pkg-ecb.el is ecb package specific code.

    elpa-pkg-*.el are other package specific files which I can check in
    as examples on how to create ELPA packages for other packages.

    Also my own setup code in ~/.emacs to make use of all these.

If that sounds like a good idea I can either check in these files under
cedet/contrib/elpa-utils (or wherever appropriate) or email a tarball to
this list.

> >> The package-unpack advice is shown below which I wrote a hack.
> >> It would be nice if there was a cleaner way to create and install
> >> cedet package.  I welcome any ideas on this or any other issues.
> 
> I haven't had time to look at ELPA so far. I agree that it would be neat
> to have CEDET-bzr packaged like that. Unfortunately, the CEDET build
> process is pretty involved because of all the grammar generation and
> stuff. The version that ships with Emacs is cheating a bit because it
> doesn't generate the grammars during build, but the maintainers just
> generate them once and then check them in. We could do something similar
> for ELPA, but at the moment, I'm afraid I don't have time to really deal
> with this.

Yes cedet installation is more involved than other packages that I
created.  However what I have now works although it relies on launching
separate emacs to run cedet-build-in-default-emacs.  It may look
confusing to the user, but it does work, so a better method to do all
this would be nice, but it is not blocking other work.