On Wed, Sep 29, 2010 at 12:41 PM, Sands Alden Fish <sands@mit.edu> wrote:
On Sep 27, 2010, at 5:42 PM, Mark Diggory wrote:


I think your overlooking that theres no need to even configure the "release repository", as its maven central. And this is why the current repo listing is only the snapshot repository. (Note eventually we want to attian "no" third party or snapshot repository entries in our POM so we can utilize the sonatype repository hosting for our releases.

But there does exist a release repository at http://maven.dspace.org/release.  Indeed, I was not aware that a typical build pulled release artifacts from Maven Central and didn't use our own release repository.  Does this suggest that if we deleted maven.dspace.org/release that it would have no impact?  The artifacts therein are not used for anything?  

maven.dspace.org/release is the point where releases are deployed to for rsync to the central repo, its contents are not used during any build process, it may as well be hidden from http viewing.

Particularly confusing to me is the fact that maven.dspace.org/release has apparently also had release builds deployed to it, right up to 1.6.2…

see above

Your logic is in error because those switches your referring to are will actually do the oposite of the goal you are seeking.  IE they will cause the snapshot repo to be included into the transitive analysis of release artifacts (something we surely do not want). 

I think you may have misunderstood my approach concerning the changes in repository declaration, as those changes explicitly turn off any potential access to the snapshot repository.  Perhaps you thought I meant to turn on the <releases> switch for the declared snapshot repository?  I was refering to declaring maven.dspace.org/release as its own <repository> entry and turning <releases> on for that, while disabling the snapshot repository that is already declared, blocking it's use in dependency resolution.

Based on the above clarification, no there is no reason to enable "releases" on maven.dspace.org/release or maven.dspace.org/snapshot

Again, the enabling of "snapshot" on maven.dspace.org/snapshot designates it a repository to look for "SNAPSHOT" releases in, it will not get releases from it, and only when the"dependency" defined in the maven pom includes "SNAPSHOT" will it check this repository.

The snapshot repository has snapshots enable on it because that is what it is for in the first place.  Your disabling it will disrupt the development process on the trunk, forcing us to use some sort of profile or other option to enable it.

I would like to understand this potential disruption better.  As I mentioned in my original email, I know that it wouldn't have a neutrual impact on trunk development, but see my comments concerning de-duplication of repository declarations and the use of the Parent POM to make changing repository configurations nearly effortless.  I would like to hear your or anyone's thoughts on this.  The repositories can be defined once in the Parent POM, and eliminated almost everywhere else.  I understand the use of the snapshot repository better now, but one question:  Does this suggest that no institution should build off of the maintenance branches to keep up with the latest bug fixes?

the snapshot repository configuration is present in all the poms because in the development trunk, in the absence of a dirct reference to the parent pom, the snapshot repo is tested for a remote copy. This allow you to check out individual projects or just the "dspace" project to perform development tests against.  It is a feature I designed into the configuration for the development environment.

Again, note those repository declarations only apply to trunk / SNAPSHOT building.  Don't use SNAPSHOT and you should not get such artifacts.

There are never release artifacts issued into the snapshot repository, only snapshots, if there are, we need to remove them (Ideally, we may want to consider that the snapshot repo should probably be purged of snapshot builds after every release).

Certainly release artifacts should never, and clearly aren't ever issued to the snapshot repository.  Again, here I am speaking of the maven.dspace.org _release_ repository.

The idea of purging snapshot builds after the release date seems like a very reasonable and clean one, and I wonder if there is a common practice in the greater Maven community with regard to this.  

I actually went back to just releasing "SNAPSHOT" marked builds rather than build versions.  I think this is probably the best way to keep things simple.  I also have enabled a Hudson build that will update the snapshot artifacts whenever there are changes in the svn trunk.  

A full release of DSpace does not look to resolve SNAPSHOTS because the "maven release process" removes them from the dependencies and forces you to choose full fledged releases.  In maven, a release will always be chosen over a snapshot when "snapshot" is not designated in the dependency version.

Please read:

Incorporating SNAPSHOT versions into the specification
Resolution of dependency ranges should not resolve to a snapshot (development version) unless it is included as an explicit boundary. There is no need to compile against development code unless you are explicitly using a new feature, under which the snapshot will become the lower bound of your version specification. As releases are considered newer than the snapshot they belong to, they will be chosen over an old snapshot if found.

This makes sense now that I know that we are releasing our build artifacts to Maven central.  I have never seen this documented anywhere.  Was this always the case or did this change at some point?

This has always been the case.

I would request that you produce a script or something that replicates the behavior that your describing because building a release version of dspace should never ever pickup snapshots in its build process.  If you are getting them, then something else has been done in your environment that is gathering the wrong artifacts or leaving behind stale builds (ie are you properly running "mvn clean package" or just "mvn package").

It is not my experience that what your suggesting will give you the holy grail you seek.  The more appropriate approach would require us to change our dependency management policy to force dependencies to always point at a specific version of the artifact by using "[ ]" around it, this would assure your build did not seek out a newer version of a dependency. We need to spend more time doing dependency analysis and fine tuning before each release.  Tim I would put a place holder where we discuss the maven dependency plugin and analysis at the next meeting.

Concerning the "roadmap" you make reference to for altering the Maven build process, it's management, and adjustments for of the use of external tools, I have not seen any public discussion or documentation about where this is headed.  Since I imagine use of and some at least minor customization of the Maven build process and dependent artifacts/repositories, as well as the chosen use of the Parent POM, perhaps these discussions should exist in the future.  

So far, your the only one outside of Tim and Graham whose actually show an interest.

I would also suggest, in line with recent suggestion that the community of developers, et al. benefit from clean and uncomplicated architectures, and in line with common Maven practices, that we keep as much as possible, unless there is a very clear benefit in doing so, to the common best practices usage of Maven and it's conventions.  This would make it much easier for everyone involved to keep up with and understand the implementation of the Maven build architecture that we employ for DSpace.  I imagine that as people begin to understand more completely the build system and its abilities in the local context, modifications of POMs, usage of local Maven repositories.

We are very "conventional" in the Maven sense.  I agree to keep it that way. The intention of using maven archetypes is quite within convention and the modifications I seek to make for managing release versioning will make things considerably easier to determine which build of artifacts you may be working with.

Lastly, I would like to suggest to anyone doing more than a `mvn clean package` that they educate themselves about a couple of Maven analytical plugins that are fairly straightforward and quite enlightening in their output.  These are:

  - mvn dependency:analyze
  - mvn dependency:tree

Especially the latter provides a quick and easy way to see exactly how dependencies are being resolved for any given Maven project, and can be run at the individual project level, or at the top level for an analysis of the entire project's dependency tree.  (Unfortunately, the output does not include the repository the artifacts are being fetched from, due to the transient nature of these sources (e.g. local downloads to a user's .m2 directory) but it provides a much clearer understanding of what is happening during the resolution process.  Also, a wipe of the build environment's .m2 directory will force, during this tool's output, resolution of each artifact, as if the project was really being built.  This can give a reasonable, albeit overly verbose, trace of resolution URLs.

Yes, we should start a wiki page documenting these practices.  Is this something you may want to start?


Mark R. Diggory
Head of U.S. Operations - @mire

http://www.atmire.com - Institutional Repository Solutions
http://www.togather.eu - Before getting together, get Tog@ther