Re: [Vesta-users] Newbie questions....

[Kenneth C. S. <ke...@xo...>]

> I'm going to prepare a magazine review about a few Open Source CMS
> (OpenCM,  Subversion, Katie, Arch, Aegis, Vesta, PRCS, etc...) and I
> have a few  questions about Vesta. For part of these questions I can
> easily find the  answers into the Vesta documentation _but_ I would
> rather like to hear from  you (Vesta developers) your opinion about
> these topics before laying down my  comparative table. Can any Vesta
> developer have a look at these questions and  send me his opinion,
> either privately or trough the Vesta list?

> Many thanks in advance for your help.

Thanks for the interest.  (Free publicity is always good!)  I'd love
to hear more about the article.

> 1) CVS Compatibility. Is it possible to import or use an existing CVS
> repository? How about exporting?

There is no built-in support for importing from or exporting to CVS.
However, the Vesta repository's filesystem interface should make
either of these pretty easy.

A "cvs checkout" into a Vesta working directory (followed by removing
the CVS meta-data) would be an effective way to import.  It would be
easy to write a script to do this across a number of tags (to import a
set of historical versions from a CVS repository) or periodically (to
keep a Vesta repository in sync with a CVS repository).

> 2) Disconnected Operations. Is it possible to checkout a project on a
> laptop  computer, fly to [your favorite desert island], come back after
> a few months  and merge your stuff into the main branch without
> destroing the repository  and without losing any data? (resorting to
> the developer for solving existing  conflicts is perfectly acceptable).

It is possible to perform a checkout to a disconnected system like a
laptop.  Vesta has replication and cross-installation facilities for
remote checkout/checkin.  While designed for collaboration between
geographically distributed groups, they work just as well with
disconnected systems like laptops.  I've done this occasionally, and
I'm aware of at least one other user that uses it this way.

As to what Greg had to say, depending on what type of checkout
operation you perform, you could wind up holding a reservation on the
next version in sequence, effectively locking part of the source tree
from new main-line checkins.  Other users would still be able to
branch or get scratch working copies in such a case.

> 3) Renaming. Is it possible to rename a file or a directory and still
> hold  all the information about its history?

What Greg said is essentially correct.  Vesta's unit of
checkout/checkin is a directory tree.  Files and directories can be
renamed between versions with no problem.

Renaming at a higher level normally doesn't make sense as it would
invalidate references from other configurations.

I realize this sounds a little confusing for non-Vesta users, so let
me give an example of the way naming works under Vesta.

Suppose you had a Vesta package named "program" whose current version
is 5 and which contains a file named "foo.c".  The location of the
package "program" in your Vesta hierarchy might be
"/vesta/example.com/project/program".  If you listed the contents of
that directory, you'd see something like this:

dr-xr-xr-x    1 dave     users         512 Nov 13  2001 1
dr-xr-xr-x    1 bob      users         512 Dec 30  2001 2
dr-xr-xr-x    1 alice    users         512 Mar 18 22:17 3
dr-xr-xr-x    1 bob      users         512 Mar 18 22:17 4
dr-xr-xr-x    1 bob      users         512 Jun 12 13:07 5
dr-xr-xr-x    1 dave     users         512 Jun 12 12:43 checkout
lr--r--r--    1 dave     users           1 Jun 12 17:53 latest -> 5

The path to version 5, which is itself a directory, would be
/vesta/example.com/project/program/5.  If you listed its contents, you
would find "foo.c" along with the other files in the package:

-r--r--r--    1 bob      users        1263 Nov 13  2001 build.ves
-r--r--r--    1 bob      users       18687 Jun 12 17:43 foo.c
-r--r--r--    1 bob      users        1670 Nov 13  2001 progs.ves

If you checked out this package, renamed "foo.c" to "bar.c", and
checked it back in, you would then find version 6 in the package:

dr-xr-xr-x    1 dave     users         512 Nov 13  2001 1
dr-xr-xr-x    1 bob      users         512 Dec 30  2001 2
dr-xr-xr-x    1 alice    users         512 Mar 18 22:17 3
dr-xr-xr-x    1 bob      users         512 Mar 18 22:17 4
dr-xr-xr-x    1 bob      users         512 Jun 12 13:07 5
dr-xr-xr-x    1 bob      users         512 Jun 12 13:07 6
dr-xr-xr-x    1 dave     users         512 Jun 12 12:43 checkout
lr--r--r--    1 dave     users           1 Jun 12 17:53 latest -> 6

And if you listed /vesta/example.com/project/program/6, you would find
your changes:

-r--r--r--    1 bob      users       18687 Jun 12 17:43 bar.c
-r--r--r--    1 bob      users        1263 Nov 13  2001 build.ves
-r--r--r--    1 bob      users        1670 Nov 13  2001 progs.ves

So within a versioned object, renaming is easy.

However, renaming /vesta/example.com/project/program to
/vesta/example.com/project/my_program would break references to
versions of this package from configurations.  Such references are
made by the path within the Vesta repository namespace,
e.g. /vesta/example.com/project/program/5 or
/vesta/example.com/project/program/6.

> 4) Atomicity. Sensitive operations are atomic? Can you be sure that
> nothing  will ever interfere with your commit?

Absolutely.  Atomic operations are really atomic.  They either happen
completely or not at all.  (Vesta has no problems like those in CVS's
sloppy concurrency model.  It's impossible for someone the check out
half of one version and half of another version.)

> 5) Access Control List at the Repository Level. Can you grant a new
> user the  access to the CMS without being forced to grant him access to
> the host  computer as well?

Yes.  As I mentioned earlier, Vesta has cross-installation
capabilities.  Users at a remote site can be granted access without
having accounts on the system running the Vesta repository.

> 6) User Authentication. Can you force the developer to use a
> public-key-based  authentication for accessing the repository?

Not at this time.  However, Vesta's authentication scheme was designed
for future extensions.  We've discussed both Kerberos and a system
based on public keys, but neither has been implemented at this time.
Currently access controls are based on the IP address and/or hostname
of the system requesting access.

Our current practice for sensitive repositories is to keep them behind
firewalls and use separate VPN software to restrict access.

> 7) Data Integrity. Can the system ensure the data integrity even in the
> event  of a serious system crash or others serious accidents?

Absolutely.  The entire Vesta system is designed to be fault-tolerant.
In nearly 4 years of use by a very demanding group, I can honestly say
that we have never experienced data loss due to a hardware or OS
failure.

> 8) Data protection. Can the repository be encrypted to ensure its
> confidentiality? Is the crypto system an integral part of the CMS or is
> it  borrowed from the underling OS?

Not at this time.

> 9) Binary files. Can the system actually manage binary files (binary
> diff and  binary patch)? How?

Absolutely.  Vesta treats files under version control like, well,
files.  Each one is a sequence of bytes.  The repository stores
exactly the bytes you ask it to, regardless of what they are.  It
never tries to understand them or modify them.

Vesta does this simply by storing complete copies of every version of
every source file.  Some people express concern over the storage space
costs of this approach, but in practice, this has not turned out to be
a problem.  In 3 years of a use, a team of over 150 engineers on a
very large project only managed to produce around 20 GB of source
data, which is not very much by today's standards of disk storage.

> 10) Process Control. Is it possible to enforce the user to perform the
> required operations in a given sequence, in a given way or within a
> given  time window?

Vesta does not have any built-in capabilities for process control.
(The Vesta repository does impose a certain set of restrictions, but I
wouldn't say that's really comparable.)

However, layering certain kinds of process control on top of Vesta is
not difficult.  In the group where I work, we have a semi-automated
nightly release process that works like this:

- Users optionally tag their new checkins as "ready for release".

- In the middle of the night, a script takes an existing configuration
and updates all its version references to point to newer versions that
have been tagged as "ready for release".

- A build is performed using the newly created configuration, and some
basic functionality tests are performed on it.  If it builds and
passes these tests, it is marked as the new "stable" configuration.
If not, it's fairly easy to identify the changes which were
incorporated (and who made them).

This is all done using scripts layered on top of Vesta.

> 11) IDE Integration. Can the CMS be integrated in a IDE? Which one?

To my knowledge nobody has done any IDE integration for Vesta yet.

Vesta has both a set of command-line tools and a C++ APIs.  In the
past we have found written integration tools to be pretty easy.

> 12) GUIs. Does the system have any GUI for simplifing the
> administrative  operations or the day-to-day use?

None that are publicly available.

The group that I work in has written an internal GUI for configuring
their builds (simplifying the selection of versions for the different
components going into a build).

> 13) Web Interface. Can the repository be accessed via web? Read-only or
>  read-write?

Yes:

http://www.scooter.cx/vestaweb/
http://pub.vestasys.org/cgi-bin/vestaweb?path=/vesta/vestasys.org

It was primarily designed for reading, but Scott has added
capabilities for users to perform checkout and checkin operations
using it.

> 14) Maturity Level. What maturity level has reached the system?
> (planning,  alpha, beta, release, in use, mature...).

Definitely mature.  It represents over 10 years of research, and
nearly 4 years of significant testing and use by a large and demanding
group.

> 15) Remote Access. Can the developer use the CMS through internet?

Absolutely.  In the setup instructions, we encourage people to use
it's replication facilities to get a copy of Vesta's source and some
examples.

> 16) Server Platform. Which platform is required for the server?
> Linux/Unix?  Windows?

> 17) Client Platform. For which platforms are available the clients?
> Linux/Unix only? Windows as well? Mac OS?

The answer is the same for both of these questions.  There are three
platforms supported at this time:

- Digital/Compaq/HP Tru64 UNIX (the platform Vesta was developed on)

- x86 Linux

- Alpha Linux

A port to PowerPC Linux is in the works, a port to Sparc Linux is
planned after that, and I hope to get it ported to Solaris (to further
broaden its possible user base).

> 18) Repository Nature. What is used to hold the data? The regular File
> System  of the host machine? A versioning file system? A special file
> system? A  database?

The Vesta repository acts like a special filesystem.  The portion
containing existing versions appears read-only to the user.  The
portion with active checkouts appears read-write.

The underlying files are stored using the host OS filesystem, with
what could be described as a custom journalled database storing the
filesystem metadata.

> 19) Licence. Can the system be used for free even in a commercial
> project?

Yes.  It's distributed under the LGPL.  I know of at least one
commercial user (Intel, where I work).  At this point, I am not
personally aware of free software projects using it other than Vesta
itself (and add-ons for it).

> 20 ) User Tracking / Auditing / Logging. Is it always possible to tell
> who,  when and how has changed a file (and a CMS parameter)?

All checkouts and checkins are logged, so it's definitely always
possible to tell who changed a versioned object and when they did it.

Repository attributes record when they were changed, but not by whom.

The system configuration is a regular file in the host OS filesystem,
so there's no direct control over it.  (However, one could use Vesta
to manage its revision.)

> 21) Concurrency. How the system manages the concurrent access to the
> same  file? Lock-Modify-Unlock? Checkout-Modify-Merge? Can the policy
> be chosen by  the administrator?

The normal path (which the checkout and checkin tools follow without
special instructions) reserves the next version in sequence on
checkout.  This is effectively a lock, as if another user tries to
check out the same package, they will be unable to reserve the next
version (because it's already reserved).

However, creating a branch does not require reserving a new version,
and can be performed concurrently with an active checkout.  If another
user wanted to make changes, they could create a branch and merge
their changes after the first user had checked in their changes.

> 22) Version Recovery. Is it always possible to recover an old version
> of any  individual object (and of a whole configuration)?

Absolutely.  This is, in fact, one of Vesta's strongest points.  As
described above, all historical versions are visible through the
repository's filesystem interface.  (You can, for example, diff
between them or grep across them directly from the command line.)

On top of this, configurations (and even entire builds) are versioned
objects.  Each one refers to specific immutable versions of all its
sub-components.  It's as easy to precisely repeat any past build as it
is to perform a new one.

Vesta has some particularly strong guarantees when it comes to build
repeatability, as the versions of all tools and libraries used by a
build are also part of its configuration.  (As far as I know, no other
free system comes anywhere near Vesta on this point, and from what
I've heard it is even significantly better than most commercial
systems.)

> 23) Version Tracking. Is it always possible to know (to browse) the
> whole  history of an individual object (and of a whole configuration)?

Absolutely.  As mentioned earlier, it all appears as a filesystem,
easily navigated with a multitude of tools.  Information about who
made checkins, when, and what comments they made are stored using the
repository's flexible attribute system.  (Scott Venier's vestaweb
interface provides a particularly nice interface for looking at the
history of a Vesta package.)

> 24) Dependency Control. Is it always possible to know which components
> are  required to build up a specific configuration of the program?

Essentially, yes.  It is always possible to know precisely which
versions of every component are *referenced* by a configuration/build.
However, it is possible that the build actually uses only a subset of
those components.

> 25) Configuration Management. Is it possible to manage a group of
> objects as  a single system? Can a specific configuration be marked
> (labelled) for a  better identification? Is the labelling required?

That depends on what you mean by "manage a group of objects as a
single system".

Vesta's notion of configurations is that a configuration (or build)
may reference specific versions of other objects.  Checking it out or
in does *not* also check out/in the referenced objects.  That would be
a separate step.

The Vesta repository has a very flexible attribute system which can be
used to attach arbitrary name/value pairs to versions.  This can be
used to record labels, among other uses.  (In the example I described
earlier, this is how users in my group tag new versions as "ready for
release".)  Use of this is not required.

> 26) Notifications. Is it possible to trigger a notification for every
> meaningful event that can affect the repository and its content?

Not at this time. There's been some discussion of adding support for
notification though.

> 27) Notifications 2. How can be delivered a notification to the human
> user?  E-mail only? Other ways?

> 28) Notifications 3. How can be delivered a notification to a program?
> E-mail? RPC? Others?

One proposal that's been discussed (and is the most likely to be
implemented in the near term) would run an arbitrary command
(configured by the administrator) on a repository action
(checkout/checkin/branch).  Such a command could do just about
anything, but it would be a per-installation setup choice.

> 29) Branching-Merging. Is it possible to branch in any moment, work for
> a  while on the branch and merge back without losing any data and
> without  crashing the system? (resorting to the human judgement for
> solving the  existing conflicts is perfectly acceptable).

It is always possible to branch from any existing version at any time.
This is never dangerous in any way, and cannot disrupt anything.

> Is it possible to merge a branch with another branch, instead of the
> original trunk, supposing that the two branch have a common
> ancestor?

> 30) Merging 2. How good is the merging mechanism? Like CVS? Better than
> CVS?  Worse?

Out of the box, Vesta does not include any built-in merging
capabilities.  However, because of the filesystem interface to the
repository, you can use any one of a number of different stand-alone
conflict resolution tools.  (I have personally used both Emacs' ediff
and the RCS merge command for this.)

One of Vesta's design goals is to minimize the number of special
"Vesta-aware" tools required to use it productively.  We feel that the
repository's filesystem interface is very helpful, by making it easy
for users to use stand-alone tools with Vesta.

If there's anything we can do to provide more information about Vesta,
or if you'd like additional clarification of anything discussed above,
please let us know.

I recommend taking a quick look at this page, which covers some of
Vesta's unique features, not all of which were touched on above:

http://www.vestasys.org/why-vesta.html

--Ken Schalk