source control / patches

Developers
2005-07-11
2013-06-05
  • Having recently set up CVS for a separate Sourceforge project, and then having switched to an alternative source-code control system (DARCS), I thought I'd raise the issue here.

    The benefits of source-code control (SCC) are (usually) well-understood. We would probably all prefer to use an SCC system for all our local develppment, so long as it were easy to use, and didn't impede collaboration. Using a local SCC system enables a developer to roll-back to previous versions of the/their code, as well as recover files.

    CVS, though popular, isn't a good match for the two requirements above This is why I now use DARCS for my local SCC. DARCS is very simple to setup, and use. In addition, DARCS is inherently very good at supporting distributed and/or collaborative development.

    For example, Francois' most recent splash-screen modification required changes to the main AOI code. This required posting either a modified AOI binary, or a modified set of AOI source. Were AOI in CVS (at least on Francois' machine), Francois could also have created a patch file of his modifications, and posted that for anyone who wished to download it, to apply to a copy of the AOI source. Francois could also have sent the patch to Peter, for easy inclusion into AOI, should Peter accept the patch.

    However, CVS has the following problems in such a scenario:

    1. It is complex to use, meaning that part-time users will typically struggle unless they find a CVS GUI that makes sense to them (I can recommend one written in Java).

    2. CVS has no support for simultaneous local and remote repositories for the same project (often called a federated SCC). This makes Francois' example above much harder than it would seem.

    In contrast, DARCS is both easy to setup, even for infrequent users, and inherently supports both local and distributed (collaborative) development on the one project. Using DARCS, Francois could simply create a patch that could be sent to Peter for inclusion, and/or distributed for others to download and use, if they wish. In addition, DARCS has the following advantages:

    1. DARCS patches are much more independent than CVS patches. This means a user can normally choose a subset of DARCS patches from a list of available patches, and successfully apply them in any order. CVS patches often need to be applied in the correct order, and one patch may rely on others, making it much harder for a user to choose only those he/she wants.

    2. DARCS supports HTTP for read-only access to repositories, meaning it should be easy to setup on the sourceforge project.

    3. DARCS supports a number of simple protocols for collaboration, including email, making it straightforward for both experiened and infrequent users to successfully create and submit patches.

    Given the above, I would suggest the following to current AOI developers (whether they be developing mainline AOI, plugins, scripts, or documentation):

    1. We each investigate an SCC system for local use (if we don't already), and in particular DARCS.

    2. We consider whether we want a central AOI repository, and if so, using which SCC system.

    Cheers!
    Nik.

     
    • Wow, it sounds like this is the perfect tool for us... I never used CVS exactly for the aforementioned reasons, so probably using DARCS would make sens.

      Francois

       
      • Csaba Nagy
        Csaba Nagy
        2005-07-11

        Just a quick comment: using CVS is still better than not using any VCS :-)

        Is there any plan to create a version control system which is publicly available ? That would make it easier to follow the current development of the project, and would give faster feedback for new code (as potential users would be able to check things out before official releases). This speeds up development eventually I guess.

        I'm not an expert in version control systems in general, but with CVS I have enough experience to (mostly) properly set up a repository if needed - if this is the only reason there's no VCS up and running yet, I volunteer to do it.

        Cheers,
        Csaba.

         
    • For information only, our group (keel) has been using sourceforge CVS, with Eclipse as it's CVS client, for what must be pretty close to 5 years now ? Could it be that long ?

      It would be inappropriate for me to push this or any other set of tools off on anyone.....but for information purposes only...... our lead guy was a die-hard VI user, and yes, we had always found cvs cranky and used the command line client....and AFTER the transition I can tell you absolutely and positively that it has been a wonderful combination.

      CVS works great (though I have heard Subversion works even better) we have roll backs and the local sandbox so can flop back and forth between versions constantly, and we do it whenever needed. Eclipse as a tool is extremely well developed as a CVS client, and we use the compare editor frequently to find version differences when necessary. We get emails of the diff file from sourceforge wth every commit. And our previous VI die hard has led the charge and got us debugging code - stepping through code as it is executed and inspecting variables - right in eclipse.

      It isn't for sissies, the first few days of Eclipse without a friend at your desk to tell you what to click can be quite a challenge, but only because it is so powerful.

      This same Eclipse/CVS setup has been taken to two different offices where I have had contracts in the past couple of years, and in both places it was extremely welll received and productivity jumped as a consequence. In the Dallas user's groups, it is a challenge to find more than a few attendees who are using anything other than Eclipse for their daily work.

      Having said all this, I can still use my Eclipse setup fine as is without impacting you guys, just wanted to let you know that it is a great pair and probably tens of thousands of developers use this combination worldwide.

       
      • Everything you're saying is most probably right, except maybe you overlooked the scale of this project:
        i) Peter codes AoI alone, save for a few suggestions, which I bet he thoroughly reviews up to the point of rewriting part of the stuff.
        ii) Kevin, Nick and I are plugin developers, so we live on our own, except when our plugins need modifications in AoI (that's not often) in which case it's best to ask Peter anyway because he's the one who would know.
        iii) As for me I live on dialup. No high bandwidth anywhere near I live (but no noise no traffic jams either). And err, to be honest, I probably don't need much collaborative tools to collaborate with my own self (but I'll have a look to Darcs).

        I have to say I don't much like IDEs because I find they often get in the way and you end up working the IDE way rather than your own. But I'm not a professional developer so my opinion is probably not worth much. Even debugging through putting breakpoints and the like can be discussed because instead on focusing where your logic is wrong you focus on where should you put the next breakpoint. Java exception printout makes the use of a debugger useless IMO (OK, maybe overdoing it a bit).

        This being said, I appreciated your post. Saying I'm mildly interested and why is just saying the truth. Maybe your next post will point out some tool I'll adopt right away.

        Also if I wish to test Eclipse in the near future (only fools never change their minds) I know who to ask for project files.

        Francois.

         
        • Csaba Nagy
          Csaba Nagy
          2005-07-11

          Francois,

          I'm fairly new to AOI, and completely beginner in 3D modeling/animation. In fact I am a professional Java developer, modeling is just a hobby project but I start to like it :-)

          In terms of IDEs: you're right they get in your way sometimes, but I've got a cool one: IntelliJ's Idea. In terms of user interface it's the best I have seen. It won't get in your way unless you really ask for it :-) It's true it needs lots of memory (I have 1G on my development machines), but that's cheap nowdays.

          Anyway, I'm making here a lobby for setting up a version control system, any kind of it, because I strongly believe it helps any project. Even CVS, which is not the most advanced one, will make it easy to have multiple versions of the code accessible to everyone, e.g. stable versions for bugfixes living in parallel with development versions for unstable stuff.

          It would also help for the project to really take off if more people could get actively involved in it. I see that Peter has some delay in answering the AOI forums, and I'm sure this is not his primary activity - this doesn't make the project more attractive to people passing by... a stronger community will share the load between it's members. Now the AOI community is probably not that big (i'm too new here to be able to assess it), but considering AOI's qualities I see a big potential for it to really take off, and then a more community friendly development process is needed, and a version control system is part of that.

          Like I've said, I program in Java for a living, and have seen the advantages of version control. Believe me, even one-man projects should use one - actually the "team" capabilities are not the main reason it was invented, but more like the branching/versioning capabilities, keeping history and being able to recover older versions. In fact CVS is not even very team friendly, but it's still more than no VCS, as I said.

          Actually I'm also sure it would help for the plugins to live in the same repository, versioned together with the server. Peter said himself in the scripting tutorial that the AOI API (which is the basis of the scripts) is subject to change (this might not be the case if it starts to settle down in capabilities), which makes the scripts core-version dependent. That's most easily dealt with with a common versioning scheme and possibly a test suite which tries the scripts against the core and reports failures so the plugin developers know when a core change causes their script to be broken. In the code base I'm working on for a living, half of the code are test cases... and they saved us many hours of headaches.

          Ok, enough of the rants. In case you care for it, I've volunteered to set up and maintain a CVS repository for the project if time (or skills) is the only problem staying in the way of it. If there are more philosophycal reasons against it, than I apologize :-)

          Cheers,
          Csaba.

           
        • Peter Eastman
          Peter Eastman
          2005-07-12

          > Java exception printout makes the use of a debugger useless IMO (OK, maybe overdoing it a bit).

          You need to take another look at symbolic debuggers!  They're incredibly powerful tools.  It's not just about getting stack traces.  It's the fact that you can stop the program at any point and then inspect the value of any variable, change its value, invoke methods, etc.  A stack trace will tell you where the program discovered something was wrong, but it won't tell you how it got to be that way.  Besides, a lot of bugs don't produce an exception.  The program just works incorrectly.

          Peter

           
          • >It's the fact that you can stop the program at any point and then inspect the value of any variable, change its value, invoke methods, etc
            Yes I know that. But usually if anything goes not as expected (anything beyond null pointer exception) I sit and stare at the code. I've used symbolic debuggers, changing variables and all that. But I think the truth is the time I lose on the first debugging I gain on the knowledge of the algorithm, and it helps me anticipate other potential problems. Or maybe is it that my mind can only work on post mortem data? ;).

            Francois

             
    • Peter Eastman
      Peter Eastman
      2005-07-12

      First, I agree that version control systems are great tools.  Even if you just work by yourself, they do some very useful things.  I used CVS at work for almost five years.  It had its issues, but generally served us well.  Recently, my company moved us to ClearCase, which was definitely *not* a step in the right direction.  I've never heard of DARCS.  I've heard good things about Subversion - if we hadn't had ClearCase forced upon us, we would probably have eventually moved to that.

      On the other hand, a single developer working alone can get by with something a lot more light weight than any of those.  For example, some IDEs have built in local version control systems.  (IDEA does, and I think Eclipse does also.)  For one person, those work quite nicely.

      The main thing you get beyond that in something like CVS or Subversion is the collaboration tools.  You can have lots of developers all checking in code together, possibly working in different branches, etc., and it helps you stay sane through all of that.  But to be honest, I don't have much interest in that for AoI.  If I were to set up a public CVS repository, I wouldn't give anyone but myself commit access.  When someone sends me some code they've written, I first make my own decision about whether the feature they've implemented is a good idea.  Then I go through their code carefully, and often end up rewriting parts of it.

      Sorry, this isn't a democracy. :)  But that's how almost all open source projects work (or at least the successful ones).  You need central control, or you get chaos.  Similarly, I wouldn't expect Nik to give me commit access for the source code to the vector renderer.  That's his project, and while I might offer suggestions or even source code, he's free to accept or reject them as he chooses.

      That's not to say that a public version control system wouldn't still be useful for some things.  For example, it would give people an easy way to see exactly what files had changed in a particular release.  But it's not that high a priority, and given the hassles it forces you to deal with (or at least, all the ones I've worked with do), I'm not in that much of a hurry to start using one.

      Peter

       
      • Csaba Nagy
        Csaba Nagy
        2005-07-12

        [snip]
        >The main thing you get beyond that in something like CVS or Subversion is the collaboration tools.

        Well, that's not the only one. IDEA's local VCS is great, but it can't do branching and tagging, which are a great help for release management. It's designed more like for the programmers like me, who try 3 different code versions in half an hour before deciding which one to use :-) In that case it's handy to revert quickly to some code from half an hour ago.

        > Sorry, this isn't a democracy. :)

        Now that's a different matter, and explains it better why you don't want a VCS :-)

        > If I were to set up a public CVS repository, I wouldn't give anyone but myself commit access.

        That's fine, and a lot of projects work that way. It is good thing to have one person approving things. But it's even better to have different persons take care of their own field of experteese, and trust each other enough to not tip on each other's toes :-)

        > Similarly, I wouldn't expect Nik to give me commit access for the source code to the vector renderer.

        But would you trust him enough to give him checkin access to the same repository which holds the core code and his renderer, and you check in only to the core, he checks in only to the renderer ? This kind of trust is also beneficial for collaborative projects. People get expert in a small domain, and take it to the perfection, while having access to the rest of the domains at least tangentially to make integration easier. For example Nick can then simply do a CVS update to see if his super-duper speed improvement still works fine with your last core changes...  I still back the idea that a common versioning system for the core and the extensions plus a good test harness to make sure integration between them works fine would be beneficial and would make release management easier.

        > But it's not that high a priority, and given the hassles it forces you to deal with (or at least, all the ones I've worked with do), I'm not in that much of a hurry to start using one.

        Now I'm puzzled... for the simple purpose of having a commonly versioned core + extensions repository I can't see any hassles from my experience with CVS (I haven't tried any other VCS). It has it's hassles for the team part of it, and very poor support for refactoring (rename file for e.g.), but the first is not an issue with the single-developer
        model, and the rename in CVS has really good support in IDEA which I suppose you're using from you're post. I wonder what are the other hassles you're talking about ? I have created a couple of CVS repositories for my own use
        and I have yet to have any hassles with them...

        OK, I obviously need a bit more transparency and trust when it comes to join a project. I usually prefer to do some piece of work and then rewrite it 10 times until the project leader accepts it than to explain what I want and wait until the busy expert writes it eventually... I'm used to do throw-away work for the sake of faster project development. I'm just an unpatient guy who wants to see fast results, even if some of the intermediate steps will get thrown away... and a public CVS and a well organized developer hierarchy which builds on trust is going that way. Of course the leader is still needed to keep order in things :-)

        OK, enough of rants for today.

        Cheers,
        Csaba.

         
    • Thanks everyone for your thoughts.

      I am really not a DARCS bigot, although I'm about to sound like one.

      I would strongly recommend anyone interested in this discussion giving DARCS a try:

      http://abridgegame.org/darcs/

      To summarise the primary difference between CVS and DARCS: CVS is based on a central shared repository model, whereas Darcs is based on a decentralised local repository model. The way we (and Peter in particular) operate on AOI, DARCS is a better fit.

      (Actually, to be more precise, CVS is based on a model of *either* a single shared repository, *or* independent local repositories that cannot easily collaborate. DARCS is based on a model of local repositories that can always collaborate easily.)

      For the notebook developers among us, darcs directly supports checking out source from a shared repository to a local repository, and disconnecting from the net. Local changes can be saved/rewound from the local repository at will. Finally, when the notebook is again connected to the net, the most recent state can be uploaded to the shared resository. This is not impossible, but much more difficult and error prone in CVS.

      Once any developer is using DARCS (or CVS) for their local development at least, it is quite simple to generate one or more patch files of changes that can be sent (automatically or manually using DARCS; manually only using CVS) by the user to other developers (eg, Peter).

      The receiving developer (eg Peter) can choose which patches to import/apply into the code base, and how, with one very easy way being through using a local DARCS repository (DARCS can apply patches directly to a repository, whereas CVS has no facilities to import/apply patches).

      In this model, no developer needs to offer write access to any repository, and different developers can use different CCS systems (I could use DARCS, Francois could use CVS, and Peter could use none, or one embedded in his IDE, for example).

      A read-only global repository could be maintained in sourceforge in either CVS or DARCS format. Someone (not necessarily, but probably, the primary developer) would be responsible for updating the global repository periodically (at least once per release, more often would be better).

      The difference between CVS and DARCS for this being that with CVS, checkin to a global respository is awkward if that same person also uses CVS to maintain a local repository of the same code. With darcs, updating the repository is simply a matter of uploading the most recent patches (which darcs generates automatically).

      So in summary:

      1. I would suggest to anyone involved in development that they try darcs, at least for local repositories. It really is much less complicated than CVS. However, CVS can manage local repositories and create patches, so if you are already comfortable with CVS, there is no need to change.

      2. We discuss the benefits of being able to generate and share patch files of changes among different developers. Based on this, some or all of us may decide to select software that creates and imports compatible patches.

      3. We discuss what benefits (if any) there might be in having a central repository - on the understanding that it will almost certainly be read-only. If we can talk Peter into maintaining one for the mainline AOI code, I would suggest DARCS as a much easier way for him to do it than CVS.

      Cheers!
      Nik

       
    • Peter Eastman
      Peter Eastman
      2005-07-16

      Thanks for the description!  That does sound like a different model than the source control systems I've worked with, and possibly better suited to projects like this.  I'll take a look at it.  What kind of IDE integration is available for it?

      Peter

       
    • > What kind of IDE integration is available for it?

      Erk! .. I have no idea. As you know, I'm not particularly IDE dependent, and tend to drive ancilliary things like CCS separately.

      Um, there is an EclipseDarcs project on sourceforge - see also:

      http://eclipsedarcs.org

      It's alpha, and seems to have only the basic features enabled so far.

      Regarding IDEA, I have found that various IDEA plugins are kept in DARCS repositories, and that a number of people think that DARCS' patch model is ideally suited to handling the results of class refactoring. However, I have not found any evidence of a DARCS plugin for IDEA :o(

      (Being a bit of a plugin junkie, I am sooooo tempted to offer to write one - but I'm being good. Until I've completed AOS, I'm on the wagon (well, except for the tiny plugin I'm working on at the moment :o))

      Cheers!
      Nik