gamedevlists-general

[GD-General] Infrastructure for Distributed Development

[Chris R. <c....@gm...>]

Hi, 

a few former colleagues and I have committed ourselves to work on a pet
project in our spare time. Now, as we are spread out all over Europe, we
somehow need an infrastructure that aids(not hinders) us in this
distributed development effort, specifically regarding game assets. 

As I am currently developing with subversion as a version control
backend, and for the fact that you can integrate subversion
functionality into your program, I was thinking we might as well go for
subversion as a data-storage backend. 

This comes with a few constraints, many consequences and a lot of
decision-making beforehand ( locking vs. conflicts, binary vs. ascii,
etc.. ), but with the ability to embed subversion directly into your
asset-management, you can probably build a version-controlled
asset-management application that might hide many if not all of the
usage complexities of subversion from non-programmers ( explaining how
to "resolve a conflict in an xml-file" over a long distance phone call
is not an option ). 

One big downside is speed - subversion gets very slow as soon as the
repositories grow in the number of files/directories.

But before spending a lot of personal time and personal effort into this
solution, I wanted to ask around if somebody has a more efficient, or
simpler solution. Maybe I intend to tackle the problem from a too
complex perspective ( especially considering this is a spare-time
project ), or maybe there is a open-source/free product out there that
already does what we need, and I simply missed it ... Any
comments/experience/suggestions?

regards and thanks in advance, 
Chris Raine

Re: [GD-General] Infrastructure for Distributed Development

[Jamie F. <ja...@qu...>]

Chris Raine wrote:
> One big downside is speed - subversion gets very slow as soon as the
> repositories grow in the number of files/directories.

I'd like to dispute this claim. Our repository is pretty huge (roughly 
10 years development with a team of up to about 10 people on the code 
side), and speed is just fine for almost all operations. The one slow 
operation is checking out the repository... and log generation when you 
have that much history!

Jamie

> But before spending a lot of personal time and personal effort into this
> solution, I wanted to ask around if somebody has a more efficient, or
> simpler solution. Maybe I intend to tackle the problem from a too
> complex perspective ( especially considering this is a spare-time
> project ), or maybe there is a open-source/free product out there that
> already does what we need, and I simply missed it ... Any
> comments/experience/suggestions?
> 
> regards and thanks in advance, 
> Chris Raine

Re: [GD-General] Infrastructure for Distributed Development

[speedy <sp...@3d...>]

Hello gamedevlists-general crew,

      my experience with SVN performance - on recent 1.4.3+ versions:

      * linux based svnserve server runs great, except for
      the checkout which is a pure CPU hog. If you have strong CPU
      on the server (unlike I here O:) you should not notice the
      slowness even if multiple users hit it. Log file generation
      should be better with 1.4.x compared to 1.3 and earlier. [Jamie,
      I'm curious which server version are you using and how many
      history revisions are there? :)]
      
      * linux clients have no performance problems even with huge
      (multi GB binary assets, 10k+ folders) repositories
      
      * windows clients have problems with working copy folder count
      - after some number, all the temporary lock files hit the HDD,
      stalling the client process and the whole system. [If anyone
      is interested in helping fix that, I've posted a detailed bug report
      / analysis to the SVN mail-lists, but I haven't had enough spare
      time to fix it yet]

      For the next project I'll reconsider using SVN and intend to
      test a few alternatives, namely - SVK, mercurial and git.
      
      Hope this was helpfull. Cheers! :)
      
Sunday, October 21, 2007, 1:10:24 PM, you wrote:

JF> Chris Raine wrote:
>> One big downside is speed - subversion gets very slow as soon as the
>> repositories grow in the number of files/directories.

JF> I'd like to dispute this claim. Our repository is pretty huge (roughly
JF> 10 years development with a team of up to about 10 people on the code 
JF> side), and speed is just fine for almost all operations. The one slow 
JF> operation is checking out the repository... and log generation when you
JF> have that much history!

JF> Jamie

>> But before spending a lot of personal time and personal effort into this
>> solution, I wanted to ask around if somebody has a more efficient, or
>> simpler solution. Maybe I intend to tackle the problem from a too
>> complex perspective ( especially considering this is a spare-time
>> project ), or maybe there is a open-source/free product out there that
>> already does what we need, and I simply missed it ... Any
>> comments/experience/suggestions?
>> 
>> regards and thanks in advance, 
>> Chris Raine


JF> -------------------------------------------------------------------------
JF> This SF.net email is sponsored by: Splunk Inc.
JF> Still grepping through log files to find problems?  Stop.
JF> Now Search log events and configuration files using AJAX and a browser.
JF> Download your FREE copy of Splunk now >> http://get.splunk.com/
JF> _______________________________________________
JF> Gamedevlists-general mailing list
JF> Gam...@li...
JF> https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
JF> Archives:
JF> http://sourceforge.net/mailarchive/forum.php?forum_id=557



-- 
Best regards,
 speedy                            mailto:sp...@3d...

Re: [GD-General] Infrastructure for Distributed Development

[Jamie F. <ja...@qu...>]

speedy wrote:
> Hello gamedevlists-general crew,
> 
>       my experience with SVN performance - on recent 1.4.3+ versions:
> 
>       * linux based svnserve server runs great, except for
>       the checkout which is a pure CPU hog. If you have strong CPU
>       on the server (unlike I here O:) you should not notice the
>       slowness even if multiple users hit it. Log file generation
>       should be better with 1.4.x compared to 1.3 and earlier. [Jamie,
>       I'm curious which server version are you using and how many
>       history revisions are there? :)]

I _think_ we're still using 1.2.1 on the server; we're looking forward 
to better branch management with 1.5. We have 45566 revisions.

Jamie