Re: [Exist-development] [ANN] Start of CODE FREEZE

[Adam R. <ad...@ex...>]

A status update -

So I am afraid to report that the svn2git process running on Amazon
stopped earlier today. It stopped because it appears that there was a
network interruption between Amazon EC2 and SourceForge's SVN server.
I cannot be certain but it looks like SourceForge's server stopped
responding.

There is no way to restart the svn2git process reliably as I have
mentioned before. So what is happening now?

I have leased a new larger EC2 instance, a CPU Intensive Medium
machine (c1.medium). I am adopting a different approach which should
be network resilient:

I have rsync'd the entire eXist SVN repo from SourceForge to my EC2
instance (~1GB) so that I have a complete local copy of the entire
repo. I have then compiled the latest Git and installed the latest
svn2git and set the svn2git process to work on the local repo.

Subjectively the processing seems much faster on the new larger
instance, but again well I guess its going to take several days. The
new instance CPU is about 2.5x faster that the one I was using
previously, the fact that no network communication is required should
also hopefully speed this up further.

I hope to post a new more positive update soon...



On 23 February 2013 11:21, Adam Retter <ad...@ex...> wrote:
> After checking the metrics of the EC2 instance this morning, it seems
> that svn2git is pretty much CPU bound on here, consistently consuming
> 75% - 100% of the CPU. I was using a m1.small instance to keep costs
> low.
>
> I could switch to a larger instance, but that would involve having to
> restart the process from scratch, svn2git does not have any reliable
> way to resume. Also for each doubling of the CPU, the costs double,
> and there is no guarantee that it will be twice as fast. I am not sure
> but I guess svn2git is single-threaded, so we could only step up to a
> m1.medium and expect to see any sort of performance improvement.
>
> As this has now been running for six days, I am more inclined to leave
> it running than switch to an m1.medium instance which is twice the
> size (as we know this will take at 3+ days to repeat what has already
> been done on the m1.small instance).
> We can re-evaluate this in a few days perhaps...
>
> On 22 February 2013 20:26, Adam Retter <ad...@ex...> wrote:
>> For those that are wondering, the process is still under way.
>>
>> Thank you all for being patient and please be certain that I will
>> announce as soon as the conversion process is complete.
>>
>> Currently we are up to r17046 of the tag eXist-2.0RC2. I am not
>> certain of the order that the repository is processed in, but I think
>> this is probably a good sign that it should complete over the weekend
>> (hopefully).
>>
>> On 17 February 2013 11:48, Adam Retter <ad...@ex...> wrote:
>>> The process running on EC2 stopped unexpectedly, this seems to be a
>>> known problem in the version of Git that ships with EC2 Linux -
>>> http://thread.gmane.org/gmane.comp.version-control.git/134936/focus=134940
>>>
>>> I have now compiled and installed the latest version of Git and have
>>> restarted the process. Stay tuned...
>>>
>>> On 15 February 2013 12:36, Adam Retter <ad...@ex...> wrote:
>>>> A further update -
>>>>
>>>> Running the process on my laptop turned out to be too difficult as I
>>>> experience various network disconnects ever 8 hours of so.
>>>>
>>>> I have now rented an Amazon EC2 instance and have restarted the
>>>> process on there, this should be much quicker. Stay tuned...
>>>>
>>>> On 14 February 2013 15:15, Adam Retter <ad...@ex...> wrote:
>>>>> Its still processing, I expect to continue the manual aspects of the
>>>>> process around 4pm UTC
>>>>>
>>>>> On 14 February 2013 12:21, Dannes Wessels <da...@ex...> wrote:
>>>>>> any update?
>>>>>>
>>>>>>
>>>>>> On Thu, Feb 14, 2013 at 1:11 AM, Adam Retter <ad...@ex...> wrote:
>>>>>>>
>>>>>>> The svn2git process is taking some time, it will continue running over
>>>>>>> night.
>>>>>>>
>>>>>>> Further updates to follow...
>>>>>>>
>>>>>>> On 13 February 2013 21:03, Adam Retter <ad...@ex...> wrote:
>>>>>>> > A code freeze is imposed on eXist at SourceForge whilst we perform a
>>>>>>> > first attempt at moving to GitHub.
>>>>>>> >
>>>>>>> > Another announcement will follow after the attempt.
>>>>>>> >
>>>>>>> > --
>>>>>>> > Adam Retter
>>>>>>> >
>>>>>>> > eXist Developer
>>>>>>> > { United Kingdom }
>>>>>>> > ad...@ex...
>>>>>>> > irc://irc.freenode.net/existdb
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Adam Retter
>>>>>>>
>>>>>>> eXist Developer
>>>>>>> { United Kingdom }
>>>>>>> ad...@ex...
>>>>>>> irc://irc.freenode.net/existdb
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------------------------------------------------------
>>>>>>> Free Next-Gen Firewall Hardware Offer
>>>>>>> Buy your Sophos next-gen firewall before the end March 2013
>>>>>>> and get the hardware for free! Learn more.
>>>>>>> http://p.sf.net/sfu/sophos-d2d-feb
>>>>>>> _______________________________________________
>>>>>>> Exist-development mailing list
>>>>>>> Exi...@li...
>>>>>>> https://lists.sourceforge.net/lists/listinfo/exist-development
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> eXist-db Native XML Database - http://exist-db.org
>>>>>> Join us on linked-in: http://www.linkedin.com/groups?gid=35624
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Adam Retter
>>>>>
>>>>> eXist Developer
>>>>> { United Kingdom }
>>>>> ad...@ex...
>>>>> irc://irc.freenode.net/existdb
>>>>
>>>>
>>>>
>>>> --
>>>> Adam Retter
>>>>
>>>> eXist Developer
>>>> { United Kingdom }
>>>> ad...@ex...
>>>> irc://irc.freenode.net/existdb
>>>
>>>
>>>
>>> --
>>> Adam Retter
>>>
>>> eXist Developer
>>> { United Kingdom }
>>> ad...@ex...
>>> irc://irc.freenode.net/existdb
>>
>>
>>
>> --
>> Adam Retter
>>
>> eXist Developer
>> { United Kingdom }
>> ad...@ex...
>> irc://irc.freenode.net/existdb
>
>
>
> --
> Adam Retter
>
> eXist Developer
> { United Kingdom }
> ad...@ex...
> irc://irc.freenode.net/existdb



--
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb