Re: [Exist-development] [ANN] Start of CODE FREEZE

[Adam R. <ad...@ex...>]

Okay I did import an initial code-base into Git but its very apparent
that it is not correct yet. The good news is that we have a repeatable
process that appears to work.

In the mean time I am lifting the code-freeze on SourceForge
Subversion to ease the pressure.
Once I have honed the conversion process, I will again initiate a code
freeze and at that point the move to GitHub should be quite quick.

On 18 March 2013 12:58, Adam Retter <ad...@ex...> wrote:
> So unfortunately, the use of git-svn failed with some sort of bug - I
> filed a bug report here and the git guys appear to be looking into it.
>
> In the mean time I switched over to looking at reposurgeon as a tool
> to use. Unfortunately I seem to have hit an error with reposurgeon
> too.
>
> So 3 tools down and no sucess.
>
> I have now switched to a different tool also confusingly called
> svn2git (this one is written in C++ and is available here -
> http://gitorious.org/svn2git/svn2git and seems to have been well used
> by the KDE project for their migrations.
>
> I have used this tool to convert our SVN repo to three Git repos and
> the process has certainly completed (very fast - just 15 mins or so).
> However, whether the results of the conversion are correct or not I am
> having a hard time understanding. The biggest problem is that our SVN
> repo was never a standard SVN layout and we really just chucked stuff
> in all over the place, this makes migration incredibly difficult as we
> do not want to inherit those mistakes. Indeed there is plenty of code
> branches that we dont even want to migrate at all.
>
> My suggestion is that I publish these repo's to GitHub in a read-only
> mode, and that everyone needs to do some QA to make sure they are
> happy with them and that the branch and commit history is as they
> would expect. Once everyone is happy (if the migration process was
> correct), I will switch them to read-write and lift the code-freeze.
>
> I will update again when the branches are on GitHub for review...
>
>
> On 10 March 2013 16:37, Adam Retter <ad...@ex...> wrote:
>> Okay time for an update -
>>
>> Unfortunately the svn2git process aborted this Friday. The console output was -
>>
>> Running command: git tag -a -m "beta test for cast as xs:base64Binary
>> - I need to finish checking this test - but eclipse wont seem to
>> "debug as junit" on linux or freebsd here - I should be able to use a
>> windows box here later tonight" "eXist-1.0@4658"
>> "svn/tags/eXist-1.0@4658"
>>
>> fatal: too many params
>>
>> command failed:
>>
>> 2>&1 git tag -a -m "beta test for cast as xs:base64Binary - I need to
>> finish checking this test - but eclipse wont seem to "debug as junit"
>> on linux or freebsd here - I should be able to use a windows box here
>> later tonight" "eXist-1.0@4658" "svn/tags/eXist-1.0@4658"
>>
>> It appears that there is a bug in svn2git whereby it does not handle
>> double-quotes in commit messages correctly, I have opened an issue for
>> this here - https://github.com/nirvdrum/svn2git/issues/107
>>
>> I also spent some time in #git of FreeNode discussing the issue and
>> getting some suggestions about how to proceed. Rather than using
>> svn2git (as recommended by GitHub's documentation), I am now using
>> git-svn directly. I have started another clone to git of our
>> subversion repository...
>>
>> I hope to have some more positive news soon for everyone.
>>
>> Sorry for the delays, unfortunately this process has not been as
>> smooth as it should have been. Certainly previous migrations I have
>> been involved in have been simpler and quicker, however Git
>> aficionados assure me that subversion repositories much larger than
>> ours have been migrated successfully.
>>
>> On 25 February 2013 22:47, Adam Retter <ad...@ex...> wrote:
>>> A status update -
>>>
>>> So I am afraid to report that the svn2git process running on Amazon
>>> stopped earlier today. It stopped because it appears that there was a
>>> network interruption between Amazon EC2 and SourceForge's SVN server.
>>> I cannot be certain but it looks like SourceForge's server stopped
>>> responding.
>>>
>>> There is no way to restart the svn2git process reliably as I have
>>> mentioned before. So what is happening now?
>>>
>>> I have leased a new larger EC2 instance, a CPU Intensive Medium
>>> machine (c1.medium). I am adopting a different approach which should
>>> be network resilient:
>>>
>>> I have rsync'd the entire eXist SVN repo from SourceForge to my EC2
>>> instance (~1GB) so that I have a complete local copy of the entire
>>> repo. I have then compiled the latest Git and installed the latest
>>> svn2git and set the svn2git process to work on the local repo.
>>>
>>> Subjectively the processing seems much faster on the new larger
>>> instance, but again well I guess its going to take several days. The
>>> new instance CPU is about 2.5x faster that the one I was using
>>> previously, the fact that no network communication is required should
>>> also hopefully speed this up further.
>>>
>>> I hope to post a new more positive update soon...
>>>
>>>
>>>
>>> On 23 February 2013 11:21, Adam Retter <ad...@ex...> wrote:
>>>> After checking the metrics of the EC2 instance this morning, it seems
>>>> that svn2git is pretty much CPU bound on here, consistently consuming
>>>> 75% - 100% of the CPU. I was using a m1.small instance to keep costs
>>>> low.
>>>>
>>>> I could switch to a larger instance, but that would involve having to
>>>> restart the process from scratch, svn2git does not have any reliable
>>>> way to resume. Also for each doubling of the CPU, the costs double,
>>>> and there is no guarantee that it will be twice as fast. I am not sure
>>>> but I guess svn2git is single-threaded, so we could only step up to a
>>>> m1.medium and expect to see any sort of performance improvement.
>>>>
>>>> As this has now been running for six days, I am more inclined to leave
>>>> it running than switch to an m1.medium instance which is twice the
>>>> size (as we know this will take at 3+ days to repeat what has already
>>>> been done on the m1.small instance).
>>>> We can re-evaluate this in a few days perhaps...
>>>>
>>>> On 22 February 2013 20:26, Adam Retter <ad...@ex...> wrote:
>>>>> For those that are wondering, the process is still under way.
>>>>>
>>>>> Thank you all for being patient and please be certain that I will
>>>>> announce as soon as the conversion process is complete.
>>>>>
>>>>> Currently we are up to r17046 of the tag eXist-2.0RC2. I am not
>>>>> certain of the order that the repository is processed in, but I think
>>>>> this is probably a good sign that it should complete over the weekend
>>>>> (hopefully).
>>>>>
>>>>> On 17 February 2013 11:48, Adam Retter <ad...@ex...> wrote:
>>>>>> The process running on EC2 stopped unexpectedly, this seems to be a
>>>>>> known problem in the version of Git that ships with EC2 Linux -
>>>>>> http://thread.gmane.org/gmane.comp.version-control.git/134936/focus=134940
>>>>>>
>>>>>> I have now compiled and installed the latest version of Git and have
>>>>>> restarted the process. Stay tuned...
>>>>>>
>>>>>> On 15 February 2013 12:36, Adam Retter <ad...@ex...> wrote:
>>>>>>> A further update -
>>>>>>>
>>>>>>> Running the process on my laptop turned out to be too difficult as I
>>>>>>> experience various network disconnects ever 8 hours of so.
>>>>>>>
>>>>>>> I have now rented an Amazon EC2 instance and have restarted the
>>>>>>> process on there, this should be much quicker. Stay tuned...
>>>>>>>
>>>>>>> On 14 February 2013 15:15, Adam Retter <ad...@ex...> wrote:
>>>>>>>> Its still processing, I expect to continue the manual aspects of the
>>>>>>>> process around 4pm UTC
>>>>>>>>
>>>>>>>> On 14 February 2013 12:21, Dannes Wessels <da...@ex...> wrote:
>>>>>>>>> any update?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Feb 14, 2013 at 1:11 AM, Adam Retter <ad...@ex...> wrote:
>>>>>>>>>>
>>>>>>>>>> The svn2git process is taking some time, it will continue running over
>>>>>>>>>> night.
>>>>>>>>>>
>>>>>>>>>> Further updates to follow...
>>>>>>>>>>
>>>>>>>>>> On 13 February 2013 21:03, Adam Retter <ad...@ex...> wrote:
>>>>>>>>>> > A code freeze is imposed on eXist at SourceForge whilst we perform a
>>>>>>>>>> > first attempt at moving to GitHub.
>>>>>>>>>> >
>>>>>>>>>> > Another announcement will follow after the attempt.
>>>>>>>>>> >
>>>>>>>>>> > --
>>>>>>>>>> > Adam Retter
>>>>>>>>>> >
>>>>>>>>>> > eXist Developer
>>>>>>>>>> > { United Kingdom }
>>>>>>>>>> > ad...@ex...
>>>>>>>>>> > irc://irc.freenode.net/existdb
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Adam Retter
>>>>>>>>>>
>>>>>>>>>> eXist Developer
>>>>>>>>>> { United Kingdom }
>>>>>>>>>> ad...@ex...
>>>>>>>>>> irc://irc.freenode.net/existdb
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>>>> Free Next-Gen Firewall Hardware Offer
>>>>>>>>>> Buy your Sophos next-gen firewall before the end March 2013
>>>>>>>>>> and get the hardware for free! Learn more.
>>>>>>>>>> http://p.sf.net/sfu/sophos-d2d-feb
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Exist-development mailing list
>>>>>>>>>> Exi...@li...
>>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/exist-development
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> eXist-db Native XML Database - http://exist-db.org
>>>>>>>>> Join us on linked-in: http://www.linkedin.com/groups?gid=35624
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Adam Retter
>>>>>>>>
>>>>>>>> eXist Developer
>>>>>>>> { United Kingdom }
>>>>>>>> ad...@ex...
>>>>>>>> irc://irc.freenode.net/existdb
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Adam Retter
>>>>>>>
>>>>>>> eXist Developer
>>>>>>> { United Kingdom }
>>>>>>> ad...@ex...
>>>>>>> irc://irc.freenode.net/existdb
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Adam Retter
>>>>>>
>>>>>> eXist Developer
>>>>>> { United Kingdom }
>>>>>> ad...@ex...
>>>>>> irc://irc.freenode.net/existdb
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Adam Retter
>>>>>
>>>>> eXist Developer
>>>>> { United Kingdom }
>>>>> ad...@ex...
>>>>> irc://irc.freenode.net/existdb
>>>>
>>>>
>>>>
>>>> --
>>>> Adam Retter
>>>>
>>>> eXist Developer
>>>> { United Kingdom }
>>>> ad...@ex...
>>>> irc://irc.freenode.net/existdb
>>>
>>>
>>>
>>> --
>>> Adam Retter
>>>
>>> eXist Developer
>>> { United Kingdom }
>>> ad...@ex...
>>> irc://irc.freenode.net/existdb
>>
>>
>>
>> --
>> Adam Retter
>>
>> eXist Developer
>> { United Kingdom }
>> ad...@ex...
>> irc://irc.freenode.net/existdb
>
>
>
> --
> Adam Retter
>
> eXist Developer
> { United Kingdom }
> ad...@ex...
> irc://irc.freenode.net/existdb



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb