[Treebase-devel] Migration update

[Vladimir G. <vga...@ne...>]

I am making progress on migrating data to the production instance, but  
it is going significantly slower than estimated.  Just loading  
matrices and trees took about 26 hours of pure running time instead of  
the 12 hours for the whole job that I estimated.  (It appears that my  
assumption of linear scaling was wrong.  I fear that the cost of  
uploading a new file is somehow dependent on the size of the database,  
as operations against the production instance look slower than against  
my empty testing instance.)  These 26 hours do not include times for  
manual checks and fix-ups that I had to do.

It can well take a couple more days from now to get through, provided  
tools performance and the rate at which problems show up and get  
mitigated remain the same.  (I am currently tracking down a hiccup  
that may prove trickier -- I'll follow up if so.)

Bill, I'd like to check with you on the number of new matrices and  
trees that were expected to be in the delta. The import tools are  
written to skip files with the names that are already in the  
database.  So, they uploaded about 590 new matrices and 720 new trees  
(compared, respectively, to 4348 files in the characters directory and  
5151 files in the trees directory).  Does this outcome look about  
right?  That is, did the data directories contain files that were  
loaded into the database earlier and did you NOT expect for them to  
affect the database?

--Vladimir