Re: [Treebase-devel] Disk space for treebase production database

[William P. <wil...@ya...>]

On Aug 6, 2010, at 8:22 AM, Hilmar Lapp wrote:

> 
> On Aug 5, 2010, at 7:18 PM, William Piel wrote:
> 
>> This seems okay to me, given our constraints (Hilmar - WDYT?). 
> 
> 
> Sounds OK to me too. It's hard to imagine that there'd be very little to notice at 123 GB size, but an annoying degree of slowdown at 180 GB. We just need to keep in mind that whatever slowdown we see, it's not going to be any less for the real database, but quite possibly more.
> 
> 	-hilmar

As a test file, I have a large NEXUS (attached) which I uploaded to the treebase.org site, and it took me 7 minutes 51 seconds. Course, I'm doing it from China (which has fairly slow internet) but I suspect that the real slowdown is with the insertions. In fact, this file is too big to delete -- once uploaded, trying to delete it causes a error trace (probably because postgres times out) -- evidently, our upload methods are more efficient than our delete methods. With 378 taxa and 15993 characters, we're looking at 6,045,354 insertions for the matrix element table alone. The submitter has uploaded a bunch of these (located in submission 10698 / study 10708), but we can't delete them.  Since we don't have "cascade delete" constraints, and with all the FKs in circular relationships, it's also not easy to issue a SQL command to delete these.  But that's beside the point. We have a benchmark file that takes a significant time to import, so we can use it for testing. 

bp