Re: [GUSDEV] out of memory when loading taxonomy

[Fernan A. <fe...@ii...>]

+----[ Jian Lu <jl...@vb...> (31.May.2006 10:45):
|
| Hi GUSDEV,
| 
| When I was using GUS plug-in to load NCBI taxonomy, it failed. The error 
| message is 'Out of Memory'. My machine's RAM is 4G which should be 
| enough to hold the data. Any hint would be appreciated. Thanks,
| 
| Jian
|
+----]

Jian,

can you post the exact error message (copy & paste)?

Is it Perl running out of memory? Or is it the db
(postgres/oracle)?

Your box has 4Gb RAM, but this doesn't mean that all of this
memory is available to the application.

Are you running the plugin from one host and the db server
in another host? If so, which one is 'getting out of
memory'?


What RDBMS are you using? Postgres? Oracle?

Have you set up your RDBMS to use the extra available
memory? Both Oracle and Postgres should be configured to use
more memory, if it's available. The default configuration
that (the one you get from a fresh install) would give you a
horrid performance and will not make use of extra RAM.


What OS are you using? Linux? FreeBSD? 

Have you tuned the OS? (through building a new kernel or
modifying kernel tunables using sysctl or something
similar?) Raising maxusers, maxfiles and/or a number of
kernel parameters can give your app more room to scale if
necessary. 


Is this a dedicated db server or is it running other
applications alongside postgres or oracle? Even if you tune
your db and OS to maximize the use of resources, running two
or more apps that compete for the same resource will lead to
degraded performance.

As you can see, there are a number of steps that you should
follow to make use of the available RAM and to optimize
performance of your DB app. 

And we need more information about your setup to be able to
guess what the problem might be.

Fernan

PS: I've succesfully loaded NCBI Taxonomy (in ~ 2hs) on
PostgreSQL running on a spare box that was not particularly
beefy (AMD Sempron, 1 Gb RAM, RAID-1 ATA100). But both the OS
(FreeBSD) and the application (PostgresSQL) were tuned to
maximize use of available resources.