[GUSDEV] ApiCommonData::Load::Plugin::InsertDbESTFiles

[Adhemar <az...@gm...>]

Hi,
I'm trying to load ESTs using InsertDbESTFiles.pm but it fails after a while
informing that it's not allowed to handle more than 10000 objects at one
time.
There's a copy of the whole output bellow.
How to fix this?
Thanks,
Adhemar



ga ApiCommonData::Load::Plugin::InsertDbESTFiles --inputFile
/storage/Data/db/SchistoDB/data/manualDelivery/SchistoDB/SjapAnhui/EST/NCBI/2011-06-03/sequence.fixed.est
--extDbName SjapAnhui_EstLibrary_RSRC --extDbRlsVer 2011-06-03
Mon Jun  6 12:32:15 2011    DSN    dbi:Oracle:host=bagre.cebio.org
;sid=schistodb30
Mon Jun  6 12:32:15 2011    PLUGIN
ApiCommonData::Load::Plugin::InsertDbESTFiles
Mon Jun  6 12:32:15 2011    ARG    algoinvo    1
Mon Jun  6 12:32:15 2011    ARG    comment
Mon Jun  6 12:32:15 2011    ARG    commit    0
Mon Jun  6 12:32:15 2011    ARG    debug    0
Mon Jun  6 12:32:15 2011    ARG    extDbName    SjapAnhui_EstLibrary_RSRC
Mon Jun  6 12:32:15 2011    ARG    extDbRlsVer    2011-06-03
Mon Jun  6 12:32:15 2011    ARG    group
Mon Jun  6 12:32:15 2011    ARG    gusconfigfile
/storage/Data/db/SchistoDB/gusApps/gus_home/config/gus.config
Mon Jun  6 12:32:15 2011    ARG    help
Mon Jun  6 12:32:15 2011    ARG    helpHTML
Mon Jun  6 12:32:15 2011    ARG    inputFile
/storage/Data/db/SchistoDB/data/manualDelivery/SchistoDB/SjapAnhui/EST/NCBI/2011-06-03/sequence.fixed.est
Mon Jun  6 12:32:15 2011    ARG    project
Mon Jun  6 12:32:15 2011    ARG    restartFile
Mon Jun  6 12:32:15 2011    ARG    sqlVerbose    0
Mon Jun  6 12:32:15 2011    ARG    user
Mon Jun  6 12:32:15 2011    ARG    verbose    0
Mon Jun  6 12:32:15 2011    ARG    veryVerbose    0
Mon Jun  6 12:32:15 2011    ARG    workflowstepid    0
Mon Jun  6 12:32:15 2011    AlgInvocationId    122
Mon Jun  6 12:32:15 2011    COMMIT    commit off


ERROR MESSAGE:  You have exceeded the maximum number of allowable objects in
memory:
  You must use the method 'undefPointerCache()' in each loop
    to clear the cache and allow garbage collection!!
 If you need > 10000 objects at one time,
   then increase the default number with the method
'setMaximumNumberOfObjects(<number>)' at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/ObjRelP/DbiDatabase.pm
line 483

GUS::ObjRelP::DbiDatabase::addToPointerCache('GUS::ObjRelP::DbiDatabase=HASH(0x1c2c5338)',
'GUS::Model::DoTS::NASequenceRef=HASH(0x20a0a978)') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/Model/GusRow.pm
line 28
    GUS::Model::GusRow::new('GUS::Model::DoTS::NASequenceRef',
'HASH(0x20a0aa20)') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/ApiCommonData/Load/Plugin/InsertDbESTFiles.pm
line 336

ApiCommonData::Load::Plugin::InsertDbESTFiles::getOrCreateReference('ApiCommonData::Load::Plugin::InsertDbESTFiles=HASH(0x1c2cacd8)',
'HASH(0x1d566110)') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/ApiCommonData/Load/Plugin/InsertDbESTFiles.pm
line 212

ApiCommonData::Load::Plugin::InsertDbESTFiles::processEST('ApiCommonData::Load::Plugin::InsertDbESTFiles=HASH(0x1c2cacd8)',
'HASH(0x1d566110)', 345, 62) called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/ApiCommonData/Load/Plugin/InsertDbESTFiles.pm
line 159

ApiCommonData::Load::Plugin::InsertDbESTFiles::run('ApiCommonData::Load::Plugin::InsertDbESTFiles=HASH(0x1c2cacd8)',
'HASH(0x1d57af38)') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/PluginMgr/GusApplication.pm
line 602
    GUS::PluginMgr::GusApplication::__ANON__() called at
/usr/local/lib/perl5/site_perl/5.12.3/Error.pm line 415
    eval {...} called at /usr/local/lib/perl5/site_perl/5.12.3/Error.pm line
407
    Error::subs::try('CODE(0x1d39cdf8)', 'HASH(0x1d57b5c8)') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/PluginMgr/GusApplication.pm
line 632

GUS::PluginMgr::GusApplication::doMajorMode_RunOrReport('GUS::PluginMgr::GusApplication=HASH(0x1b7c1d80)',
'ApiCommonData::Load::Plugin::InsertDbESTFiles', 1) called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/PluginMgr/GusApplication.pm
line 502

GUS::PluginMgr::GusApplication::doMajorMode_Run('GUS::PluginMgr::GusApplication=HASH(0x1b7c1d80)',
'ApiCommonData::Load::Plugin::InsertDbESTFiles') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/PluginMgr/GusApplication.pm
line 388

GUS::PluginMgr::GusApplication::doMajorMode('GUS::PluginMgr::GusApplication=HASH(0x1b7c1d80)',
'ApiCommonData::Load::Plugin::InsertDbESTFiles') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/PluginMgr/GusApplication.pm
line 297

GUS::PluginMgr::GusApplication::parseAndRun('GUS::PluginMgr::GusApplication=HASH(0x1b7c1d80)',
'ARRAY(0x1b7e0b48)') called
**** /projects/gusApps/gus_home/bin/ga on line 11

STACK TRACE:
You have exceeded the maximum number of allowable objects in memory:
  You must use the method 'undefPointerCache()' in each loop
    to clear the cache and allow garbage collection!!
 If you need > 10000 objects at one time,
   then increase the default number with the method
'setMaximumNumberOfObjects(<number>)' at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/ObjRelP/DbiDatabase.pm
line 483

GUS::ObjRelP::DbiDatabase::addToPointerCache('GUS::ObjRelP::DbiDatabase=HASH(0x1c2c5338)',
'GUS::Model::DoTS::NASequenceRef=HASH(0x20a0a978)') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/Model/GusRow.pm
line 28
    GUS::Model::GusRow::new('GUS::Model::DoTS::NASequenceRef',
'HASH(0x20a0aa20)') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/ApiCommonData/Load/Plugin/InsertDbESTFiles.pm
line 336

ApiCommonData::Load::Plugin::InsertDbESTFiles::getOrCreateReference('ApiCommonData::Load::Plugin::InsertDbESTFiles=HASH(0x1c2cacd8)',
'HASH(0x1d566110)') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/ApiCommonData/Load/Plugin/InsertDbESTFiles.pm
line 212

ApiCommonData::Load::Plugin::InsertDbESTFiles::processEST('ApiCommonData::Load::Plugin::InsertDbESTFiles=HASH(0x1c2cacd8)',
'HASH(0x1d566110)', 345, 62) called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/ApiCommonData/Load/Plugin/InsertDbESTFiles.pm
line 159

ApiCommonData::Load::Plugin::InsertDbESTFiles::run('ApiCommonData::Load::Plugin::InsertDbESTFiles=HASH(0x1c2cacd8)',
'HASH(0x1d57af38)') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/PluginMgr/GusApplication.pm
line 602

GUS::PluginMgr::GusApplication::doMajorMode_RunOrReport('GUS::PluginMgr::GusApplication=HASH(0x1b7c1d80)',
'ApiCommonData::Load::Plugin::InsertDbESTFiles', 1) called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/PluginMgr/GusApplication.pm
line 502

GUS::PluginMgr::GusApplication::doMajorMode_Run('GUS::PluginMgr::GusApplication=HASH(0x1b7c1d80)',
'ApiCommonData::Load::Plugin::InsertDbESTFiles') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/PluginMgr/GusApplication.pm
line 388

GUS::PluginMgr::GusApplication::doMajorMode('GUS::PluginMgr::GusApplication=HASH(0x1b7c1d80)',
'ApiCommonData::Load::Plugin::InsertDbESTFiles') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/PluginMgr/GusApplication.pm
line 297

GUS::PluginMgr::GusApplication::parseAndRun('GUS::PluginMgr::GusApplication=HASH(0x1b7c1d80)',
'ARRAY(0x1b7e0b48)') called at
/usr/local/lib/perl5/site_perl/5.12.3/Error.pm line 306
    Error::subs::run_clauses('HASH(0x1d57b5c8)', 'You have exceeded the
maximum number of allowable objects in ...', undef, 'ARRAY(0x1c08c988)')
called at /usr/local/lib/perl5/site_perl/5.12.3/Error.pm line 426
    Error::subs::try('CODE(0x1d39cdf8)', 'HASH(0x1d57b5c8)') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/PluginMgr/GusApplication.pm
line 632

GUS::PluginMgr::GusApplication::doMajorMode_RunOrReport('GUS::PluginMgr::GusApplication=HASH(0x1b7c1d80)',
'ApiCommonData::Load::Plugin::InsertDbESTFiles', 1) called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/PluginMgr/GusApplication.pm
line 502

GUS::PluginMgr::GusApplication::doMajorMode_Run('GUS::PluginMgr::GusApplication=HASH(0x1b7c1d80)',
'ApiCommonData::Load::Plugin::InsertDbESTFiles') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/PluginMgr/GusApplication.pm
line 388

GUS::PluginMgr::GusApplication::doMajorMode('GUS::PluginMgr::GusApplication=HASH(0x1b7c1d80)',
'ApiCommonData::Load::Plugin::InsertDbESTFiles') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/PluginMgr/GusApplication.pm
line 297

GUS::PluginMgr::GusApplication::parseAndRun('GUS::PluginMgr::GusApplication=HASH(0x1b7c1d80)',
'ARRAY(0x1b7e0b48)') called at /projects/gusApps/gus_home/bin/ga line 11


DBD::Oracle::st execute failed: ORA-12899: value too large for column
"CORE"."ALGORITHMINVOCATION"."RESULT" (actual: 2869, maximum: 350) (DBD
ERROR: error possibly near <*> indicator at char 143 in '
    UPDATE  Core.AlgorithmInvocation
    SET
    modification_date = SYSDATE,
    end_time = SYSDATE,
    row_alg_invocation_id = :p1,
    result = :<*>p2
    WHERE   algorithm_invocation_id = :p3') [for Statement "
    UPDATE  Core.AlgorithmInvocation
    SET
    modification_date = SYSDATE,
    end_time = SYSDATE,
    row_alg_invocation_id = ?,
    result = ?
    WHERE   algorithm_invocation_id = ?" with ParamValues: :p1='122',
:p2='You have exceeded the maximum number of allowable objects in memory:
  You must use the method 'undefPointerCache()' in each loop
    to clear the cache and allow garbage collection!!
 If you need > 10000 objects at one time,
   then increase the default number with the method
'setMaximumNumberOfObjects(<number>)' at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/ObjRelP/DbiDatabase.pm
line 483

GUS::ObjRelP::DbiDatabase::addToPointerCache('GUS::ObjRelP::DbiDatabase=HASH(0x1c2c5338)',
'GUS::Model::DoTS::NASequenceRef=HASH(0x20a0a978)') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/Model/GusRow.pm
line 28
    GUS::Model::GusRow::new('GUS::Model::DoTS::NASequenceRef',
'HASH(0x20a0aa20)') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/ApiCommonData/Load/Plugin/InsertDbESTFiles.pm
line 336

ApiCommonData::Load::Plugin::InsertDbESTFiles::getOrCreateReference('ApiCommonData::Load::Plugin::InsertDbESTFiles=HASH(0x1c2cacd8)',
'HASH(0x1d566110)...', :p3='122'] at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/ObjRelP/DbiDbHandle.pm
line 147, <ESTs> line 149834.


 SQL ERROR!! involving

    UPDATE  Core.AlgorithmInvocation
    SET
    modification_date = SYSDATE,
    end_time = SYSDATE,
    row_alg_invocation_id = ?,
    result = ?
    WHERE   algorithm_invocation_id = ?
 Values: 122, You have exceeded the maximum number of allowable objects in
memory:
  You must use the method 'undefPointerCache()' in each loop
    to clear the cache and allow garbage collection!!
 If you need > 10000 objects at one time,
   then increase the default number with the method
'setMaximumNumberOfObjects(<number>)' at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/ObjRelP/DbiDatabase.pm
line 483

GUS::ObjRelP::DbiDatabase::addToPointerCache('GUS::ObjRelP::DbiDatabase=HASH(0x1c2c5338)',
'GUS::Model::DoTS::NASequenceRef=HASH(0x20a0a978)') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/Model/GusRow.pm
line 28
    GUS::Model::GusRow::new('GUS::Model::DoTS::NASequenceRef',
'HASH(0x20a0aa20)') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/ApiCommonData/Load/Plugin/InsertDbESTFiles.pm
line 336

ApiCommonData::Load::Plugin::InsertDbESTFiles::getOrCreateReference('ApiCommonData::Load::Plugin::InsertDbESTFiles=HASH(0x1c2cacd8)',
'HASH(0x1d566110)') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/ApiCommonData/Load/Plugin/InsertDbESTFiles.pm
line 212

ApiCommonData::Load::Plugin::InsertDbESTFiles::processEST('ApiCommonData::Load::Plugin::InsertDbESTFiles=HASH(0x1c2cacd8)',
'HASH(0x1d566110)', 345, 62) called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/ApiCommonData/Load/Plugin/InsertDbESTFiles.pm
line 159

ApiCommonData::Load::Plugin::InsertDbESTFiles::run('ApiCommonData::Load::Plugin::InsertDbESTFiles=HASH(0x1c2cacd8)',
'HASH(0x1d57af38)') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/PluginMgr/GusApplication.pm
line 602
    GUS::PluginMgr::GusApplication::__ANON__() called at
/usr/local/lib/perl5/site_perl/5.12.3/Error.pm line 415
    eval {...} called at /usr/local/lib/perl5/site_perl/5.12.3/Error.pm line
407
    Error::subs::try('CODE(0x1d39cdf8)', 'HASH(0x1d57b5c8)') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/PluginMgr/GusApplication.pm
line 632

GUS::PluginMgr::GusApplication::doMajorMode_RunOrReport('GUS::PluginMgr::GusApplication=HASH(0x1b7c1d80)',
'ApiCommonData::Load::Plugin::InsertDbESTFiles', 1) called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/PluginMgr/GusApplication.pm
line 502

GUS::PluginMgr::GusApplication::doMajorMode_Run('GUS::PluginMgr::GusApplication=HASH(0x1b7c1d80)',
'ApiCommonData::Load::Plugin::InsertDbESTFiles') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/PluginMgr/GusApplication.pm
line 388

GUS::PluginMgr::GusApplication::doMajorMode('GUS::PluginMgr::GusApplication=HASH(0x1b7c1d80)',
'ApiCommonData::Load::Plugin::InsertDbESTFiles') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/PluginMgr/GusApplication.pm
line 297

GUS::PluginMgr::GusApplication::parseAndRun('GUS::PluginMgr::GusApplication=HASH(0x1b7c1d80)',
'ARRAY(0x1b7e0b48)') called, 122 at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/ObjRelP/DbiDbHandle.pm
line 166

GUS::ObjRelP::DbiDbHandle::death('GUS::ObjRelP::DbiDbHandle=HASH(0x1cf994e0)',
'\x{a} SQL ERROR!! involving\x{a} \x{a}    UPDATE
Core.AlgorithmInvocatio...') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/ObjRelP/DbiDbHandle.pm
line 150

GUS::ObjRelP::DbiDbHandle::sqlExec('GUS::ObjRelP::DbiDbHandle=HASH(0x1cf994e0)',
'GUS::ObjRelP::DbiDbHandle::st=HASH(0x1d9e8db8)', 'ARRAY(0x209b7700)',
'\x{a}    UPDATE  Core.AlgorithmInvocation\x{a}    SET
\x{a}\x{9}modificat...') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/ObjRelP/DbiRow.pm
line 731

GUS::ObjRelP::DbiRow::quote_and_update('GUS::Model::Core::AlgorithmInvocation=HASH(0x1c5d7e28)',
'HASH(0x1bc91080)', 'HASH(0x20a0a588)') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/ObjRelP/DbiRow.pm
line 612

GUS::ObjRelP::DbiRow::update('GUS::Model::Core::AlgorithmInvocation=HASH(0x1c5d7e28)')
called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/Model/GusRow.pm
line 1681

GUS::Model::GusRow::submit('GUS::Model::Core::AlgorithmInvocation=HASH(0x1c5d7e28)',
1) called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/PluginMgr/GusApplication.pm
line 1250

GUS::PluginMgr::GusApplication::closeInvocation('GUS::PluginMgr::GusApplication=HASH(0x1b7c1d80)',
'ApiCommonData::Load::Plugin::InsertDbESTFiles=HASH(0x1c2cacd8)', 'You have
exceeded the maximum number of allowable objects in ...') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/PluginMgr/GusApplication.pm
line 623
    GUS::PluginMgr::GusApplication::__ANON__() called at
/usr/local/lib/perl5/site_perl/5.12.3/Error.pm line 429
    Error::subs::try('CODE(0x1d39cdf8)', 'HASH(0x1d57b5c8)') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/PluginMgr/GusApplication.pm
line 632

GUS::PluginMgr::GusApplication::doMajorMode_RunOrReport('GUS::PluginMgr::GusApplication=HASH(0x1b7c1d80)',
'ApiCommonData::Load::Plugin::InsertDbESTFiles', 1) called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/PluginMgr/GusApplication.pm
line 502

GUS::PluginMgr::GusApplication::doMajorMode_Run('GUS::PluginMgr::GusApplication=HASH(0x1b7c1d80)',
'ApiCommonData::Load::Plugin::InsertDbESTFiles') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/PluginMgr/GusApplication.pm
line 388

GUS::PluginMgr::GusApplication::doMajorMode('GUS::PluginMgr::GusApplication=HASH(0x1b7c1d80)',
'ApiCommonData::Load::Plugin::InsertDbESTFiles') called at
/storage/Data/db/SchistoDB/gusApps/gus_home/lib/perl/GUS/PluginMgr/GusApplication.pm
line 297

GUS::PluginMgr::GusApplication::parseAndRun('GUS::PluginMgr::GusApplication=HASH(0x1b7c1d80)',
'ARRAY(0x1b7e0b48)') called at /projects/gusApps/gus_home/bin/ga line 11