XBRLAPI is taking large amout of time,space

Help
2012-03-28
2013-05-02
  • vineeth mohan

    vineeth mohan - 2012-03-28

    First run
    Time took to load a xbrl document of 2 MB -  3 minutes
    Space taken by database folder  - 195 MB

    Second run
    Time took to load a xbrl document of 8 MB -  11 minutes
    Space taken by database folder  - 253 MB

    What is the reason for so much time and space. Is this expected. Between i copied the xml to my localhost and ran the commands so it cant of any network's .

    Command ran and output received - https://gist.github.com/2225829

     
  • Geoffrey Shuetrim

    That sounds very slow and very demanding of database space.  I presume you are using the Oracle Berkeley XML Database?  Can you look at how many documents have been loaded during the process?  If the instance causes discovery of a huge DTS, that might account for the performance issues.

    Regards

    Geoff S

     
  • vineeth mohan

    vineeth mohan - 2012-03-29

    Yes , i am using berkely db.
    I just loaded the 2 links shown in the gist.
    You can see the size of each file i downloaded in the cache directory

    ./http/null/localhost/-1/null/null/xbrls/goog-20111231_cal.xml 125K
    ./http/null/localhost/-1/null/null/xbrls/goog-20111231_def.xml 547K
    ./http/null/localhost/-1/null/null/xbrls/goog-20111231_lab.xml 765K
    ./http/null/localhost/-1/null/null/xbrls/goog-20111231_pre.xml 643K
    ./http/null/localhost/-1/null/null/xbrls/goog-20111231.xml 1.9M
    ./http/null/localhost/-1/null/null/xbrls/goog-20111231.xsd 109K
    ./http/null/localhost/-1/null/null/xbrls/newOne/aapl-20111231_cal.xml 65K
    ./http/null/localhost/-1/null/null/xbrls/newOne/aapl-20111231_def.xml 244K
    ./http/null/localhost/-1/null/null/xbrls/newOne/aapl-20111231_lab.xml 360K
    ./http/null/localhost/-1/null/null/xbrls/newOne/aapl-20111231_pre.xml 289K
    ./http/null/localhost/-1/null/null/xbrls/newOne/aapl-20111231.xml 895K
    ./http/null/localhost/-1/null/null/xbrls/newOne/aapl-20111231.xsd 50K
    ./http/null/www.w3.org/-1/null/null/2001/03/datatypes.dtd 6.3K
    ./http/null/www.w3.org/-1/null/null/2001/03/XMLSchema.dtd 16K
    ./http/null/www.w3.org/-1/null/null/2001/03/xml.xsd 4.7K
    ./http/null/www.xbrlapi.org/-1/null/null/xml/schemas/s4s.xsd 91K
    ./http/null/www.xbrl.org/-1/null/null/2003/xbrl-instance-2003-12-31.xsd 24K
    ./http/null/www.xbrl.org/-1/null/null/2003/xbrl-linkbase-2003-12-31.xsd 16K
    ./http/null/www.xbrl.org/-1/null/null/2003/xl-2003-12-31.xsd 8.6K
    ./http/null/www.xbrl.org/-1/null/null/2003/xlink-2003-12-31.xsd 3.3K
    ./http/null/www.xbrl.org/-1/null/null/2005/xbrldt-2005.xsd 3.9K
    ./http/null/www.xbrl.org/-1/null/null/2006/ref-2006-02-27.xsd 6.4K
    ./http/null/www.xbrl.org/-1/null/null/dtr/type/nonNumeric-2009-12-16.xsd 4.0K
    ./http/null/www.xbrl.org/-1/null/null/dtr/type/numeric-2009-12-16.xsd 3.4K
    ./http/null/www.xbrl.org/-1/null/null/lrr/role/negated-2008-03-31.xsd 1.5K
    ./http/null/xbrl.fasb.org/-1/null/null/us-gaap/2011/elts/us-gaap-2011-01-31.xsd 4.0M
    ./http/null/xbrl.fasb.org/-1/null/null/us-gaap/2011/elts/us-roles-2011-01-31.xsd 194K
    ./http/null/xbrl.fasb.org/-1/null/null/us-gaap/2011/elts/us-types-2011-01-31.xsd 4.7K
    ./http/null/xbrl.sec.gov/-1/null/null/country/2011/country-2011-01-31.xsd 43K
    ./http/null/xbrl.sec.gov/-1/null/null/dei/2011/dei-2011-01-31.xsd 36K
    ./http/null/xbrl.sec.gov/-1/null/null/invest/2011/invest-2011-01-31.xsd 24K

     
  • vineeth mohan

    vineeth mohan - 2012-03-29

    I discovered that one of my basic assumption was wrong.
    I had taken the assumption that the files required for processing xbrl are as follows
    aapl-20111231_cal.xml  aapl-20111231_def.xml  aapl-20111231_lab.xml  aapl-20111231_pre.xml  aapl-20111231.xml  aapl-20111231.xsd

    But then it turns out it needs more file to do the same.
    Can you tell me what kind of files its trying to pull ?

     
  • vineeth mohan

    vineeth mohan - 2012-03-29

    Still it doesnt adds up , for what purpose is the 200 mb database folder used for ? Is it anything related to berkely db metadata or something ?

     
  • vineeth mohan

    vineeth mohan - 2012-03-30

    Hello @Geoff ,

    If there is any more input that you need to analyze the problem , let me know.
    Meanwhile can you tell me what is the reason for this behavior ?

    Thanks
              Vineeth

     

Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:

JavaScript is required for this form.





No, thanks