XBRLAPI is taking large amout of time,space

[vineeth mohan]

First run
Time took to load a xbrl document of 2 MB -  3 minutes
Space taken by database folder  - 195 MB

Second run
Time took to load a xbrl document of 8 MB -  11 minutes
Space taken by database folder  - 253 MB

What is the reason for so much time and space. Is this expected. Between i copied the xml to my localhost and ran the commands so it cant of any network's .

Command ran and output received - https://gist.github.com/2225829

[Geoffrey Shuetrim]

That sounds very slow and very demanding of database space.  I presume you are using the Oracle Berkeley XML Database?  Can you look at how many documents have been loaded during the process?  If the instance causes discovery of a huge DTS, that might account for the performance issues.

Regards

Geoff S

[vineeth mohan]

Yes , i am using berkely db.
I just loaded the 2 links shown in the gist.
You can see the size of each file i downloaded in the cache directory

./http/null/localhost/-1/null/null/xbrls/goog-20111231_cal.xml 125K
./http/null/localhost/-1/null/null/xbrls/goog-20111231_def.xml 547K
./http/null/localhost/-1/null/null/xbrls/goog-20111231_lab.xml 765K
./http/null/localhost/-1/null/null/xbrls/goog-20111231_pre.xml 643K
./http/null/localhost/-1/null/null/xbrls/goog-20111231.xml 1.9M
./http/null/localhost/-1/null/null/xbrls/goog-20111231.xsd 109K
./http/null/localhost/-1/null/null/xbrls/newOne/aapl-20111231_cal.xml 65K
./http/null/localhost/-1/null/null/xbrls/newOne/aapl-20111231_def.xml 244K
./http/null/localhost/-1/null/null/xbrls/newOne/aapl-20111231_lab.xml 360K
./http/null/localhost/-1/null/null/xbrls/newOne/aapl-20111231_pre.xml 289K
./http/null/localhost/-1/null/null/xbrls/newOne/aapl-20111231.xml 895K
./http/null/localhost/-1/null/null/xbrls/newOne/aapl-20111231.xsd 50K
./http/null/www.w3.org/-1/null/null/2001/03/datatypes.dtd 6.3K
./http/null/www.w3.org/-1/null/null/2001/03/XMLSchema.dtd 16K
./http/null/www.w3.org/-1/null/null/2001/03/xml.xsd 4.7K
./http/null/www.xbrlapi.org/-1/null/null/xml/schemas/s4s.xsd 91K
./http/null/www.xbrl.org/-1/null/null/2003/xbrl-instance-2003-12-31.xsd 24K
./http/null/www.xbrl.org/-1/null/null/2003/xbrl-linkbase-2003-12-31.xsd 16K
./http/null/www.xbrl.org/-1/null/null/2003/xl-2003-12-31.xsd 8.6K
./http/null/www.xbrl.org/-1/null/null/2003/xlink-2003-12-31.xsd 3.3K
./http/null/www.xbrl.org/-1/null/null/2005/xbrldt-2005.xsd 3.9K
./http/null/www.xbrl.org/-1/null/null/2006/ref-2006-02-27.xsd 6.4K
./http/null/www.xbrl.org/-1/null/null/dtr/type/nonNumeric-2009-12-16.xsd 4.0K
./http/null/www.xbrl.org/-1/null/null/dtr/type/numeric-2009-12-16.xsd 3.4K
./http/null/www.xbrl.org/-1/null/null/lrr/role/negated-2008-03-31.xsd 1.5K
./http/null/xbrl.fasb.org/-1/null/null/us-gaap/2011/elts/us-gaap-2011-01-31.xsd 4.0M
./http/null/xbrl.fasb.org/-1/null/null/us-gaap/2011/elts/us-roles-2011-01-31.xsd 194K
./http/null/xbrl.fasb.org/-1/null/null/us-gaap/2011/elts/us-types-2011-01-31.xsd 4.7K
./http/null/xbrl.sec.gov/-1/null/null/country/2011/country-2011-01-31.xsd 43K
./http/null/xbrl.sec.gov/-1/null/null/dei/2011/dei-2011-01-31.xsd 36K
./http/null/xbrl.sec.gov/-1/null/null/invest/2011/invest-2011-01-31.xsd 24K

[vineeth mohan]

I discovered that one of my basic assumption was wrong.
I had taken the assumption that the files required for processing xbrl are as follows
aapl-20111231_cal.xml  aapl-20111231_def.xml  aapl-20111231_lab.xml  aapl-20111231_pre.xml  aapl-20111231.xml  aapl-20111231.xsd

But then it turns out it needs more file to do the same.
Can you tell me what kind of files its trying to pull ?

[vineeth mohan]

Still it doesnt adds up , for what purpose is the 200 mb database folder used for ? Is it anything related to berkely db metadata or something ?

[vineeth mohan]

Hello @Geoff ,

If there is any more input that you need to analyze the problem , let me know.
Meanwhile can you tell me what is the reason for this behavior ?

Thanks
          Vineeth