When loading a bit Turtle dataset from a file path, I get a "form too large" error (files are >400M)
loading smaller files by drag-n-drop or by file path works fine. Errors only happen with the big files (not just with one, but with several I have tried)
Linux/3.2.0-80-generic amd64
Intel(R) Core(TM) i7-3930K CPU @ 3.20GHz Family 6 Model 45 Stepping 7, GenuineIntel #CPU=12
Oracle Corporation 1.7.0_80
freeMemory=245798672
buildVersion=1.5.1
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The best way to load large data sets is to specify their URL and let the
server read the data from the URL. This also works when the graph is local
on the server since you can use a file URL.
You are hitting a limit with the size of the HTTP request body. I do not
offhand remember the property that controls the maximum size of a request
entity for jetty, but there is one. If you google with the method and text
of the exception you should be able to find this easily enough.
There will always be some limit when posting a large request entity. This
is because jetty has to buffer the entity in memory on the server. That is
why it is more scalable to send the URL of the data set you want to load.
Look at the NSS wiki page for some options. You could use SPARQL UPDATE
(the LOAD command) or the POST WITH URLs method on the NSS REST API.
When loading a bit Turtle dataset from a file path, I get a "form too
large" error (files are >400M)
loading smaller files by drag-n-drop or by file path works fine. Errors
only happen with the big files (not just with one, but with several I have
tried)
Linux/3.2.0-80-generic amd64
Intel(R) Core(TM) i7-3930K CPU @ 3.20GHz Family 6 Model 45 Stepping 7,
GenuineIntel #CPU=12
Oracle Corporation 1.7.0_80
freeMemory=245798672
buildVersion=1.5.1
Blazegraph™ http://www.blazegraph.com/ is our ultra high-performance
graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
APIs. MapGraph™ http://www.systap.com/mapgraph is our disruptive new
technology to use GPUs to accelerate data-parallel graph analytics.
CONFIDENTIALITY NOTICE: This email and its contents and attachments are
for the sole use of the intended recipient(s) and are confidential or
proprietary to SYSTAP. Any unauthorized review, use, disclosure,
dissemination or copying of this email or its contents or attachments is
prohibited. If you have received this communication in error, please notify
the sender by reply email and permanently delete all copies of the email
and its contents and attachments.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
When loading a bit Turtle dataset from a file path, I get a "form too large" error (files are >400M)
loading smaller files by drag-n-drop or by file path works fine. Errors only happen with the big files (not just with one, but with several I have tried)
Linux/3.2.0-80-generic amd64
Intel(R) Core(TM) i7-3930K CPU @ 3.20GHz Family 6 Model 45 Stepping 7, GenuineIntel #CPU=12
Oracle Corporation 1.7.0_80
freeMemory=245798672
buildVersion=1.5.1
The best way to load large data sets is to specify their URL and let the
server read the data from the URL. This also works when the graph is local
on the server since you can use a file URL.
You are hitting a limit with the size of the HTTP request body. I do not
offhand remember the property that controls the maximum size of a request
entity for jetty, but there is one. If you google with the method and text
of the exception you should be able to find this easily enough.
There will always be some limit when posting a large request entity. This
is because jetty has to buffer the entity in memory on the server. That is
why it is more scalable to send the URL of the data set you want to load.
Look at the NSS wiki page for some options. You could use SPARQL UPDATE
(the LOAD command) or the POST WITH URLs method on the NSS REST API.
Thanks,
Bryan
On Wednesday, April 29, 2015, Mark Wilkinson markwilkinson@users.sf.net amp#106;amp#97;amp#118;amp#97;amp#115;amp#99;amp#114;amp#105;amp#112;amp#116;amp#58;amp#95;amp#101;amp#40;amp#37;amp#55;amp#66;amp#37;amp#55;amp#68;amp#44;amp#39;amp#99;amp#118;amp#109;amp#108;amp#39;amp#44;amp#39;amp#109;amp#97;amp#114;amp#107;amp#119;amp#105;amp#108;amp#107;amp#105;amp#110;amp#115;amp#111;amp#110;amp#64;amp#117;amp#115;amp#101;amp#114;amp#115;amp#46;amp#115;amp#102;amp#46;amp#110;amp#101;amp#116;amp#39;amp#41;amp#59; wrote:
--
Bryan Thompson
Chief Scientist & Founder
SYSTAP, LLC
4501 Tower Road
Greensboro, NC 27410
bryan@systap.com
http://blazegraph.com
http://blog.bigdata.com http://bigdata.com
http://mapgraph.io
Blazegraph™ http://www.blazegraph.com/ is our ultra high-performance
graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
APIs. MapGraph™ http://www.systap.com/mapgraph is our disruptive new
technology to use GPUs to accelerate data-parallel graph analytics.
CONFIDENTIALITY NOTICE: This email and its contents and attachments are
for the sole use of the intended recipient(s) and are confidential or
proprietary to SYSTAP. Any unauthorized review, use, disclosure,
dissemination or copying of this email or its contents or attachments is
prohibited. If you have received this communication in error, please notify
the sender by reply email and permanently delete all copies of the email
and its contents and attachments.