So, looking at my current logs, I'm seeing 16 - 32 second delays from sending the request to getting a response. The data is loading very slowly (2 - 5 docs/sec) with 200 threads loading data. Any idea why this might be? Any settings to make it use more of the resources? The load average is currently 1.53 and that is mostly from PostgreSQL which is running smoothly.
The current logs I'm looking at don't report a 30 second load time. I'm investigating further to determine the difference between the current logs and the logs that did report a 30 second load time. I'm thinking that I might have been looking at the wrong logs after updating the branching factors and a couple of other metrics. Will report back with what I find.
I have bigdata running on a high performance computer and am not seeing the resource usage I expect. I'm running a program to load RDF documents to the store. When I have just a few threads running, say 6, the store keeps up. However, when I increase to 30, 50, 400 the store begins to have trouble keeping up. At 400 threads there is a 30 second delay between the call to insert data to bigdata and the response. During this time, the load average never exceeds 8 and the RAM is around 30 or 40 GB. This...