Re: [Bigdata-developers] Fastest way to load data via Bulk Data Load

[Bryan T. <br...@bl...>]

Edgar,

There is no single configuration for maximum load throughput.  Instead
there are a variety of steps you can take to improve load performance. For
example, right sizing the jvm, using fast disk, maximizing inlining, etc.
Beyond these steps and those detailed on the wiki, we look at the entire
system to identify and remove bottlenecks.

Thanks,
Bryan
On May 31, 2016 5:28 PM, "Edgar Rodriguez-Diaz" <ed...@sy...> wrote:

> A correction here on the data size, it’s not 180G - it’s 18G of a gzip
> trig file exported by blazegraph; number of triples is correct.
>
> > On May 31, 2016, at 10:42 AM, Edgar Rodriguez-Diaz <ed...@sy...>
> wrote:
> >
> > Hi,
> >
> > I’ve been trying to use the DataLoader tool for bulk loading a very
> large file into blazegraph (~180G with ~4 billion triples) with and empty
> journal file, but I’m noticing a performance degradation on the rate of
> triples/s loaded. It started at around 55K and after 200 M triples the rate
> is around 32K, the rate keeps going down consistently.
> > What is the configuration to get the best performance out of the bulk
> load into blazegraph?
> >
> > Thanks.
> >
> > - Edgar
>
>
>
> ------------------------------------------------------------------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth and
> traffic
> patterns at an interface-level. Reveals which users, apps, and protocols
> are
> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
> J-Flow, sFlow and other flows. Make informed decisions using capacity
> planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
> _______________________________________________
> Bigdata-developers mailing list
> Big...@li...
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>