Re: [Bigdata-developers] Requesting clarification on BigData internal structure with regards to compression

[Fredah B <fre...@gm...>]

Dear Brad, Bryan,

Thank you so much for the quick response and the great feedback! The
explanation given is very clear and I'm even more thankful for the
reference to the architecture.

One again, thank you so much and I'll be in touch.

Regards,

Fredah

On Tue, May 5, 2015 at 4:51 AM, Brad Bebee <be...@sy...> wrote:

> Fredah,
>
> To add to Bryan's message, we'd love also to hear about your project and
> successes you have with it.  Let us know as you progress and we can feature
> it on our Blog and/or Wiki.  In addition to the open source support, we do
> also have a few other options; just drop a line if that's of interest to
> you as well.
>
> Regards, --Brad
>
> On Mon, May 4, 2015 at 8:10 PM, Bryan Thompson <br...@sy...> wrote:
>
>> Fredah,
>>
>> Please see
>> http://www.blazegraph.com/whitepapers/bigdata_architecture_whitepaper.pdf
>> for a good overview of the platform. The data are stored in indices
>> (B+Trees).  Prefix compression is used as well as a variety of other
>> compression techniques. We then encode the index pages into a
>> compressed representation that can be interpreted without
>> decompression of the page.  We do not currently compress the index
>> page records using a block compression strategy.  This is possible,
>> but there are other compression techniques that I would implement
>> first.
>>
>> Thanks,
>> Bryan
>> ----
>> Bryan Thompson
>> Chief Scientist & Founder
>> SYSTAP, LLC
>> 4501 Tower Road
>> Greensboro, NC 27410
>> br...@sy...
>> http://blazegraph.com
>> http://blog.bigdata.com
>> http://mapgraph.io
>>
>> Blazegraph™ is our ultra high-performance graph database that supports
>> both RDF/SPARQL and Tinkerpop/Blueprints APIs.  MapGraph™ is our
>> disruptive new technology to use GPUs to accelerate data-parallel
>> graph analytics.
>>
>> CONFIDENTIALITY NOTICE:  This email and its contents and attachments
>> are for the sole use of the intended recipient(s) and are confidential
>> or proprietary to SYSTAP. Any unauthorized review, use, disclosure,
>> dissemination or copying of this email or its contents or attachments
>> is prohibited. If you have received this communication in error,
>> please notify the sender by reply email and permanently delete all
>> copies of the email and its contents and attachments.
>>
>>
>> On Mon, May 4, 2015 at 7:04 PM, Fredah Banda <fre...@ma...>
>> wrote:
>> > Dear Team,
>> >
>> >
>> > I plan on using your SPARQL engine for my project implementation. I’m
>> > impressed by the tremendous work you have put in to make this engine a
>> > success however I did notice that the underlying infrastructure and
>> > compression technique used are encapsulated. I need to fully understand
>> how
>> > the data is processed from start to finish especially with regards to
>> the
>> > compression. Are there by any chance papers that have been written that
>> > cover the compression and decompression used in your engine or is it
>> > possible to refer me to someone who may be able to explain it to me?
>> >
>> >
>> > Also, is compression default or is turned on and off depending on the
>> data
>> > load of the system? I was also wondering how you store the data
>> internally.
>> > As in, what format is the data stored? Is it an internally created
>> > representation or one of the standard RDF representations?
>> >
>> >
>> > I would really appreciate your assistance in answering these questions
>> and
>> > look forward to hearing from you soon.
>> >
>> >
>> > Best Regards,
>> >
>> >
>> > Fredah
>> >
>> >
>> >
>> ------------------------------------------------------------------------------
>> > One dashboard for servers and applications across Physical-Virtual-Cloud
>> > Widest out-of-the-box monitoring support with 50+ applications
>> > Performance metrics, stats and reports that give you Actionable Insights
>> > Deep dive visibility with transaction tracing using APM Insight.
>> > http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
>> > _______________________________________________
>> > Bigdata-developers mailing list
>> > Big...@li...
>> > https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>> >
>>
>>
>> ------------------------------------------------------------------------------
>> One dashboard for servers and applications across Physical-Virtual-Cloud
>> Widest out-of-the-box monitoring support with 50+ applications
>> Performance metrics, stats and reports that give you Actionable Insights
>> Deep dive visibility with transaction tracing using APM Insight.
>> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
>> _______________________________________________
>> Bigdata-developers mailing list
>> Big...@li...
>> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>>
>
>
>
> --
> _______________
> Brad Bebee
> Managing Partner
> SYSTAP, LLC
> e:  be...@sy...
> m: 202.642.7961
> f:  571.367.5000
> w:  www.systap.com
>
> Blazegraph™ <http://www.blazegraph.com> is our ultra high-performance
> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
> APIs.  MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new
> technology to use GPUs to accelerate data-parallel graph analytics.
>
> CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
> for the sole use of the intended recipient(s) and are confidential or
> proprietary to SYSTAP, LLC.  Any unauthorized review, use, disclosure,
> dissemination or copying of this email or its contents or attachments is
> prohibited. If you have received this communication in error, please notify
> the sender by reply email and permanently delete all copies of the email
> and its contents and attachments.
>
>
> ------------------------------------------------------------------------------
> One dashboard for servers and applications across Physical-Virtual-Cloud
> Widest out-of-the-box monitoring support with 50+ applications
> Performance metrics, stats and reports that give you Actionable Insights
> Deep dive visibility with transaction tracing using APM Insight.
> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> _______________________________________________
> Bigdata-developers mailing list
> Big...@li...
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>
>


-- 
Best Regards,


Fredah Temwani Banda