Re: [Bigdata-developers] Requesting clarification on BigData internal structure with regards to compression

[Bryan T. <br...@sy...>]

Fredah,

Please see http://www.blazegraph.com/whitepapers/bigdata_architecture_whitepaper.pdf
for a good overview of the platform. The data are stored in indices
(B+Trees).  Prefix compression is used as well as a variety of other
compression techniques. We then encode the index pages into a
compressed representation that can be interpreted without
decompression of the page.  We do not currently compress the index
page records using a block compression strategy.  This is possible,
but there are other compression techniques that I would implement
first.

Thanks,
Bryan
----
Bryan Thompson
Chief Scientist & Founder
SYSTAP, LLC
4501 Tower Road
Greensboro, NC 27410
br...@sy...
http://blazegraph.com
http://blog.bigdata.com
http://mapgraph.io

Blazegraph™ is our ultra high-performance graph database that supports
both RDF/SPARQL and Tinkerpop/Blueprints APIs.  MapGraph™ is our
disruptive new technology to use GPUs to accelerate data-parallel
graph analytics.

CONFIDENTIALITY NOTICE:  This email and its contents and attachments
are for the sole use of the intended recipient(s) and are confidential
or proprietary to SYSTAP. Any unauthorized review, use, disclosure,
dissemination or copying of this email or its contents or attachments
is prohibited. If you have received this communication in error,
please notify the sender by reply email and permanently delete all
copies of the email and its contents and attachments.


On Mon, May 4, 2015 at 7:04 PM, Fredah Banda <fre...@ma...> wrote:
> Dear Team,
>
>
> I plan on using your SPARQL engine for my project implementation. I’m
> impressed by the tremendous work you have put in to make this engine a
> success however I did notice that the underlying infrastructure and
> compression technique used are encapsulated. I need to fully understand how
> the data is processed from start to finish especially with regards to the
> compression. Are there by any chance papers that have been written that
> cover the compression and decompression used in your engine or is it
> possible to refer me to someone who may be able to explain it to me?
>
>
> Also, is compression default or is turned on and off depending on the data
> load of the system? I was also wondering how you store the data internally.
> As in, what format is the data stored? Is it an internally created
> representation or one of the standard RDF representations?
>
>
> I would really appreciate your assistance in answering these questions and
> look forward to hearing from you soon.
>
>
> Best Regards,
>
>
> Fredah
>
>
> ------------------------------------------------------------------------------
> One dashboard for servers and applications across Physical-Virtual-Cloud
> Widest out-of-the-box monitoring support with 50+ applications
> Performance metrics, stats and reports that give you Actionable Insights
> Deep dive visibility with transaction tracing using APM Insight.
> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> _______________________________________________
> Bigdata-developers mailing list
> Big...@li...
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>