mpgraph-developers Mailing List for mapgraph

Massively Parallel Graph processing on GPUs -- now part of Blazegraph

Status: Beta

Brought to you by: beebs, hyandell, mrpersonick, thompsonbry

mpgraph-developers — developer discussions

You can subscribe to this list here.

2013	_Jan	_Feb	_Mar	_Apr	_May	_Jun	_Jul	_Aug (3)	_Sep	_Oct	_Nov	_Dec
2014	_Jan	_Feb	_Mar	_Apr	_May	_Jun	_Jul	_Aug	_Sep	_Oct (2)	_Nov (5)	_Dec
2015	_Jan (13)	_Feb (12)	_Mar (1)	_Apr (1)	_May	_Jun	_Jul (5)	_Aug (7)	_Sep	_Oct (1)	_Nov	_Dec

Flat | Threaded

1 2 > >> (Page 1 of 2)

[mpgraph-developers] some questions about MapGraph's gather operation

From: 张军昭 <zha...@tj...> - 2015-10-30 08:36:49

         Two month ago,i wrote a letter to ask something about MapGraph.i would like to thank james for his explanation.during this time i thought something about MapGraph and GAS model.


         Pregel is a graph model to deal with graph,it makes us to think like a vertex.i consider GAS model as a fine-parallel implementation of pregel.but,i find some shortcomings about GAS and MapGraph,i want to extend GAS model,so i wrote a letter again.


james said that:
gather_sum: defines the reduction operation used to reduce gathered values to a single value, only needs to be defined if the algorithm will perform a gatherleft and right are the two gather values to be reduces, the output is the reduced valuegather_edge: defines the operation used to produce a gather value for an edgevertex_id is the vertex id of the vertex being expanded from, edge_id is the id of the edge in the edgelist, neighbor_id_in is the vertex id of the vertex being expanded to, vertex_list and edge_list are references to the VertexType and EdgeType structs containing user defined data, new_value should be set with the gather value for the edge.


here is my question:
Q1:what function is gather_vertex? james not said thatQ2:can i modify gather_sum's reduce operation?and what should i do?pregel can send a messege to his neighbour vertex,if reduce operation's operators is just "+",that MapGraph cannot transfer a messege from one vertex to its neighbours,how can i modify operators "+" to others,for example "multiply","min","max" and so on?i need your help,looking forward to your reply!



Zhaojun Zhang
zha...@tj...

[mpgraph-developers] How as very small start-up can we employ and test MapGraph?

From: Marco I. <ipp...@gm...> - 2015-08-27 13:21:44

Hi all,
I'm really interested in testing, using and deploying MapGraph in the
application we are building.
So I would like to ask you how, as very small start-up with, per
definition, scarse starting resources, we can employ and test MapGraph
within our application.

Kind regards.
Marco

Re: [mpgraph-developers] something about how to Write your own MapGraph algorithm

From: Marco I. <ipp...@gm...> - 2015-08-27 13:06:20

Hi James,
thank you very much for your kind explanation.
I'm looking forward to try,test and deploy your very promising technologies.

But I need to ask another question, for which, in order to not be here
"off-topic", I'm going to write another email.

In the meanwhile, thanks again.
I'm sure, while getting through the technical aspects, I will ask you again
something more specific.
Kind regards.
Marco

2015-08-24 20:05 GMT+02:00 James Lewis <jl...@sy...>:

> Hello JunZhao and Marco,
>
> I will explain some about how to implement an algorithm in MapGraph, as
> well as answer the questions posed by JunZhao.
>
> The first step is to create a struct which provides the necessary
> functions and values which define your algorithm and give the MapGraph
> framework information about how to run your algorithm. See the bfs struct
> defined in bfs.h for an example. The struct includes the following:
>
>    - typedef statements which specify the types of values used
>       - DataType: the type used for frontier values, in algorithms that
>       add a value to the frontier
>       - GatherType: the type used in gather operations
>       - VertexId: the type to use for vertex identifiers
>       - SizeT: the type to use for sizes
>    - VertexType and EdgeType structs which define the data you want
>    associated with vertices and edges, which will be accessible to your
>    algorithm.
>       - VertexType is used for per vertex values
>       - EdgeType is used for per edge values
>    - An Initialize method, which is responsible for properly initializing
>    the data in the VertexType and EdgeType structs, as well as for setting the
>    initial frontier.
>       - The parameters nodes, edges, d_row_offsets, d_column_indices,
>       d_column_offsets, and d_row_indices provide the Initialize method with
>       access to the CSR and CSC representation of the graph the algorithm is
>       being run on. The parameters which are pointers are device pointers.
>       -  The parameter directed indicates whether the graph is being
>       treated as directed or undirected (it is treated as a bool)
>       - The parameters num_srcs and srcs indicate how many vertices are
>       initially active and which vertices they are respectively
>       - The parameters vertex_list and edge_list are references to the
>       VertexType and EdgeType structs which will be used. They need to be
>       initialized by allocating their memory (on the device) and setting the
>       values to what they should be given the source vertices and graph
>       information.
>       - The parameter d_frontier_keys provides access to the initial
>       frontier (the vertex ids of active vertices) the source vertices need to be
>       added to both the first and second arrays (the third array is for multi-GPU
>       and isn't used in single-GPU)
>       - The parameter d_frontier_values provides access to the initial
>       frontier values (the value associated with each vertex id in the frontier)
>       the initial values for the source vertices need to be set in the first and
>       second arrays (again the third is not used in single-GPU)
>    - Parameters for the MapGraph framework
>       - gatherOverEdges(): indicates whether a gather should be
>       performed, and if so whether it should be over in, out, or both edges.
>       - applyOverEdges(): indicates whether an apply should be performed,
>       and if so whether is should be over all vertices or only over vertices in
>       the frontier.
>       - postApplyOverEdges(): indicates whether a post-apply should be
>       performed, and if so whether is should be over all vertices or only over
>       vertices in the frontier.
>       - expandOverEdges(): indicates whether the expand phase should be
>       performed, and if so whether is should be over in, out, or both edges.
>    - Functors which define your algorithm
>       - gather_sum: defines the reduction operation used to reduce
>       gathered values to a single value, only needs to be defined if the
>       algorithm will perform a gather
>          - left and right are the two gather values to be reduces, the
>          output is the reduced value
>       - gather_edge: defines the operation used to produce a gather value
>       for an edge
>          - vertex_id is the vertex id of the vertex being expanded from,
>          edge_id is the id of the edge in the edgelist, neighbor_id_in is the vertex
>          id of the vertex being expanded to, vertex_list and edge_list are
>          references to the VertexType and EdgeType structs containing user defined
>          data, new_value should be set with the gather value for the edge.
>       - apply: defines the operation used when applying gathered values
>       to a vertex
>          - vertex_id is the id of the vertex, iteration is the number of
>          the current iteration, gathervalue is the reduced gather value for the
>          vertex (or a value set using the frontier value in the contract method),
>          vertex_list and edge_list provide a reference to the user defined data
>          structs, changed is used by expand_vertex to determine whether a vertex
>          should be expanded from.
>       - expand_vertex: this operation is called on each vertex in the
>       frontier to determine whether its neighbors should be added to the frontier.
>          - vertex_id is the id of the vertex, changed is the value set in
>          apply, edge_list and vertex_list provide access to the user defined data
>          structs
>       - expand_edge: this operation adds a vertex to the frontier along
>       with a value for each edge being expanded.
>          - changed is the value set in apply (for the source vertex),
>          iteration is the number of the current iteration, vertex_id is the id of
>          the vertex, neighbor_id_in is the id of the neighbor vertex, edge_id is the
>          id of the edge in the edge list, edge_list and vertex_list provide access
>          to the user defined data structs, frontier is set with the vertex id to add
>          to the frontier, misc_value  is set with the value to add to the frontier
>          values.
>       - contract: this operation is called after edges are expanded, it
>       allows for removing edges from the frontier before the next iteration
>          - iteration is the number of the current iteration, vertex_id is
>          the id of the vertex, edge_list and vertex_list provide access to the user
>          defined data structs, misc_value is the value in the frontier values,
>          gather_tmp provides access to the reduced gather values
>          - For algorithms that need to use a frontier value in their
>          apply method, the value in gather_tmp for the vertex should be set using
>          the frontier method, an atomic operation should be used as the vertex
>          operated on may be present in the frontier multiple times with different
>          frontier values
>
>
> In general an algorithm should perform a gather, or use frontier values,
> but not both.
>
> To start implementing your own algorithm I would suggest starting with a
> copy of one of the included algorithms and modifying it to perform your
> algorithm.
>
> The gather_sum for all algorithms returns left + right, but only PageRank
> performs a gather, so in the other algorithms the gather_sum is not used.
>
> I hope this answers your questions.
>
> Cheers,
>
> James
>
> ________________
> T. James Lewis
> CUDA Researcher
> Systap LLC
> jl...@sy...
> Tel: (801)-860-8752
>
> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs.
> MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new
> technology to use GPUs to accelerate data-parallel graph analytics.
>
> CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
> for the sole use of the intended recipient(s) and are confidential or
> proprietary to SYSTAP, LLC.  Any unauthorized review, use, disclosure,
> dissemination or copying of this email or its contents or attachments is
> prohibited. If you have received this communication in error, please notify
> the sender by reply email and permanently delete all copies of the email
> and its contents and attachments.
>
> On Sat, Aug 22, 2015 at 8:18 AM, Bryan Thompson <br...@sy...> wrote:
>
>> Hello, both James and I have been traveling this week. We will try to
>> respond to your questions next week.
>>
>> Thanks,
>> Bryan
>>
>>
>> On Saturday, August 22, 2015, Zhisong Fu <fuz...@sy...> wrote:
>>
>>> I'm sorry for the late replay. I am not working on the project anymore.
>>> I suggest you contact Bryan for this.
>>>
>>> Zhisong
>>>
>>>
>>>
>>> 在 2015年8月22日，18:12，Marco Ippolito <ipp...@gm...> 写道：
>>>
>>> Hi all,
>>> like JunZhao I ' m interested in MapGraph but I'm experiencing the same
>>> problems. Is there a thoughrogh explained example?
>>> Kind regards.
>>> Marco
>>> Il 20/ago/2015 14:44, "张军昭" <zha...@tj...> ha scritto:
>>>
>>>>
>>>>
>>>>   My name is JunZhao Zhang,I am a graduate student in tianjin
>>>> university,Tianjin,China.My research is semantic web information
>>>> management.recently,I have read your paper *MapGraph: A High Level API
>>>> for Fast Development of High Performance Graph Analytics on GPUs
>>>> <http://./papers/MapGraph-SIGMOD-2014.pdf>,* and i am very interested
>>>> in your job.I think cuda maybe a efficient way to deal with graph，i want to
>>>> use MapGraph's high level api to write a new algorithm，but i am faced
>>>> with some problem,so i write this letter to you.
>>>>
>>>>   Q1:Initialize(const int directed, const int nodes, const int edges,
>>>> int num_srcs, int* srcs, int* d_row_offsets, int* d_column_indices,
>>>> int* d_column_offsets, int* d_row_indices, DataType* d_edge_values, VertexType
>>>> &vertex_list, EdgeType &edge_list, int* d_frontier_keys[3], MiscType*
>>>> d_frontier_values[3]). The function has so many parameters and i feel
>>>> confused.i wander  what "MiscType* d_frontier_values[3]" in this used
>>>> for?why d_frontier_values's length is 3?
>>>>
>>>>    Q2:in struct gather_edge,the operator's parameter new_value,what is
>>>> its function? what's more,the new_value here,whether its value is equal to
>>>> the gathervlaue in struct apply?
>>>>
>>>>   Q3:in struct gather_sum,all algorithm's return value is
>>>> right+left,why?
>>>>
>>>>   Q4:MapGraph's doc is so simple that i can't figure out how to write a
>>>> algorithm,can you email me some detail information？
>>>>
>>>>   I am a new researcher  in GPU and cuda,my question maybe naive to
>>>> you.but i need your help,looking forward to your reply！thank you.
>>>>
>>>> JunZhao zhang
>>>> Tianjin university,Tianjin,China
>>>> zha...@tj...
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>>
>>>> _______________________________________________
>>>> mpgraph-developers mailing list
>>>> mpg...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/mpgraph-developers
>>>>
>>>>
>>> ------------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> mpgraph-developers mailing list
>>> mpg...@li...
>>> https://lists.sourceforge.net/lists/listinfo/mpgraph-developers
>>>
>>>
>>
>> --
>> ----
>> Bryan Thompson
>> Chief Scientist & Founder
>> SYSTAP, LLC
>> 4501 Tower Road
>> Greensboro, NC 27410
>> br...@sy...
>> http://blazegraph.com
>> http://blog.bigdata.com <http://bigdata.com>
>> http://mapgraph.io
>>
>> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
>> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
>> APIs.  MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new
>> technology to use GPUs to accelerate data-parallel graph analytics.
>>
>> <http://smartdata2015.dataversity.net/>
>>
>> CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
>> for the sole use of the intended recipient(s) and are confidential or
>> proprietary to SYSTAP. Any unauthorized review, use, disclosure,
>> dissemination or copying of this email or its contents or attachments is
>> prohibited. If you have received this communication in error, please notify
>> the sender by reply email and permanently delete all copies of the email
>> and its contents and attachments.
>>
>>
>

Re: [mpgraph-developers] something about how to Write your own MapGraph algorithm

From: James L. <jl...@sy...> - 2015-08-24 18:34:28

Hello JunZhao and Marco,

I will explain some about how to implement an algorithm in MapGraph, as
well as answer the questions posed by JunZhao.

The first step is to create a struct which provides the necessary functions
and values which define your algorithm and give the MapGraph framework
information about how to run your algorithm. See the bfs struct defined in
bfs.h for an example. The struct includes the following:

   - typedef statements which specify the types of values used
      - DataType: the type used for frontier values, in algorithms that add
      a value to the frontier
      - GatherType: the type used in gather operations
      - VertexId: the type to use for vertex identifiers
      - SizeT: the type to use for sizes
   - VertexType and EdgeType structs which define the data you want
   associated with vertices and edges, which will be accessible to your
   algorithm.
      - VertexType is used for per vertex values
      - EdgeType is used for per edge values
   - An Initialize method, which is responsible for properly initializing
   the data in the VertexType and EdgeType structs, as well as for setting the
   initial frontier.
      - The parameters nodes, edges, d_row_offsets, d_column_indices,
      d_column_offsets, and d_row_indices provide the Initialize method with
      access to the CSR and CSC representation of the graph the algorithm is
      being run on. The parameters which are pointers are device pointers.
      -  The parameter directed indicates whether the graph is being
      treated as directed or undirected (it is treated as a bool)
      - The parameters num_srcs and srcs indicate how many vertices are
      initially active and which vertices they are respectively
      - The parameters vertex_list and edge_list are references to the
      VertexType and EdgeType structs which will be used. They need to be
      initialized by allocating their memory (on the device) and setting the
      values to what they should be given the source vertices and graph
      information.
      - The parameter d_frontier_keys provides access to the initial
      frontier (the vertex ids of active vertices) the source vertices
need to be
      added to both the first and second arrays (the third array is
for multi-GPU
      and isn't used in single-GPU)
      - The parameter d_frontier_values provides access to the initial
      frontier values (the value associated with each vertex id in the
frontier)
      the initial values for the source vertices need to be set in the
first and
      second arrays (again the third is not used in single-GPU)
   - Parameters for the MapGraph framework
      - gatherOverEdges(): indicates whether a gather should be performed,
      and if so whether it should be over in, out, or both edges.
      - applyOverEdges(): indicates whether an apply should be performed,
      and if so whether is should be over all vertices or only over vertices in
      the frontier.
      - postApplyOverEdges(): indicates whether a post-apply should be
      performed, and if so whether is should be over all vertices or only over
      vertices in the frontier.
      - expandOverEdges(): indicates whether the expand phase should be
      performed, and if so whether is should be over in, out, or both edges.
   - Functors which define your algorithm
      - gather_sum: defines the reduction operation used to reduce gathered
      values to a single value, only needs to be defined if the algorithm will
      perform a gather
         - left and right are the two gather values to be reduces, the
         output is the reduced value
      - gather_edge: defines the operation used to produce a gather value
      for an edge
         - vertex_id is the vertex id of the vertex being expanded from,
         edge_id is the id of the edge in the edgelist, neighbor_id_in
is the vertex
         id of the vertex being expanded to, vertex_list and edge_list are
         references to the VertexType and EdgeType structs containing
user defined
         data, new_value should be set with the gather value for the edge.
      - apply: defines the operation used when applying gathered values to
      a vertex
         - vertex_id is the id of the vertex, iteration is the number of
         the current iteration, gathervalue is the reduced gather value for the
         vertex (or a value set using the frontier value in the
contract method),
         vertex_list and edge_list provide a reference to the user defined data
         structs, changed is used by expand_vertex to determine
whether a vertex
         should be expanded from.
      - expand_vertex: this operation is called on each vertex in the
      frontier to determine whether its neighbors should be added to
the frontier.
         - vertex_id is the id of the vertex, changed is the value set in
         apply, edge_list and vertex_list provide access to the user
defined data
         structs
      - expand_edge: this operation adds a vertex to the frontier along
      with a value for each edge being expanded.
         - changed is the value set in apply (for the source vertex),
         iteration is the number of the current iteration, vertex_id
is the id of
         the vertex, neighbor_id_in is the id of the neighbor vertex,
edge_id is the
         id of the edge in the edge list, edge_list and vertex_list
provide access
         to the user defined data structs, frontier is set with the
vertex id to add
         to the frontier, misc_value  is set with the value to add to
the frontier
         values.
      - contract: this operation is called after edges are expanded, it
      allows for removing edges from the frontier before the next iteration
         - iteration is the number of the current iteration, vertex_id is
         the id of the vertex, edge_list and vertex_list provide
access to the user
         defined data structs, misc_value is the value in the frontier values,
         gather_tmp provides access to the reduced gather values
         - For algorithms that need to use a frontier value in their apply
         method, the value in gather_tmp for the vertex should be set using the
         frontier method, an atomic operation should be used as the
vertex operated
         on may be present in the frontier multiple times with
different frontier
         values


In general an algorithm should perform a gather, or use frontier values,
but not both.

To start implementing your own algorithm I would suggest starting with a
copy of one of the included algorithms and modifying it to perform your
algorithm.

The gather_sum for all algorithms returns left + right, but only PageRank
performs a gather, so in the other algorithms the gather_sum is not used.

I hope this answers your questions.

Cheers,

James

________________
T. James Lewis
CUDA Researcher
Systap LLC
jl...@sy...
Tel: (801)-860-8752

Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs.
MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new technology
to use GPUs to accelerate data-parallel graph analytics.

CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
for the sole use of the intended recipient(s) and are confidential or
proprietary to SYSTAP, LLC.  Any unauthorized review, use, disclosure,
dissemination or copying of this email or its contents or attachments is
prohibited. If you have received this communication in error, please notify
the sender by reply email and permanently delete all copies of the email
and its contents and attachments.

On Sat, Aug 22, 2015 at 8:18 AM, Bryan Thompson <br...@sy...> wrote:

> Hello, both James and I have been traveling this week. We will try to
> respond to your questions next week.
>
> Thanks,
> Bryan
>
>
> On Saturday, August 22, 2015, Zhisong Fu <fuz...@sy...> wrote:
>
>> I'm sorry for the late replay. I am not working on the project anymore. I
>> suggest you contact Bryan for this.
>>
>> Zhisong
>>
>>
>>
>> 在 2015年8月22日，18:12，Marco Ippolito <ipp...@gm...> 写道：
>>
>> Hi all,
>> like JunZhao I ' m interested in MapGraph but I'm experiencing the same
>> problems. Is there a thoughrogh explained example?
>> Kind regards.
>> Marco
>> Il 20/ago/2015 14:44, "张军昭" <zha...@tj...> ha scritto:
>>
>>>
>>>
>>>   My name is JunZhao Zhang,I am a graduate student in tianjin
>>> university,Tianjin,China.My research is semantic web information
>>> management.recently,I have read your paper *MapGraph: A High Level API
>>> for Fast Development of High Performance Graph Analytics on GPUs
>>> <http://./papers/MapGraph-SIGMOD-2014.pdf>,* and i am very interested
>>> in your job.I think cuda maybe a efficient way to deal with graph，i want to
>>> use MapGraph's high level api to write a new algorithm，but i am faced
>>> with some problem,so i write this letter to you.
>>>
>>>   Q1:Initialize(const int directed, const int nodes, const int edges,
>>> int num_srcs, int* srcs, int* d_row_offsets, int* d_column_indices,
>>> int* d_column_offsets, int* d_row_indices, DataType* d_edge_values, VertexType
>>> &vertex_list, EdgeType &edge_list, int* d_frontier_keys[3], MiscType*
>>> d_frontier_values[3]). The function has so many parameters and i feel
>>> confused.i wander  what "MiscType* d_frontier_values[3]" in this used
>>> for?why d_frontier_values's length is 3?
>>>
>>>    Q2:in struct gather_edge,the operator's parameter new_value,what is
>>> its function? what's more,the new_value here,whether its value is equal to
>>> the gathervlaue in struct apply?
>>>
>>>   Q3:in struct gather_sum,all algorithm's return value is right+left,why?
>>>
>>>   Q4:MapGraph's doc is so simple that i can't figure out how to write a
>>> algorithm,can you email me some detail information？
>>>
>>>   I am a new researcher  in GPU and cuda,my question maybe naive to
>>> you.but i need your help,looking forward to your reply！thank you.
>>>
>>> JunZhao zhang
>>> Tianjin university,Tianjin,China
>>> zha...@tj...
>>>
>>>
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> mpgraph-developers mailing list
>>> mpg...@li...
>>> https://lists.sourceforge.net/lists/listinfo/mpgraph-developers
>>>
>>>
>> ------------------------------------------------------------------------------
>>
>> _______________________________________________
>> mpgraph-developers mailing list
>> mpg...@li...
>> https://lists.sourceforge.net/lists/listinfo/mpgraph-developers
>>
>>
>
> --
> ----
> Bryan Thompson
> Chief Scientist & Founder
> SYSTAP, LLC
> 4501 Tower Road
> Greensboro, NC 27410
> br...@sy...
> http://blazegraph.com
> http://blog.bigdata.com <http://bigdata.com>
> http://mapgraph.io
>
> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
> APIs.  MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new
> technology to use GPUs to accelerate data-parallel graph analytics.
>
> <http://smartdata2015.dataversity.net/>
>
> CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
> for the sole use of the intended recipient(s) and are confidential or
> proprietary to SYSTAP. Any unauthorized review, use, disclosure,
> dissemination or copying of this email or its contents or attachments is
> prohibited. If you have received this communication in error, please notify
> the sender by reply email and permanently delete all copies of the email
> and its contents and attachments.
>
>

Re: [mpgraph-developers] something about how to Write your own MapGraph algorithm

From: Bryan T. <br...@sy...> - 2015-08-22 12:18:53

Hello, both James and I have been traveling this week. We will try to
respond to your questions next week.

Thanks,
Bryan

On Saturday, August 22, 2015, Zhisong Fu <fuz...@sy...> wrote:

> I'm sorry for the late replay. I am not working on the project anymore. I
> suggest you contact Bryan for this.
>
> Zhisong
>
>
>
> 在 2015年8月22日，18:12，Marco Ippolito <ipp...@gm...
> <javascript:_e(%7B%7D,'cvml','ipp...@gm...');>> 写道：
>
> Hi all,
> like JunZhao I ' m interested in MapGraph but I'm experiencing the same
> problems. Is there a thoughrogh explained example?
> Kind regards.
> Marco
> Il 20/ago/2015 14:44, "张军昭" <zha...@tj...
> <javascript:_e(%7B%7D,'cvml','zha...@tj...');>> ha scritto:
>
>>
>>
>>   My name is JunZhao Zhang,I am a graduate student in tianjin
>> university,Tianjin,China.My research is semantic web information
>> management.recently,I have read your paper *MapGraph: A High Level API
>> for Fast Development of High Performance Graph Analytics on GPUs
>> <http://./papers/MapGraph-SIGMOD-2014.pdf>,* and i am very interested in
>> your job.I think cuda maybe a efficient way to deal with graph，i want to
>> use MapGraph's high level api to write a new algorithm，but i am faced
>> with some problem,so i write this letter to you.
>>
>>   Q1:Initialize(const int directed, const int nodes, const int edges, int
>> num_srcs, int* srcs, int* d_row_offsets, int* d_column_indices, int*
>> d_column_offsets, int* d_row_indices, DataType* d_edge_values, VertexType
>> &vertex_list, EdgeType &edge_list, int* d_frontier_keys[3], MiscType*
>> d_frontier_values[3]). The function has so many parameters and i feel
>> confused.i wander  what "MiscType* d_frontier_values[3]" in this used
>> for?why d_frontier_values's length is 3?
>>
>>    Q2:in struct gather_edge,the operator's parameter new_value,what is
>> its function? what's more,the new_value here,whether its value is equal to
>> the gathervlaue in struct apply?
>>
>>   Q3:in struct gather_sum,all algorithm's return value is right+left,why?
>>
>>   Q4:MapGraph's doc is so simple that i can't figure out how to write a
>> algorithm,can you email me some detail information？
>>
>>   I am a new researcher  in GPU and cuda,my question maybe naive to
>> you.but i need your help,looking forward to your reply！thank you.
>>
>> JunZhao zhang
>> Tianjin university,Tianjin,China
>> zha...@tj...
>> <javascript:_e(%7B%7D,'cvml','zha...@tj...');>
>>
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>>
>> _______________________________________________
>> mpgraph-developers mailing list
>> mpg...@li...
>> <javascript:_e(%7B%7D,'cvml','mpg...@li...');>
>> https://lists.sourceforge.net/lists/listinfo/mpgraph-developers
>>
>>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> mpgraph-developers mailing list
> mpg...@li...
> <javascript:_e(%7B%7D,'cvml','mpg...@li...');>
> https://lists.sourceforge.net/lists/listinfo/mpgraph-developers
>
>

-- 
----
Bryan Thompson
Chief Scientist & Founder
SYSTAP, LLC
4501 Tower Road
Greensboro, NC 27410
br...@sy...
http://blazegraph.com
http://blog.bigdata.com <http://bigdata.com>
http://mapgraph.io

Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
APIs.  MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new
technology to use GPUs to accelerate data-parallel graph analytics.

<http://smartdata2015.dataversity.net/>

CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
for the sole use of the intended recipient(s) and are confidential or
proprietary to SYSTAP. Any unauthorized review, use, disclosure,
dissemination or copying of this email or its contents or attachments is
prohibited. If you have received this communication in error, please notify
the sender by reply email and permanently delete all copies of the email
and its contents and attachments.

Re: [mpgraph-developers] something about how to Write your own MapGraph algorithm

From: Zhisong Fu <fuz...@sy...> - 2015-08-22 12:04:42

I'm sorry for the late replay. I am not working on the project anymore. I suggest you contact Bryan for this. 

Zhisong 



> 在 2015年8月22日，18:12，Marco Ippolito <ipp...@gm...> 写道：
> 
> Hi all,
> like JunZhao I ' m interested in MapGraph but I'm experiencing the same problems. Is there a thoughrogh explained example?
> Kind regards.
> Marco
> 
> Il 20/ago/2015 14:44, "张军昭" <zha...@tj...> ha scritto:
>> 
>> 
>>   My name is JunZhao Zhang,I am a graduate student in tianjin university,Tianjin,China.My research is semantic web information management.recently,I have read your paper MapGraph: A High Level API for Fast Development of High Performance Graph Analytics on GPUs, and i am very interested in your job.I think cuda maybe a efficient way to deal with graph，i want to use MapGraph's high level api to write a new algorithm，but i am faced with some problem,so i write this letter to you.
>> 
>>   Q1:Initialize(const int directed, const int nodes, const int edges, int num_srcs, int* srcs, int* d_row_offsets, int* d_column_indices, int* d_column_offsets, int* d_row_indices, DataType* d_edge_values, VertexType &vertex_list, EdgeType &edge_list, int* d_frontier_keys[3], MiscType* d_frontier_values[3]). The function has so many parameters and i feel confused.i wander  what "MiscType* d_frontier_values[3]" in this used for?why d_frontier_values's length is 3?
>> 
>>    Q2:in struct gather_edge,the operator's parameter new_value,what is its function? what's more,the new_value here,whether its value is equal to the gathervlaue in struct apply?
>> 
>>   Q3:in struct gather_sum,all algorithm's return value is right+left,why?
>> 
>>   Q4:MapGraph's doc is so simple that i can't figure out how to write a algorithm,can you email me some detail information？
>> 
>>   I am a new researcher  in GPU and cuda,my question maybe naive to you.but i need your help,looking forward to your reply！thank you.
>> 
>> JunZhao zhang
>> Tianjin university,Tianjin,China
>> zha...@tj...
>>   
>> 
>> 
>> 
>> ------------------------------------------------------------------------------
>> 
>> _______________________________________________
>> mpgraph-developers mailing list
>> mpg...@li...
>> https://lists.sourceforge.net/lists/listinfo/mpgraph-developers
> ------------------------------------------------------------------------------
> _______________________________________________
> mpgraph-developers mailing list
> mpg...@li...
> https://lists.sourceforge.net/lists/listinfo/mpgraph-developers

Re: [mpgraph-developers] something about how to Write your own MapGraph algorithm

From: Marco I. <ipp...@gm...> - 2015-08-22 10:12:54

Hi all,
like JunZhao I ' m interested in MapGraph but I'm experiencing the same
problems. Is there a thoughrogh explained example?
Kind regards.
Marco
Il 20/ago/2015 14:44, "张军昭" <zha...@tj...> ha scritto:

>
>
>   My name is JunZhao Zhang,I am a graduate student in tianjin
> university,Tianjin,China.My research is semantic web information
> management.recently,I have read your paper *MapGraph: A High Level API
> for Fast Development of High Performance Graph Analytics on GPUs
> <http://./papers/MapGraph-SIGMOD-2014.pdf>,* and i am very interested in
> your job.I think cuda maybe a efficient way to deal with graph，i want to
> use MapGraph's high level api to write a new algorithm，but i am faced
> with some problem,so i write this letter to you.
>
>   Q1:Initialize(const int directed, const int nodes, const int edges, int
> num_srcs, int* srcs, int* d_row_offsets, int* d_column_indices, int*
> d_column_offsets, int* d_row_indices, DataType* d_edge_values, VertexType
> &vertex_list, EdgeType &edge_list, int* d_frontier_keys[3], MiscType*
> d_frontier_values[3]). The function has so many parameters and i feel
> confused.i wander  what "MiscType* d_frontier_values[3]" in this used
> for?why d_frontier_values's length is 3?
>
>    Q2:in struct gather_edge,the operator's parameter new_value,what is
> its function? what's more,the new_value here,whether its value is equal to
> the gathervlaue in struct apply?
>
>   Q3:in struct gather_sum,all algorithm's return value is right+left,why?
>
>   Q4:MapGraph's doc is so simple that i can't figure out how to write a
> algorithm,can you email me some detail information？
>
>   I am a new researcher  in GPU and cuda,my question maybe naive to
> you.but i need your help,looking forward to your reply！thank you.
>
> JunZhao zhang
> Tianjin university,Tianjin,China
> zha...@tj...
>
>
>
>
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> mpgraph-developers mailing list
> mpg...@li...
> https://lists.sourceforge.net/lists/listinfo/mpgraph-developers
>
>

[mpgraph-developers] something about how to Write your own MapGraph algorithm

From: 张军昭 <zha...@tj...> - 2015-08-20 12:44:14


  My name is JunZhao Zhang,I am a graduate student in tianjin university,Tianjin,China.My research is semantic web information management.recently,I have read your paper MapGraph: A High Level API for Fast Development of High Performance Graph Analytics on GPUs, and i am very interested in your job.I think cuda maybe a efficient way to deal with graph，i want to use MapGraph's high level api to write a new algorithm，but i am faced with some problem,so i write this letter to you.


  Q1:Initialize(const int directed, const int nodes, const int edges, int num_srcs, int* srcs, int* d_row_offsets, int* d_column_indices, int* d_column_offsets, int* d_row_indices, DataType* d_edge_values, VertexType &vertex_list, EdgeType &edge_list, int* d_frontier_keys[3], MiscType* d_frontier_values[3]). The function has so many parameters and i feel confused.i wander  what "MiscType* d_frontier_values[3]" in this used for?why d_frontier_values's length is 3?


   Q2:in struct gather_edge,the operator's parameter new_value,what is its function? what's more,the new_value here,whether its value is equal to the gathervlaue in struct apply?


  Q3:in struct gather_sum,all algorithm's return value is right+left,why?


  Q4:MapGraph's doc is so simple that i can't figure out how to write a algorithm,can you email me some detail information？


  I am a new researcher  in GPU and cuda,my question maybe naive to you.but i need your help,looking forward to your reply！thank you.


JunZhao zhang
Tianjin university,Tianjin,China
zha...@tj...

Re: [mpgraph-developers] Problem with increasing the no of iterations in pagerank

From: James L. <jl...@sy...> - 2015-07-07 19:24:31

It makes sense that it would take longer with the comparison value set
lower. During each iteration only vertices which have had their value
changed are evaluated. When it is set to 0.01f and converges in  60
iterations, most of the vertices have already converged much earlier than
the 60th iteration and so less work is being done per iteration. When it is
set to 0.00001f it is likely that almost all the vertices are being
evaluated for all 60 iterations, as it isn't close to actually converging
then, so in each iteration more work is done and the runtime is higher.

Cheers,

James

________________
James Lewis
CUDA Researcher
Systap LLC
jl...@sy...
Tel: (801)-860-8752

Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs.
MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new technology
to use GPUs to accelerate data-parallel graph analytics.

CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
for the sole use of the intended recipient(s) and are confidential or
proprietary to SYSTAP, LLC.  Any unauthorized review, use, disclosure,
dissemination or copying of this email or its contents or attachments is
prohibited. If you have received this communication in error, please notify
the sender by reply email and permanently delete all copies of the email
and its contents and attachments.

On Tue, Jul 7, 2015 at 4:28 AM, ashutosh raghuvanshi <
ash...@gm...> wrote:

> I made the change
>
> if (fabs(oldvalue - newvalue) < 0.01f)
>         changed = 0;
>
> to:
>
> if (fabs(oldvalue - newvalue) < 0.00001f)
>         changed = 0;
>
> earlier it was conversing on 60 iterations on LiveJounral and took
> 2seconds to do so, now it converse on much more iterations but when with
> fabs(oldvalue - newvalue) < 0.00001f I limit the iterations to 60 by -p
> iter_num=60 it takes 4.7 seconds can you help me with why this happens?
>
> On Sun, Jul 5, 2015 at 12:57 PM, ashutosh raghuvanshi <
> ash...@gm...> wrote:
>
>> Thanks James Lewis and Bryan Thomson, it worked after making changed in
>> Pagerank.h file
>>
>> Ashutosh
>>
>> On Sat, Jul 4, 2015 at 9:17 PM, James Lewis <jl...@sy...> wrote:
>>
>>> If you want to increase the accuracy of the result, you want to change
>>> the value used to consider an element changed. This is found in the apply
>>> struct defined in PageRank.h:
>>>
>>> if (fabs(oldvalue - newvalue) < 0.01f)
>>>         changed = 0;
>>>
>>> Changing the 0.01f to a lower value will increase the accuracy of the
>>> result, and require more iterations to converge. You may also need to
>>> specify a higher value for iter_num so that it won't terminate before it
>>> converges.
>>>
>>> Cheers,
>>>
>>> James
>>>
>>> ________________
>>> James Lewis
>>> CUDA Researcher
>>> Systap LLC
>>> jl...@sy...
>>> Tel: (801)-860-8752
>>>
>>> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
>>> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs.
>>> MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new
>>> technology to use GPUs to accelerate data-parallel graph analytics.
>>>
>>> CONFIDENTIALITY NOTICE:  This email and its contents and attachments
>>> are for the sole use of the intended recipient(s) and are confidential or
>>> proprietary to SYSTAP, LLC.  Any unauthorized review, use, disclosure,
>>> dissemination or copying of this email or its contents or attachments is
>>> prohibited. If you have received this communication in error, please notify
>>> the sender by reply email and permanently delete all copies of the email
>>> and its contents and attachments.
>>>
>>> On Sat, Jul 4, 2015 at 7:27 AM, ashutosh raghuvanshi <
>>> ash...@gm...> wrote:
>>>
>>>>  Hello, I am working on a research on to taking graph processing and
>>>> graph database processing on to GPGPU ans I am using Mapgraph against other
>>>> applicatons on multicore CPU and CPU clustures.
>>>>
>>>> I need help in the proccess.
>>>>
>>>> If i run Pagerank on soc-livejournal1 dataset it completes in 60
>>>> iterations but if I want to run it for more number of times then what to do?
>>>> -p iter_num=50 (any no. less than 60) works but -p iter_num=70 (any no.
>>>> greater than 60 doesn't work)
>>>>
>>>> even tol=0.1 is only for checking...that also doesn't go for more
>>>> accurate results.
>>>>
>>>> so please help ASAP.
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Don't Limit Your Business. Reach for the Cloud.
>>>> GigeNET's Cloud Solutions provide you with the tools and support that
>>>> you need to offload your IT needs and focus on growing your business.
>>>> Configured For All Businesses. Start Your Cloud Today.
>>>> https://www.gigenetcloud.com/
>>>> _______________________________________________
>>>> mpgraph-developers mailing list
>>>> mpg...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/mpgraph-developers
>>>>
>>>>
>>>
>>
>

Re: [mpgraph-developers] Problem with increasing the no of iterations in pagerank

From: ashutosh r. <ash...@gm...> - 2015-07-05 07:27:08

Thanks James Lewis and Bryan Thomson, it worked after making changed in
Pagerank.h file

Ashutosh

On Sat, Jul 4, 2015 at 9:17 PM, James Lewis <jl...@sy...> wrote:

> If you want to increase the accuracy of the result, you want to change the
> value used to consider an element changed. This is found in the apply
> struct defined in PageRank.h:
>
> if (fabs(oldvalue - newvalue) < 0.01f)
>         changed = 0;
>
> Changing the 0.01f to a lower value will increase the accuracy of the
> result, and require more iterations to converge. You may also need to
> specify a higher value for iter_num so that it won't terminate before it
> converges.
>
> Cheers,
>
> James
>
> ________________
> James Lewis
> CUDA Researcher
> Systap LLC
> jl...@sy...
> Tel: (801)-860-8752
>
> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs.
> MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new
> technology to use GPUs to accelerate data-parallel graph analytics.
>
> CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
> for the sole use of the intended recipient(s) and are confidential or
> proprietary to SYSTAP, LLC.  Any unauthorized review, use, disclosure,
> dissemination or copying of this email or its contents or attachments is
> prohibited. If you have received this communication in error, please notify
> the sender by reply email and permanently delete all copies of the email
> and its contents and attachments.
>
> On Sat, Jul 4, 2015 at 7:27 AM, ashutosh raghuvanshi <
> ash...@gm...> wrote:
>
>>  Hello, I am working on a research on to taking graph processing and
>> graph database processing on to GPGPU ans I am using Mapgraph against other
>> applicatons on multicore CPU and CPU clustures.
>>
>> I need help in the proccess.
>>
>> If i run Pagerank on soc-livejournal1 dataset it completes in 60
>> iterations but if I want to run it for more number of times then what to do?
>> -p iter_num=50 (any no. less than 60) works but -p iter_num=70 (any no.
>> greater than 60 doesn't work)
>>
>> even tol=0.1 is only for checking...that also doesn't go for more
>> accurate results.
>>
>> so please help ASAP.
>>
>>
>> ------------------------------------------------------------------------------
>> Don't Limit Your Business. Reach for the Cloud.
>> GigeNET's Cloud Solutions provide you with the tools and support that
>> you need to offload your IT needs and focus on growing your business.
>> Configured For All Businesses. Start Your Cloud Today.
>> https://www.gigenetcloud.com/
>> _______________________________________________
>> mpgraph-developers mailing list
>> mpg...@li...
>> https://lists.sourceforge.net/lists/listinfo/mpgraph-developers
>>
>>
>

Re: [mpgraph-developers] Problem with increasing the no of iterations in pagerank

From: James L. <jl...@sy...> - 2015-07-04 16:15:40

If you want to increase the accuracy of the result, you want to change the
value used to consider an element changed. This is found in the apply
struct defined in PageRank.h:

if (fabs(oldvalue - newvalue) < 0.01f)
        changed = 0;

Changing the 0.01f to a lower value will increase the accuracy of the
result, and require more iterations to converge. You may also need to
specify a higher value for iter_num so that it won't terminate before it
converges.

Cheers,

James

________________
James Lewis
CUDA Researcher
Systap LLC
jl...@sy...
Tel: (801)-860-8752

Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs.
MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new technology
to use GPUs to accelerate data-parallel graph analytics.

CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
for the sole use of the intended recipient(s) and are confidential or
proprietary to SYSTAP, LLC.  Any unauthorized review, use, disclosure,
dissemination or copying of this email or its contents or attachments is
prohibited. If you have received this communication in error, please notify
the sender by reply email and permanently delete all copies of the email
and its contents and attachments.

On Sat, Jul 4, 2015 at 7:27 AM, ashutosh raghuvanshi <
ash...@gm...> wrote:

>  Hello, I am working on a research on to taking graph processing and
> graph database processing on to GPGPU ans I am using Mapgraph against other
> applicatons on multicore CPU and CPU clustures.
>
> I need help in the proccess.
>
> If i run Pagerank on soc-livejournal1 dataset it completes in 60
> iterations but if I want to run it for more number of times then what to do?
> -p iter_num=50 (any no. less than 60) works but -p iter_num=70 (any no.
> greater than 60 doesn't work)
>
> even tol=0.1 is only for checking...that also doesn't go for more accurate
> results.
>
> so please help ASAP.
>
>
> ------------------------------------------------------------------------------
> Don't Limit Your Business. Reach for the Cloud.
> GigeNET's Cloud Solutions provide you with the tools and support that
> you need to offload your IT needs and focus on growing your business.
> Configured For All Businesses. Start Your Cloud Today.
> https://www.gigenetcloud.com/
> _______________________________________________
> mpgraph-developers mailing list
> mpg...@li...
> https://lists.sourceforge.net/lists/listinfo/mpgraph-developers
>
>

Re: [mpgraph-developers] Problem with increasing the no of iterations in pagerank

From: Bryan T. <br...@sy...> - 2015-07-04 12:04:58

Have you tried to modify pagerank.cu?

If the problem that it does not do more iterations or that the results do
not converge if you request more iterations?

The page rank implementation uses a delta rule. It terminates if the
frontier becomes empty regardless of the number of iterations specified.

Thanks,
Bryan

On Saturday, July 4, 2015, ashutosh raghuvanshi <
ash...@gm...> wrote:

>  Hello, I am working on a research on to taking graph processing and
> graph database processing on to GPGPU ans I am using Mapgraph against other
> applicatons on multicore CPU and CPU clustures.
>
> I need help in the proccess.
>
> If i run Pagerank on soc-livejournal1 dataset it completes in 60
> iterations but if I want to run it for more number of times then what to do?
> -p iter_num=50 (any no. less than 60) works but -p iter_num=70 (any no.
> greater than 60 doesn't work)
>
> even tol=0.1 is only for checking...that also doesn't go for more accurate
> results.
>
> so please help ASAP.
>

-- 
----
Bryan Thompson
Chief Scientist & Founder
SYSTAP, LLC
4501 Tower Road
Greensboro, NC 27410
br...@sy...
http://blazegraph.com
http://blog.bigdata.com <http://bigdata.com>
http://mapgraph.io

Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
APIs.  MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new
technology to use GPUs to accelerate data-parallel graph analytics.

<http://smartdata2015.dataversity.net/>

CONFIDENTIALITY NOTICE:  This email and its contents and attachments are
for the sole use of the intended recipient(s) and are confidential or
proprietary to SYSTAP. Any unauthorized review, use, disclosure,
dissemination or copying of this email or its contents or attachments is
prohibited. If you have received this communication in error, please notify
the sender by reply email and permanently delete all copies of the email
and its contents and attachments.

[mpgraph-developers] Problem with increasing the no of iterations in pagerank

From: ashutosh r. <ash...@gm...> - 2015-07-04 11:27:21

Hello, I am working on a research on to taking graph processing and graph
database processing on to GPGPU ans I am using Mapgraph against other
applicatons on multicore CPU and CPU clustures.

I need help in the proccess.

If i run Pagerank on soc-livejournal1 dataset it completes in 60 iterations
but if I want to run it for more number of times then what to do?
-p iter_num=50 (any no. less than 60) works but -p iter_num=70 (any no.
greater than 60 doesn't work)

even tol=0.1 is only for checking...that also doesn't go for more accurate
results.

so please help ASAP.