I'm a developer on the Arvados (http://arvados.org) project and we are
currently evaluating a migration to Bigdata from Postgres. Bigdata
looks like a really good fit to our needs, but there two key features
that we require, but available based on the documentation or searching
the mailing lists I have not been able to determine if they are
available:
a) Is there a security/permission system for queries, or hooks to
implement such a thing (e.g. filtering reads on a per-triple basis,
validating inserts/deletes)?
b) Is there a way to get notifications of change sets?
Thanks,
Peter
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The 2nd issue is the easiest, and can give you the means to solve the first issue as well. What you do is register a custom service with bigdata (as in a SPARQL federated query SERVICE uri {….}). The service will be able to observe all transaction starts and can choose to register a listener for mutation events on the KB instance (triples or quads, added or removed). This is addressed on the wiki. See the part of the page on "monitoring updates".
The 1st issue has been solved in many different ways. One approach is to register a service (as above). This will give you access to the change log and you can use that to secure add/remove of triples or quads. If you want to rewrite each triple pattern to a union with ACLS, you can either do this in the application or you can do this using an ASTOptimizer. Sometimes this is done by attaching a FILTER to the triple pattern. Sometimes it is done by joining against another triple pattern. Sometimes the additional data is maintained as statements about statements. Sometimes the additional data is maintained in a quads mode meta-graph or within the graph. Sometimes the resolution of the security information for the authenticated user is done at the start of the query and the set of named graphs is reduced to only those that are visible to that user, and this can be accelerated using a custom index from user to named graphs and their access privileges and you can maintain that custom index by registering a custom service as described above.
We have not bundled a specific security model into bigdata precisely because there are so many different ways that people approach this, and those differences seem to be driven by specific business requirements (security on the named graph, security on a document that does not correspond to a named graph, security on a triple, etc.).
Please let me know if you run into difficulty when trying to integrate a specific security model and I can try to point you to the right interfaces.
I'm a developer on the Arvados (http://arvados.org) project and we are
currently evaluating a migration to Bigdata from Postgres. Bigdata
looks like a really good fit to our needs, but there two key features
that we require, but available based on the documentation or searching
the mailing lists I have not been able to determine if they are
available:
a) Is there a security/permission system for queries, or hooks to
implement such a thing (e.g. filtering reads on a per-triple basis,
validating inserts/deletes)?
b) Is there a way to get notifications of change sets?
I already asked this question some time ago. As I understood it is up to us (developers) to implement security.
I personally use quad store and thus going to make security restrictions relying in contexts that are not hard to get. To get elements of query you can use native bigdata queries and get their abstract syntax tree from which you can get parts of query. For updates you can assign changelogger that will see what quads will be inserted or removed, You can try to pass closure there that will close trascation before it will be commited.
A lot of depends on how you organize working with bigdata in your app. Do users use it
a sparql endpoint or most of the usages are inside classes of your app or both?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Security is rather common question, I think it is even worth having a separate wiki page.
By the way, I looked into BigData InMemChangeLog, I see there is a resolve function. I wonder, is it really needed? In my changelog implementation I had no troubles with extracting rdf statements out of IChangeRecord after they were committed
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
That provides an efficient batch oriented resolution of the IVs to BigdataValues and the ISPOs to BigdataStatements. Whether you need that or not depends on whether you need the externalized openrdf object model. You can do this without batching, but batching is much more efficient.
Bryan
Security is rather common question, I think it is even worth having a separate wiki page.
By the way, I looked into BigData InMemChangeLog, I see there is a resolve function. I wonder, is it really needed? In my changelog implementation I had no troubles with extracting rdf statements out of IChangeRecord after they were committed
Hello,
I'm a developer on the Arvados (http://arvados.org) project and we are
currently evaluating a migration to Bigdata from Postgres. Bigdata
looks like a really good fit to our needs, but there two key features
that we require, but available based on the documentation or searching
the mailing lists I have not been able to determine if they are
available:
a) Is there a security/permission system for queries, or hooks to
implement such a thing (e.g. filtering reads on a per-triple basis,
validating inserts/deletes)?
b) Is there a way to get notifications of change sets?
Thanks,
Peter
Peter,
The 2nd issue is the easiest, and can give you the means to solve the first issue as well. What you do is register a custom service with bigdata (as in a SPARQL federated query SERVICE uri {….}). The service will be able to observe all transaction starts and can choose to register a listener for mutation events on the KB instance (triples or quads, added or removed). This is addressed on the wiki. See the part of the page on "monitoring updates".
The 1st issue has been solved in many different ways. One approach is to register a service (as above). This will give you access to the change log and you can use that to secure add/remove of triples or quads. If you want to rewrite each triple pattern to a union with ACLS, you can either do this in the application or you can do this using an ASTOptimizer. Sometimes this is done by attaching a FILTER to the triple pattern. Sometimes it is done by joining against another triple pattern. Sometimes the additional data is maintained as statements about statements. Sometimes the additional data is maintained in a quads mode meta-graph or within the graph. Sometimes the resolution of the security information for the authenticated user is done at the start of the query and the set of named graphs is reduced to only those that are visible to that user, and this can be accelerated using a custom index from user to named graphs and their access privileges and you can maintain that custom index by registering a custom service as described above.
We have not bundled a specific security model into bigdata precisely because there are so many different ways that people approach this, and those differences seem to be driven by specific business requirements (security on the named graph, security on a document that does not correspond to a named graph, security on a triple, etc.).
Please let me know if you run into difficulty when trying to integrate a specific security model and I can try to point you to the right interfaces.
Thanks,
Bryan
From: Peter Amstutz tetron@users.sf.netamp#116;amp#101;amp#116;amp#114;amp#111;amp#110;amp#64;amp#117;amp#115;amp#101;amp#114;amp#115;amp#46;amp#115;amp#102;amp#46;amp#110;amp#101;amp#116;
Reply-To: "[bigdata:discussion]" 676946@discussion.bigdata.p.re.sf.netamp#54;amp#55;amp#54;amp#57;amp#52;amp#54;amp#64;amp#100;amp#105;amp#115;amp#99;amp#117;amp#115;amp#115;amp#105;amp#111;amp#110;amp#46;amp#98;amp#105;amp#103;amp#100;amp#97;amp#116;amp#97;amp#46;amp#112;amp#46;amp#114;amp#101;amp#46;amp#115;amp#102;amp#46;amp#110;amp#101;amp#116;
Date: Thursday, March 6, 2014 10:19 PM
To: "[bigdata:discussion]" 676946@discussion.bigdata.p.re.sf.netamp#54;amp#55;amp#54;amp#57;amp#52;amp#54;amp#64;amp#100;amp#105;amp#115;amp#99;amp#117;amp#115;amp#115;amp#105;amp#111;amp#110;amp#46;amp#98;amp#105;amp#103;amp#100;amp#97;amp#116;amp#97;amp#46;amp#112;amp#46;amp#114;amp#101;amp#46;amp#115;amp#102;amp#46;amp#110;amp#101;amp#116;
Subject: [bigdata:discussion] permissions, database notification features?
Hello,
I'm a developer on the Arvados (http://arvados.org) project and we are
currently evaluating a migration to Bigdata from Postgres. Bigdata
looks like a really good fit to our needs, but there two key features
that we require, but available based on the documentation or searching
the mailing lists I have not been able to determine if they are
available:
a) Is there a security/permission system for queries, or hooks to
implement such a thing (e.g. filtering reads on a per-triple basis,
validating inserts/deletes)?
b) Is there a way to get notifications of change sets?
Thanks,
Peter
permissions, database notification features?http://sourceforge.net/p/bigdata/discussion/676946/thread/0c67fe7b/?limit=25#51bc
Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/bigdata/discussion/676946/
To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/
I already asked this question some time ago. As I understood it is up to us (developers) to implement security.
I personally use quad store and thus going to make security restrictions relying in contexts that are not hard to get. To get elements of query you can use native bigdata queries and get their abstract syntax tree from which you can get parts of query. For updates you can assign changelogger that will see what quads will be inserted or removed, You can try to pass closure there that will close trascation before it will be commited.
A lot of depends on how you organize working with bigdata in your app. Do users use it
a sparql endpoint or most of the usages are inside classes of your app or both?
Security is rather common question, I think it is even worth having a separate wiki page.
By the way, I looked into BigData InMemChangeLog, I see there is a resolve function. I wonder, is it really needed? In my changelog implementation I had no troubles with extracting rdf statements out of IChangeRecord after they were committed
That provides an efficient batch oriented resolution of the IVs to BigdataValues and the ISPOs to BigdataStatements. Whether you need that or not depends on whether you need the externalized openrdf object model. You can do this without batching, but batching is much more efficient.
Bryan
From: Anton Kulaga antonkulaga@users.sf.netamp#97;amp#110;amp#116;amp#111;amp#110;amp#107;amp#117;amp#108;amp#97;amp#103;amp#97;amp#64;amp#117;amp#115;amp#101;amp#114;amp#115;amp#46;amp#115;amp#102;amp#46;amp#110;amp#101;amp#116;
Reply-To: "[bigdata:discussion]" 676946@discussion.bigdata.p.re.sf.netamp#54;amp#55;amp#54;amp#57;amp#52;amp#54;amp#64;amp#100;amp#105;amp#115;amp#99;amp#117;amp#115;amp#115;amp#105;amp#111;amp#110;amp#46;amp#98;amp#105;amp#103;amp#100;amp#97;amp#116;amp#97;amp#46;amp#112;amp#46;amp#114;amp#101;amp#46;amp#115;amp#102;amp#46;amp#110;amp#101;amp#116;
Date: Friday, March 7, 2014 7:02 AM
To: "[bigdata:discussion]" 676946@discussion.bigdata.p.re.sf.netamp#54;amp#55;amp#54;amp#57;amp#52;amp#54;amp#64;amp#100;amp#105;amp#115;amp#99;amp#117;amp#115;amp#115;amp#105;amp#111;amp#110;amp#46;amp#98;amp#105;amp#103;amp#100;amp#97;amp#116;amp#97;amp#46;amp#112;amp#46;amp#114;amp#101;amp#46;amp#115;amp#102;amp#46;amp#110;amp#101;amp#116;
Subject: [bigdata:discussion] permissions, database notification features?
Security is rather common question, I think it is even worth having a separate wiki page.
By the way, I looked into BigData InMemChangeLog, I see there is a resolve function. I wonder, is it really needed? In my changelog implementation I had no troubles with extracting rdf statements out of IChangeRecord after they were committed
permissions, database notification features?https://sourceforge.net/p/bigdata/discussion/676946/thread/0c67fe7b/?limit=25#c682
Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/bigdata/discussion/676946/
To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/