Návrh DN Template pravidel

Authors:

Template rules are always compiled into ordinary DN rules. Change of an instance of a template causes update of raw DN rule. Before creation of an instance of any template a target rule group has to be selected.

Replace Rule Template

Built-in properties:

odcs:propertyEndpointsBackup
odcs:propertySubjectBackup
odcs:propertyObjectBackup

Schema:

CREATE TABLE DB.ODCLEANSTORE.DN_REPLACE_TEMPLATE_INSTANCES
(
  id INTEGER NOT NULL IDENTITY PRIMARY KEY,
  rawRuleId INTEGER NOT NULL,
  propertyName NVARCHAR(255) NOT NULL,
  pattern NVARCHAR(255) NOT NULL,
  replacement NVARCHAR(255) NOT NULL,

  FOREIGN KEY (rawRuleId) REFERENCES DB.ODCLEANSTORE.DN_RULES(id) ON DELETE CASCADE
);

Example of a template instance as an entry of the table:

13, '<http://.../...#...>', '(.)(.)', '$2$1'

meaning that rule with id = 13 in DB.ODCLEANSTORE.DN_RULES is a compilation of:

PropertyName = 'http://.../...#...',
Pattern = '(.)(.)',
Replacement = '$2$1'

Such rule replaces all triples

{?s '<http://.../...#...>' ?o}

with

{?s '<http://.../...#...>' fn:replace(str(?o), '(.)(.)', '$2$1')}

Raw rule to be composed for each instance:

//Creates a backup of ?s ?p ?o  for affected ?p (= %PROPERTY_NAME%)
"INSERT", "{?p <" + ODCS.propertyEndpointsBackup + "> [<" + ODCS.propertySubjectBackup + "> ?s; <" + ODCS.propertyObjectBackup + "> ?o]} WHERE {GRAPH $$graph$$ {?s ?p ?o} FILTER (?p = %PROPERTY_NAME%)}"

//Removes old values
"DELETE", "{?s ?p ?o} WHERE {GRAPH $$graph$$ {?s ?p ?o} FILTER (?p = %PROPERTY_NAME%)}"

//Creates new values with replace
"INSERT, "{?s ?p ?x} WHERE {GRAPH $$graph$$ {SELECT ?s ?p (fn:replace(str(?o), %PATTERN%, %REPLACEMENT%)) AS ?x WHERE {?p <" + ODCS.propertyEndpointsBackup + "> ?b. ?b <" + ODCS.propertySubjectBackup + "> ?s; <" + ODCS.propertyObjectBackup + "> ?o}}}"

//Removes backups
"DELETE, "{?s ?p ?b1. ?b1 ?b2 ?x} WHERE {GRAPH $$graph$$ {?s ?p ?b1. ?b1 ?b2 ?x} FILTER (?p = <" + ODCS.propertyEndpointsBackup + ">)}"

Now there is a better way to do that without special properties

//Remove old and Insert altered value at the same time with SPARUL MODIFY
"MODIFY", "DELETE {?s ?p ?o} INSERT {?s ?p ?x} WHERE {GRAPH $$graph$$ {SELECT ?s ?p ?o (fn:replace(str(?o), %PATTERN%, %REPLACEMENT%)) AS ?x WHERE {{?s ?p ?o} FILTER (?p = %PROPERTY_NAME%)}}}"

Rename Rule Template

Schema:

CREATE TABLE DB.ODCLEANSTORE.DN_REPLACE_TEMPLATE_INSTANCES
(
  id INTEGER NOT NULL IDENTITY PRIMARY KEY,
  rawRuleId INTEGER NOT NULL,
  sourcePropertyName NVARCHAR(255) NOT NULL,
  targetPropertyName NVARCHAR(255) NOT NULL,

  FOREIGN KEY (rawRuleId) REFERENCES DB.ODCLEANSTORE.DN_RULES(id) ON DELETE CASCADE
);

Changing:

?s sourcePropertyName ?o

into

?s targetPropertyName ?o

Raw rule to be composed for each instance:

//Create new property
"INSERT", "{?s %TARGET_PROPERTY_NAME% ?o} WHERE {GRAPH $$graph$$ {?s %SOURCE_PROPERTY_NAME% ?o}}"

//Drop leftovers
"DELETE", "{?s ?p ?o} WHERE {GRAPH $$graph$$ {?s ?p ?o} FILTER (?p = %SOURCE_PROPERTY_NAME%)}"

Now there is a better way to do that

"MODIFY", "DELETE {?s ?p ?o} INSERT {?s %TARGET_PROPERTY_NAME% ?o} WHERE {{?s ?p ?o} FILTER (?p = %SOURCE_PROPERTY_NAME%)}"

Filter Rule Template

Schema:

CREATE TABLE DB.ODCLEANSTORE.DN_FILTER_TEMPLATE_INSTANCES
(
  id INTEGER NOT NULL IDENTITY PRIMARY KEY,
  rawRuleId INTEGER NOT NULL,
  propertyName NVARCHAR(255) NOT NULL,
  pattern NVARCHAR(255) NOT NULL,
  keep SMALLINT NOT NULL DEFAULT 1,

  FOREIGN KEY (rawRuleId) REFERENCES DB.ODCLEANSTORE.DN_RULES(id) ON DELETE CASCADE
);

Raw rule to be composed for each instance:

//Keep == true:
"DELETE", "{?s ?p ?o} WHERE {GRAPH $$graph$$ {?s ?p ?o} FILTER (?p = %PROPERTY_NAME% AND !fn:match(str(?o), %PATTERN%))}"

//Keep == false:
"DELETE", "{?s ?p ?o} WHERE {GRAPH $$graph$$ {?s ?p ?o} FILTER (?p = %PROPERTY_NAME% AND fn:match(str(?o), %PATTERN%))}"

Split Rule Template

These can only exist for fixed structures:

"A, B" can be split with

fn:replace("A, B", "^(.), (.)$", '$1')
fn:replace("A, B", "^(.), (.)$", '$2')

can be used to split into two (or any arbitrary (fixed) number of) values.

Concatenate Rule Template

Schema:

CREATE TABLE DB.ODCLEANSTORE.DN_CONCATENATE_TEMPLATE_INSTANCES
(
  id INTEGER NOT NULL IDENTITY PRIMARY KEY,
  rawRuleId INTEGER NOT NULL,
  propertyName NVARCHAR(255) NOT NULL,
  delimiter NVARCHAR(255) NOT NULL,

  FOREIGN KEY (rawRuleId) REFERENCES DB.ODCLEANSTORE.DN_RULES(id) ON DELETE CASCADE
);

Raw rule to be composed for each instance:

//Remove when there is at least a pair that can be concatenated and insert the concatenation
"MODIFY", "DELETE {?s ?p ?o} INSERT {?s ?p ?c} WHERE { GRAPH $$graph$$ {SELECT ?s ?p (sql:group_concat(str(?o), '%s')) AS ?c WHERE {?s ?p ?o} GROUP BY ?s ?p HAVING COUNT(?o) > 1} {?s ?p ?o} FILTER (?p = %s)}"

Administrace

There will be a set of web pages for each template (ListInstancesPage, EditInstancePage, AddNewInstancePage).

Each instance of a template is bound to exactly one raw rule. Any changes to template instance affect this raw rule:

1) Delete Template Instance = Delete Its Raw Rule
2) Edit Template Instance = Replace/Regenerate(Remove Old) Its Raw Rule (Whichever is simpler to implement)
3) Add Template Instance = Construct New Raw Rule

Edits of raw rules that are a product of rule generation from a template are not propagated to the template instances (Some changes might be out of scope of the template).

Allowing such edits may be confusing for users and therefore might be forbidden. On the other hand it also puts more power in the users hands.

Because any change of the template instance cause all prior custom modifications to the raw rule disappear, it seems more reasonable to disallow modifications to raw rules.

Adding Instances

Form with all fields needed for the rule construction

Editing Instance

Form with all fields needed for the rule construction pre-filled with values of the instance

Code Enhancement

DEFINE input:default-graph-uri

allows user to omit

GRAPH $$graph$$

when there is no subquery

Template pravidla

Motivace: zadávání pravidel jako posloupnosti INSERT / DELETE sparql útržků může být zdlouhavé, nepřehledné. Některé části různých pravidel budou velmi podobné a budou se opakovat (v případech kdy se sémantika pravidel značně podobá), úprava sdíleného kódu bude nepříjemná pokud takových pravidel bude mnoho.

Řešení: pravidla bude možné generovat z šablon. Otázkou je jak se budou šablony uchovávat a jak se budou pravidla kompletovat. V principu máme 2 návrhy:

současný
- Tabulka jednotlivých šablon (TEMPLATES)
  - př.: Replace Template
- Tabulky součástí dané komponenty (TEMPLATE_FIELDS) daných typů (TEMPLATE_FIELD_TYPES)
  - př.: PropertyName, RegExp, Replacement, Constant SPARUL#1, Constant SPARUL#2, ...
- Tabulka popisující konstanty - útržky SPARUL, znichž se pravidlo bude skládat (TEMPLATE_CONSTANTS)
  - př.: Constant SPARUL #1 = "{?a ?b ?c} WHERE {GRAPH $$graph$$ {SELECT replace(?o,"
- Tabulka popisující tvorbu instancí pravidel generovaných podle některé šablony (TEMPLATE_RECIPES) - kolik a jaké komponenty INSERT / DELETE se při generovaní vytvoří
  - př.: Replace Template = INSERT #1, DELETE #1, INSERT #2, DELETE #2
- Tabulka popisující tvorbu konkrétní komponenty (TEMPLATE_COMPONENT_RECIPES) - pořadí seskládání součástí (hodnot jednotlivých FIELDS ať už konstantních SPARUL nebo hodnot zadaných uživatelem)
  - př: INSERT #1 = Constant SPARUL#1, RegExp, Replacement, Constant SPARUL#2, PropertyName
- Tabulka instancí (TEMPLATE_INSTANCES) - mapuje vytvořené pravidlo na šablonu
  - př.: pravidlo #1 z DN_RULES je instancí #1 Replace Template
- Tabulka hodnot nekonstantních součástí šablony (TEMPLATE_VALUES)
  - př.: PropertyName instance #1 = http://purl.org/...
- Výhody - podpora pro rozšiřitelnost za běhu (změny šablon, opravy kódu)
- Nevýhody - složité schéma, generování pravidel vyžaduje čtení z db ze 2 tabulek (demo kód v datanorm větvi v DNImpl balíčku), wicket prý podporuje pouze fixní HTML takže frontend by buď měl fixní obrazovky pro každou existující a podporovanou šablonu nebo by se musela správa řešit nějak nepřirozeně (omezení na wicket nemusí v budoucnu existovat?)
zjednodušující
- pro každou šablonu bude fixní tabulka, ve které budou zachyceny jednotlivé pole, která uživatel definuje při tvorbě pravidla dle této šablony.
- Výhody - jednoduchost
- Nevýhody - špatná rozšiřitelnost, kompilace pravidel zadrátovaná v kódu
- př.:

Replace Template

propertyName	regexp	replacement
http://...	(.)_.	$1

Discussion

Dušan Rychnovský - 2012-07-30

Ahoj,

na frontendu budu pracovat s necim takovymto v pozici business entity:

class ReplaceDNRule { Long id; String propertyName; String regexp; String replacement; }

Mohl bys sem prosim pripsat, jak by ve verzi 1 vypadaly nasledujici queries?

select replace pravidla s id X (vysledkem dotazu jeden radek se ctyrmi sloupci)

select vsech replace pravidel serazenych podle propertyName/regexp/replacement - pocinaje Xtym pravidlem a konce Ytym? (vysledkem dotazu ntice radku o ctyrech sloupcich)

insert replace pravidla na zaklade instance ReplaceDNRule (kde id == null)

update replace pravidla na zaklade instance ReplaceDNRule s danym id

delete replace pravidla s danym id

Diky moc,
Dusan

Last edit: Dušan Rychnovský 2012-07-30
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

ODClean Store Wiki

Linked Data management tool

Návrh DN Template pravidel

Replace Rule Template

Rename Rule Template

Filter Rule Template

Split Rule Template

Concatenate Rule Template

Administrace

Adding Instances

Editing Instance

Code Enhancement

Template pravidla

Replace Template

Discussion