Hi Igor,

I just realized I never responded to this - sorry for the long delay.

So - I didn't realize that all the columns would be defined by values of a single property. Given that, what you're saying makes sense. I would just suggest some slight changes to the syntax you proposed. For the "traceability matrix", with just an 'X' in each cell, a query like this:

{{#ask:[[Category:Components]]
|?implements
|format=pivot
|columns property=implements
}}

(i.e., no need to set "X" - it's assumed.)

...and for the more-complex "dependency matrix", a query like this:

{{#ask:[[Category:Dependencies]]
|?declared for
|?depends on
|?comment
|format=pivot
|rows property=declared for
|columns property=depends on
|cells property=comment
}}

In both cases, the parameter names are hopefully a little clearer, and parameters are all separated by pipes, not commas.

-Yaron


On Mon, Jun 6, 2011 at 6:01 PM, Igor Lobanov <igor.lobanov@ymail.com> wrote:
Yaron,

I agree with you with regard to benefits of having this feature implemented as a result formatter, thank you. So this is my primary approach.

I find the syntax you've proposed a bit confusing. The question mark sign in the #ask query means that you're adding additional fields to the output. However, in our 'reference example' there's no such properties as 'create' and 'read', they are string literal values of the 'access type' property. Have you meant that there is a 'create' relation between System and Data?

I've thought about the syntax and I'd like to share my idea. Yaron, sorry if it's exactly what's you had in mind, in this case I think detailed explanation is useful to have and probably reuse later.

Idea is that the query is used to fetch actual cell contents. Then you just specify which properties of query result are used to define rows and columns. This approach effectively allows to use the either internal objects or normal pages to fill out the cells.

Couple of examples to illustrate the idea:

(1) Requirements to components traceability
- there are categories: Requirement, Component
- each Requirement has title (page name)
- each Component has title (page name) and a multi-valued relation 'implements' to Requirement

In order to build traceability matrix, you have to query Components. The Component title is used to produce values for rows. Values for cells are constant, that is 'X' literal value since we are only interested in showing if there a relation. In addition we only need to specify which property is used to derive columns, that is 'implements' in this case.

The in-line query is looks like the following:
{{#ask:[[Category:Components]]
|?implements
|format=pivot,columns=implements,cells='X'}}

(2) Dependencies matrix
- there are categories: Components, Dependencies
- each Component has a title (page name) and a set of internal objects of category Dependencies (category is assigned by template)
- each Dependency has an internal name (not interesting), a back reference to a declaring Component 'declared for', a reference 'depends on' to another component which the declaring component has the dependency from, and a string value property 'comment' that adds come informal insight on the nature of the dependency.

Dependency matrix should have components both on rows and columns. In order to build it we need to query Dependencies. Values for cells are taken from the dependencies comments, values for rows are from 'declared for' values of dependencies, and finally values for columns are from 'depends on' values.

The query should look like the following:
{{#ask:[[Category:Dependencies]]
|?declared for
|?depends on
|?comment
|format=pivot,rows=declared for,columns=depends on,cells=comment}}

It seems that this approach is versatile and still concise. What's your view?

--
Best regards,
Igor Lobanov

From: Yaron Koren <yaron@wikiworks.com>
To: Igor Lobanov <igor.lobanov@ymail.com>
Cc: Patrick Nagel <mail@patrick-nagel.net>; "semediawiki-devel@lists.sourceforge.net" <semediawiki-devel@lists.sourceforge.net>
Sent: Monday, 6 June 2011, 16:00

Subject: Re: [SMW-devel] Matrix result formatter

Hi Igor,

It seems to me like this could be done as a query result... the example you gave might be callable like this:

{{#ask:[[Category:Systems]]|?create=blue|?update=yellow|?read=green|format=traceabilitymatrix}}

There might be no reason to need to specify the "Data" category, i.e. the set of columns, if they're already covered by the values of the properties. But if there are any empty columns, you might want to specify it manually - you could do that with something like:

{{#ask:[[Category:Systems]]|?create=blue|?update=yellow|?read=green|columns=Data|format=traceabilitymatrix}}

In general, I would say, if you can implement this as a result format, that would definitely be the better approach - it'll be easier for you to implement, easier for admins to install (assuming it were added to Semantic Result Formats), and easier for users to use, since they would already know the #ask syntax.

-Yaron

On Fri, Jun 3, 2011 at 7:03 PM, Igor Lobanov <igor.lobanov@ymail.com> wrote:
Patrick, Yaron,

Thank you for your feedback!

I can see a problem with the 'result format' approach that originally forced me to start thinking about separate directive. This is what I refer to as 'inner join syndrome', i.e. you will have only those rows and columns in a matrix that have at least one relation of interest. In other words there won't be any empty rows and columns. So initially it seems that in order to build proper relationship matrix one need to specify three queries: rows of interest, columns of interest and actual cells. Other approach is to extend query expression to support something like OPTIONAL operator in SPARQL, but it doesn't seem to be any easier. However, I'll explore this possibility once I build up more understanding of how query execution works, so thanks for this idea.

As for real world example, here it is. I use SMW as an implementation for architectural knowledge base and components repository. There is a lot of stuff going on, but in particular I have simple ontology used to relate enterprise-wide data and applications. There are two categories: Systems and Data. In addition, every System page has a set of internal records (form-driven) that relate the System with a particular kind of Data and has additional metadata such as type of dependency (create/read/update), middleware used to access the data, and some arbitrary comments. What I'd like to produce (and I did, albeit in a suboptimal way) is a 'CRUD matrix' with Systems listed as rows, kinds of Data as columns and a short summary of the relation. First, it's colour-coded: 'create' is blue, 'update' is yellow, 'read' is green. Second, there is a link to the middleware page and a comment displayed. All of these is provided by custom template that uses parser functions. Hope this explanation is clear enough, but I could be more specific.
--
Best regards,
Igor Lobanov

From: Yaron Koren <yaron@wikiworks.com>
To: Patrick Nagel <mail@patrick-nagel.net>
Cc: semediawiki-devel@lists.sourceforge.net
Sent: Thursday, 2 June 2011, 15:45
Subject: Re: [SMW-devel] Matrix result formatter

Hi Igor,

Before I go further, let me note how refreshing it is to hear someone asking for advice on this mailing list before creating a new SMW extension. :) It happens more rarely than you might think.

So - Patrick is right that this would ideally be created as a new SMW result format, and ideally a format added to the Semantic Result Formats extension. However - it depends a lot on the complexity of your implementation, whether such a thing is doable.

Could you give an example of the "traceability" you would like to illustrate, as a calculation based on various properties and pages? In other words, for a single cell in the matrix, how would it be determined whether to put an "x" there, what color to set it to, etc.? I think that would help a lot in trying to figure out the right approach.

-Yaron

On Wed, Jun 1, 2011 at 10:36 PM, Patrick Nagel <mail@patrick-nagel.net> wrote:
Hi Igor,

On 2011-06-02 07:11, Igor Lobanov wrote:
> I'm looking for a way to produce matrices out of relations between
> arbitrary types of objects/pages in SMW. Simple example would be a
> 'traceability matrix' between pages within such categories as 'Business
> Requirement' and 'System Requirement'. Constructed matrix would have
> business requirements and system requirements as row and column headers
> respectively, while actual elements would be either emply (no
> traceability) or flagged (traceability exists).
>
> Good example is provided in the Wikipedia article:
> http://en.wikipedia.org/wiki/Traceability_matrix
>
> More complex example (one that I have in mind) would be based not on a
> simple relationship, but on internal objects representing relationships
> among various objects with additional scalar properties. In this case,
> cell content would be more flexible and somehow derived from the
> properties, e.g. color, icons, formatting, etc.
>
> Next, I'm able to produce such matrices using a moderately complex
> incantation based on obscure things like arrays and loops, but it's
> O(NxM), needs to be tailored for a particular type of matrix, and look
> awful in general.
>
> I believe that general-purpose matrix/pivot style result formatter for
> inline queries would be good for SMW. Moreover, I'd like to try to blow
> the dust from my PHP skills and actually produce one. What I'm looking
> for is some informed advice on how to approach this one.
>
> My initial idea is to create an extension that introduces a
> {{#pivot...}} instruction that actually does underlying query and
> formatting. Does it make sense? Or it is something that can't be built
> outside the SMW core?

That sounds indeed very useful. I'm just a user of SMW, but I think the
best way to implement this would be as an extension similar to Semantic
Result Formats, or even better, within that extension. It would then not
be a separate parser function, but just an output format, that can be
chosen via the 'format' parameter to an #ask query.

Patrick.

--
Key ID: 0x86E346D4 http://patrick-nagel.net/key.asc
Fingerprint: 7745 E1BE FA8B FBAD 76AB 2BFC C981 E686 86E3 46D4


------------------------------------------------------------------------------
Simplify data backup and recovery for your virtual environment with vRanger.
Installation's a snap, and flexible recovery options mean your data is safe,
secure and there when you need it. Data protection magic?
Nope - It's vRanger. Get your free trial download today.
http://p.sf.net/sfu/quest-sfdev2dev
_______________________________________________
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel




--
WikiWorks MediaWiki Consulting http://wikiworks.com

------------------------------------------------------------------------------
Simplify data backup and recovery for your virtual environment with vRanger.
Installation's a snap, and flexible recovery options mean your data is safe,
secure and there when you need it. Data protection magic?
Nope - It's vRanger. Get your free trial download today.
http://p.sf.net/sfu/quest-sfdev2dev
_______________________________________________
Semediawiki-devel mailing list
Semediawiki-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel





--
WikiWorks MediaWiki Consulting http://wikiworks.com





--
WikiWorks MediaWiki Consulting http://wikiworks.com