NaryODPExperiment

From STLab

Nary Relation Patterns - In vitro study settings

We have looked for RDF/OWL representations of three logical patterns proposed by Hayes (), and we have singled out seven logical patterns. Then we have compared the resulting models against a multi-dimensional design space, including the following dimensions:

Quantitative, objective dimensions:

(a) amount of axioms needed for the representation of each design, calculated on a normalized set of 10000 individuals per ontology (b) expressivity of the resulting model, in terms of description logic varieties (that are associated with specific computational complexity) (c) time needed to check the consistency of the model (d) time needed to classify the model (e) amount of newly generated constants needed

Qualitative, objective dimensions:

(f) ability to support DL reasoning with reference to the full n-ary model (g) ability to support polymorphism (possibility to add new arguments when needed) (h) preservation of FOL relation topology, which we call relation footprint (ability to fully navigate the graph structure of n-ary relations instances)

Qualitative, subjective dimension:

(i) intuitiveness of representation and usability

We represent qualitative dimensions on a three-value scale (good, limited, none).

the patterns are represented in OWL2.

The patterns have been exemplified with reference to a leading example of time indexing, representing the fact that Garibaldi landed in Sicily in 1860 (during the so-called Expedition of the Thousand). We restricted this in-vitro study to a simple case, but larger arities are just more complex variations of the same design patterns.

In order to create a critical mass of individuals and axioms for ontologies that sample the patterns, we have used SyGENiA (Synthetic GENerator of instance Axioms) 4, which, contrarily to many random axiom generators, allows to specify a query that is used to generate ABox axioms from an ontology.

We have used Pellet 2.35 that produces accurate accounts of time used by the reasoner, running on a MacBookPro 3.06 GHz Intel Core Duo processor, with 8GB of RAM and MacOSX 10.7.3.

We have not attempted to balance axioms with specific constructs, hoping that SyGENiA would provide it by using the queries provided. This is not completely true however, and the results, iterated 10 times to neutralize random effects of concurrent processes, reflect the lack of a real balance. Since we were interested in a first assessment of these dimensions, we have decided to live with some imbalance derived from the synthetic ABox, deferring to in vivo studies a more grounded assessment.

Those settings were used for gathering results on dimensions (a) to (f), and (h). For (g), it comes directly from the logical constructs.

(i) has been evaluated anecdotically and subjectively by the authors and their correspondants in the ontology design community. Now, a user study has been performed with five phd students that were novices, having only attended a phd course on ontology design. They have been asked to rank on the three-value scale the intuitiveness of the seven solutions, after they have tried to model the running example independently. The results show similar results as those from the anectodical ranking, and will be reported here in detail soon.