KnowledgeExtractionToolEval
From STLab
(→A landscape analysis of some knowledge extraction tools) |
(→A landscape analysis of some knowledge extraction tools) |
||
| Line 5: | Line 5: | ||
A technical report about the analysis (including only some of the results) can be downloaded [http://stlab.istc.cnr.it/documents/papers/ketoolstudy.pdf here]. | A technical report about the analysis (including only some of the results) can be downloaded [http://stlab.istc.cnr.it/documents/papers/ketoolstudy.pdf here]. | ||
| - | The table shows, for each ''basic NLP task'' corresponding to a ''basic knowledge engineering task'' (e.g. '''sense tagging''' in NLP roughly maps to '''entity typing''' in KE), the absolute and normalized results (in terms of | + | The table shows, for each ''basic NLP task'' corresponding to a ''basic knowledge engineering task'' (e.g. '''sense tagging''' in NLP roughly maps to '''entity typing''' in KE), the absolute and normalized results (in terms of P(recision), R(ecall(, F(-measure), A(ccuracy)) obtained by each KE tool. |
| - | The ''Merging'' column contains the results when all outputs are merged. The merged results are also used as the upper limit for evaluating the recall | + | The ''Merging'' column contains the results when all outputs are merged. The merged results are also used as the upper limit for evaluating the recall, f-measure, and accuracy of the individual tools. |
Values in red are better than the average by one std; values in green are better than the average by .5 std. | Values in red are better than the average by one std; values in green are better than the average by .5 std. | ||
Revision as of 10:16, 29 January 2013
A landscape analysis of some knowledge extraction tools
This page includes the complete results for a landscape analysis of Knowledge Extraction (KE) tools.
A technical report about the analysis (including only some of the results) can be downloaded here.
The table shows, for each basic NLP task corresponding to a basic knowledge engineering task (e.g. sense tagging in NLP roughly maps to entity typing in KE), the absolute and normalized results (in terms of P(recision), R(ecall(, F(-measure), A(ccuracy)) obtained by each KE tool.
The Merging column contains the results when all outputs are merged. The merged results are also used as the upper limit for evaluating the recall, f-measure, and accuracy of the individual tools.
Values in red are better than the average by one std; values in green are better than the average by .5 std.
The results for semantic role labelling and frame detection include the combined results for: frames; role-filler pairs; role labels; filler targets. See a dedicated table for analytic results on this basic task.
