TY - GEN
T1 - An Evaluation of Information Extraction Tools for Identifying Health Claims in News Headlines
AU - Yuan, Shi
AU - Yu, Bei
N1 - Publisher Copyright:
© 2018 EventStory 2018 - Events and Stories in the News, Proceedings of the Workshop. All rights reserved.
PY - 2018
Y1 - 2018
N2 - This study evaluates the performance of four information extraction tools (extractors) on identifying health claims in health news headlines. A health claim is defined as a triplet: IV (what is being manipulated), DV (what is being measured) and their relation. Tools that can identify health claims provide the foundation for evaluating the accuracy of these claims against authoritative resources. The evaluation result shows that 26% headlines do not include health claims, and all extractors face difficulty separating them from the rest. For those with health claims, OPENIE-5.0 performed the best with F-measure at 0.6 level for extracting “IV-relation-DV”. However, the characteristic linguistic structures in health news headlines, such as incomplete sentences and non-verb relations, pose particular challenge to existing tools.
AB - This study evaluates the performance of four information extraction tools (extractors) on identifying health claims in health news headlines. A health claim is defined as a triplet: IV (what is being manipulated), DV (what is being measured) and their relation. Tools that can identify health claims provide the foundation for evaluating the accuracy of these claims against authoritative resources. The evaluation result shows that 26% headlines do not include health claims, and all extractors face difficulty separating them from the rest. For those with health claims, OPENIE-5.0 performed the best with F-measure at 0.6 level for extracting “IV-relation-DV”. However, the characteristic linguistic structures in health news headlines, such as incomplete sentences and non-verb relations, pose particular challenge to existing tools.
UR - http://www.scopus.com/inward/record.url?scp=85062636909&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85062636909&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85062636909
T3 - EventStory 2018 - Events and Stories in the News, Proceedings of the Workshop
SP - 34
EP - 43
BT - EventStory 2018 - Events and Stories in the News, Proceedings of the Workshop
PB - Association for Computational Linguistics (ACL)
T2 - 2018 Events and Stories in the News Workshop, EventStory 2018, colocated with the 27th International Conference on Computational Linguistics, COLING 2018
Y2 - 20 August 2018
ER -