AMIA Annu Symp Proc - Inter-annotator reliability of medical events, coreferences and temporal relations in clinical narratives by annotators with varying levels of clinical expertise.

Tópicos

{ extract(1171) text(1153) clinic(932) }
{ research(1218) medic(880) student(794) }
{ time(1939) patient(1703) rate(768) }
{ concept(1167) ontolog(924) domain(897) }
{ use(2086) technolog(871) perceiv(783) }
{ control(1307) perform(991) simul(935) }
{ care(1570) inform(1187) nurs(1089) }
{ howev(809) still(633) remain(590) }
{ risk(3053) factor(974) diseas(938) }
{ visual(1396) interact(850) tool(830) }
{ compound(1573) activ(1297) structur(1058) }
{ high(1669) rate(1365) level(1280) }
{ model(3404) distribut(989) bayesian(671) }
{ can(774) often(719) complex(702) }
{ inform(2794) health(2639) internet(1427) }
{ imag(1057) registr(996) error(939) }
{ method(1219) similar(1157) match(930) }
{ featur(3375) classif(2383) classifi(1994) }
{ imag(2830) propos(1344) filter(1198) }
{ clinic(1479) use(1117) guidelin(835) }
{ design(1359) user(1324) use(1319) }
{ case(1353) use(1143) diagnosi(1136) }
{ studi(1410) differ(1259) use(1210) }
{ research(1085) discuss(1038) issu(1018) }
{ spatial(1525) area(1432) region(1030) }
{ cost(1906) reduc(1198) effect(832) }
{ gene(2352) biolog(1181) express(1162) }
{ structur(1116) can(940) graph(676) }
{ implement(1333) system(1263) develop(1122) }
{ detect(2391) sensit(1101) algorithm(908) }
{ imag(1947) propos(1133) code(1026) }
{ data(1737) use(1416) pattern(1282) }
{ system(1976) rule(880) can(841) }
{ measur(2081) correl(1212) valu(896) }
{ bind(1733) structur(1185) ligand(1036) }
{ sequenc(1873) structur(1644) protein(1328) }
{ network(2748) neural(1063) input(814) }
{ imag(2675) segment(2577) method(1081) }
{ patient(2315) diseas(1263) diabet(1191) }
{ take(945) account(800) differ(722) }
{ studi(2440) review(1878) systemat(933) }
{ motion(1329) object(1292) video(1091) }
{ assess(1506) score(1403) qualiti(1306) }
{ treatment(1704) effect(941) patient(846) }
{ surgeri(1148) surgic(1085) robot(1054) }
{ framework(1458) process(801) describ(734) }
{ problem(2511) optim(1539) algorithm(950) }
{ error(1145) method(1030) estim(1020) }
{ chang(1828) time(1643) increas(1301) }
{ learn(2355) train(1041) set(1003) }
{ algorithm(1844) comput(1787) effici(935) }
{ method(1557) propos(1049) approach(1037) }
{ data(1714) softwar(1251) tool(1186) }
{ model(2220) cell(1177) simul(1124) }
{ general(901) number(790) one(736) }
{ method(984) reconstruct(947) comput(926) }
{ search(2224) databas(1162) retriev(909) }
{ featur(1941) imag(1645) propos(1176) }
{ data(3963) clinic(1234) research(1004) }
{ perform(999) metric(946) measur(919) }
{ system(1050) medic(1026) inform(1018) }
{ import(1318) role(1303) understand(862) }
{ model(2341) predict(2261) use(1141) }
{ perform(1367) use(1326) method(1137) }
{ studi(1119) effect(1106) posit(819) }
{ blood(1257) pressur(1144) flow(957) }
{ record(1888) medic(1808) patient(1693) }
{ health(3367) inform(1360) care(1135) }
{ model(3480) simul(1196) paramet(876) }
{ monitor(1329) mobil(1314) devic(1160) }
{ ehr(2073) health(1662) electron(1139) }
{ state(1844) use(1261) util(961) }
{ patient(2837) hospit(1953) medic(668) }
{ model(2656) set(1616) predict(1553) }
{ data(2317) use(1299) case(1017) }
{ age(1611) year(1155) adult(843) }
{ medic(1828) order(1363) alert(1069) }
{ signal(2180) analysi(812) frequenc(800) }
{ group(2977) signific(1463) compar(1072) }
{ sampl(1606) size(1419) use(1276) }
{ data(3008) multipl(1320) sourc(1022) }
{ first(2504) two(1366) second(1323) }
{ intervent(3218) particip(2042) group(1664) }
{ activ(1138) subject(705) human(624) }
{ patient(1821) servic(1111) care(1106) }
{ can(981) present(881) function(850) }
{ analysi(2126) use(1163) compon(1037) }
{ health(1844) social(1437) communiti(874) }
{ cancer(2502) breast(956) screen(824) }
{ use(976) code(926) identifi(902) }
{ use(1733) differ(960) four(931) }
{ drug(1928) target(777) effect(648) }
{ result(1111) use(1088) new(759) }
{ survey(1388) particip(1329) question(1065) }
{ estim(2440) model(1874) function(577) }
{ decis(3086) make(1611) patient(1517) }
{ process(1125) use(805) approach(778) }
{ activ(1452) weight(1219) physic(1104) }
{ method(1969) cluster(1462) data(1082) }
{ method(2212) result(1239) propos(1039) }

Resumo

The manual annotation of clinical narratives is an important step for training and validating the performance of automated systems that utilize these clinical narratives. We build an annotation specification to capture medical events, and coreferences and temporal relations between medical events in clinical text. Unfortunately, the process of clinical data annotation is both time consuming and costly. Many annotation efforts have used physicians to annotate the data. We investigate using annotators that are current students or graduates from diverse clinical backgrounds with varying levels of clinical experience. In spite of this diversity, the annotation agreement across our team of annotators is high; the average inter-annotator kappa statistic for medical events, coreferences, temporal relations, and medical event concept unique identifiers was 0.843, 0.859, 0.833, and 0.806, respectively. We describe methods towards leveraging the annotations to support temporal reasoning with medical events.

Resumo Limpo

manual annot clinic narrat import step train valid perform autom system util clinic narrat build annot specif captur medic event corefer tempor relat medic event clinic text unfortun process clinic data annot time consum cost mani annot effort use physician annot data investig use annot current student graduat divers clinic background vari level clinic experi spite divers annot agreement across team annot high averag interannot kappa statist medic event corefer tempor relat medic event concept uniqu identifi respect describ method toward leverag annot support tempor reason medic event

Resumos Similares

J Am Med Inform Assoc - Anaphoric relations in the clinical narrative: corpus creation. ( 0,914710152719422 )
J Biomed Inform - MedTime: a temporal information extraction system for clinical narratives. ( 0,912698462250167 )
J Am Med Inform Assoc - An end-to-end system to identify temporal relation in discharge summaries: 2012 i2b2 challenge. ( 0,909710010816584 )
J Am Med Inform Assoc - Combining rules and machine learning for extraction of temporal expressions and events from clinical narratives. ( 0,909073128349568 )
Appl Clin Inform - Representation of information about family relatives as structured data in electronic health records. ( 0,905795022342797 )
J Biomed Inform - Development and evaluation of RapTAT: a machine learning system for concept mapping of phrases from medical narratives. ( 0,903453354778121 )
J Am Med Inform Assoc - Assessing the role of a medication-indication resource in the treatment relation extraction from clinical text. ( 0,899444430707738 )
J. Med. Internet Res. - Web 2.0-based crowdsourcing for high-quality gold standard development in clinical natural language processing. ( 0,898895921826301 )
J Am Med Inform Assoc - A sense inventory for clinical abbreviations and acronyms created using clinical notes and medical dictionary resources. ( 0,89605664241133 )
J Biomed Inform - Anaphoric reference in clinical reports: characteristics of an annotated corpus. ( 0,892005611756629 )
J Biomed Inform - Semi-automatic semantic annotation of PubMed queries: a study on quality, efficiency, satisfaction. ( 0,89039644852897 )
J Am Med Inform Assoc - Knowledge-based biomedical word sense disambiguation: an evaluation and application to clinical document classification. ( 0,890251984076197 )
J Am Med Inform Assoc - Evaluating temporal relations in clinical text: 2012 i2b2 Challenge. ( 0,889387141765572 )
Artif Intell Med - Biomedical events extraction using the hidden vector state model. ( 0,888866311136766 )
J Biomed Inform - A concept-driven biomedical knowledge extraction and visualization framework for conceptualization of text corpora. ( 0,887730594878285 )
J Am Med Inform Assoc - A hybrid system for temporal information extraction from clinical text. ( 0,887116365259953 )
J Am Med Inform Assoc - Eventual situations for timeline extraction from clinical reports. ( 0,881456346512267 )
J Am Med Inform Assoc - Evaluating the impact of pre-annotation on annotation speed and potential bias: natural language processing gold standard development for clinical named entity recognition in clinical trial announcements. ( 0,880536682800786 )
AMIA Annu Symp Proc - Throw the bath water out, keep the baby: keeping medically-relevant terms for text mining. ( 0,87922455901679 )
J Med Syst - Redactable signatures for signed CDA Documents. ( 0,872783469822522 )
J Biomed Inform - Towards generating a patient's timeline: extracting temporal relationships from clinical notes. ( 0,871246549380451 )
J Biomed Inform - UMLS content views appropriate for NLP processing of the biomedical literature vs. clinical text. ( 0,861928108742129 )
AMIA Annu Symp Proc - Semantic processing to identify adverse drug event information from black box warnings. ( 0,860106077988187 )
Int J Med Inform - Detecting temporal expressions in medical narratives. ( 0,857097498082306 )
Int J Med Inform - Bootstrapping a de-identification system for narrative patient records: cost-performance tradeoffs. ( 0,856511831358591 )
AMIA Annu Symp Proc - Mapping annotations with textual evidence using an scLDA model. ( 0,855411219519776 )
J Am Med Inform Assoc - Assisted annotation of medical free text using RapTAT. ( 0,854018045688599 )
J Am Med Inform Assoc - MedXN: an open source medication extraction and normalization tool for clinical text. ( 0,853761309941514 )
J Biomed Inform - Evaluating the effects of machine pre-annotation and an interactive annotation interface on manual de-identification of clinical text. ( 0,849430441968342 )
J Biomed Inform - Lessons learnt from the DDIExtraction-2013 Shared Task. ( 0,847517030665294 )
Comput Math Methods Med - Ranking biomedical annotations with annotator's semantic relevancy. ( 0,847148508317171 )
J Am Med Inform Assoc - Automatic discourse connective detection in biomedical text. ( 0,84533547755013 )
AMIA Annu Symp Proc - Natural language processing for lines and devices in portable chest x-rays. ( 0,845214689175257 )
AMIA Annu Symp Proc - Natural language processing to extract follow-up provider information from hospital discharge summaries. ( 0,841683355708993 )
AMIA Annu Symp Proc - Automatically pairing measured findings across narrative abdomen CT reports. ( 0,841419615647106 )
AMIA Annu Symp Proc - Extracting Concepts Related to Homelessness from the Free Text of VA Electronic Medical Records. ( 0,840541268511309 )
AMIA Annu Symp Proc - Building gold standard corpora for medical natural language processing tasks. ( 0,840276224061199 )
J Biomed Inform - A new clustering method for detecting rare senses of abbreviations in clinical notes. ( 0,839847660499291 )
Brief. Bioinformatics - A survey on annotation tools for the biomedical literature. ( 0,839808238301427 )
J Biomed Inform - Extraction of events and temporal expressions from clinical narratives. ( 0,838013809667821 )
J Biomed Inform - Desiderata for ontologies to be used in semantic annotation of biomedical documents. ( 0,837401927300445 )
J Am Med Inform Assoc - A flexible framework for recognizing events, temporal expressions, and temporal relations in clinical text. ( 0,836111824040922 )
J Biomed Inform - Identifying non-elliptical entity mentions in a coordinated NP with ellipses. ( 0,835771483059027 )
J Biomed Inform - Lexical patterns, features and knowledge resources for coreference resolution in clinical notes. ( 0,834345461402025 )
J Biomed Inform - NCBI disease corpus: a resource for disease name recognition and concept normalization. ( 0,834192220283118 )
J Biomed Inform - Text de-identification for privacy protection: a study of its impact on clinical text information content. ( 0,833381385412129 )
Int J Med Inform - Detection of infectious symptoms from VA emergency department and primary care clinical documentation. ( 0,833144014019552 )
J Am Med Inform Assoc - 2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text. ( 0,827917034524332 )
AMIA Annu Symp Proc - Towards a semantic lexicon for clinical natural language processing. ( 0,824665516010933 )
Comput Methods Programs Biomed - BioAnnote: a software platform for annotating biomedical documents with application in medical learning environments. ( 0,822895874256136 )
J Am Med Inform Assoc - MITRE system for clinical assertion status classification. ( 0,817965997246181 )
J Am Med Inform Assoc - Towards comprehensive syntactic and semantic annotations of the clinical narrative. ( 0,816791682204282 )
AMIA Annu Symp Proc - Pharmacovigilance on twitter? Mining tweets for adverse drug reactions. ( 0,816131212616685 )
J Am Med Inform Assoc - The effect of word familiarity on actual and perceived text difficulty. ( 0,814571900755378 )
AMIA Annu Symp Proc - Sophia: A Expedient UMLS Concept Extraction Annotator. ( 0,813892082899067 )
J Biomed Inform - Evaluating measures of semantic similarity and relatedness to disambiguate terms in biomedical text. ( 0,812651214229922 )
AMIA Annu Symp Proc - Critical finding capture in the impression section of radiology reports. ( 0,812089372681481 )
J Biomed Inform - Annotating temporal information in clinical narratives. ( 0,811319093758558 )
BMC Med Inform Decis Mak - Text summarization as a decision support aid. ( 0,809998718638444 )
J Biomed Inform - Automatically extracting information needs from complex clinical questions. ( 0,809783285317891 )
J Am Med Inform Assoc - A study of machine-learning-based approaches to extract clinical entities and their assertions from discharge summaries. ( 0,807845604144248 )
AMIA Annu Symp Proc - Developing a section labeler for clinical documents. ( 0,807575535581923 )
J Biomed Inform - The DDI corpus: an annotated corpus with pharmacological substances and drug-drug interactions. ( 0,806840793723354 )
J Biomed Inform - Text summarization in the biomedical domain: a systematic review of recent research. ( 0,806177061908087 )
J Am Med Inform Assoc - Syntactic parsing of clinical text: guideline and corpus development with handling ill-formed sentences. ( 0,805504148312788 )
J Integr Bioinform - Automatic extraction of microorganisms and their habitats from free text using text mining workflows. ( 0,803725715811955 )
Int J Med Inform - A methodology to enhance spatial understanding of disease outbreak events reported in news articles. ( 0,803021843434124 )
AMIA Annu Symp Proc - Voice-dictated versus typed-in clinician notes: linguistic properties and the potential implications on natural language processing. ( 0,802625169978 )
AMIA Annu Symp Proc - Detecting abbreviations in discharge summaries using machine learning methods. ( 0,800512984179525 )
AMIA Annu Symp Proc - Automated illustration of patients instructions. ( 0,80025628953608 )
AMIA Annu Symp Proc - Extracting semantic lexicons from discharge summaries using machine learning and the C-Value method. ( 0,799933123725635 )
AMIA Annu Symp Proc - A cloud-based approach to medical NLP. ( 0,798018715460668 )
J Am Med Inform Assoc - Using rule-based natural language processing to improve disease normalization in biomedical text. ( 0,79687980373629 )
J Am Med Inform Assoc - Extracting drug indication information from structured product labels using natural language processing. ( 0,793607413423353 )
Sci Data - Building the graph of medicine from millions of clinical narratives. ( 0,791733887058848 )
AMIA Annu Symp Proc - It's about this and that: a description of anaphoric expressions in clinical text. ( 0,791706542559146 )
J Am Med Inform Assoc - Large-scale evaluation of automated clinical note de-identification and its impact on information extraction. ( 0,791440681379047 )
AMIA Annu Symp Proc - TagLine: Information Extraction for Semi-Structured Text in Medical Progress Notes. ( 0,789581770477326 )
BMC Med Inform Decis Mak - The freetext matching algorithm: a computer program to extract diagnoses and causes of death from unstructured text in electronic health records. ( 0,789429345770171 )
AMIA Annu Symp Proc - Semantic processing to identify adverse drug event information from black box warnings. ( 0,788349831088753 )
J Am Med Inform Assoc - Comparison of a semi-automatic annotation tool and a natural language processing application for the generation of clinical statement entries. ( 0,787113697039038 )
J Am Med Inform Assoc - Automated clinical trial eligibility prescreening: increasing the efficiency of patient identification for clinical trials in the emergency department. ( 0,786244769878931 )
J Biomed Inform - Semantator: semantic annotator for converting biomedical text to linked data. ( 0,786239115166998 )
J Biomed Inform - Ontology modularization to improve semantic medical image annotation. ( 0,784249759996 )
J Biomed Inform - Enhancing clinical concept extraction with distributional semantics. ( 0,782392347940803 )
AMIA Annu Symp Proc - Qualitative analysis of workflow modifications used to generate the reference standard for the 2010 i2b2/VA challenge. ( 0,78198354064502 )
AMIA Annu Symp Proc - A comparative study of current Clinical Natural Language Processing systems on handling abbreviations in discharge summaries. ( 0,781710336276126 )
J Am Med Inform Assoc - Automatic abstraction of imaging observations with their characteristics from mammography reports. ( 0,779894549787985 )
Perspect Health Inf Manag - A comparison of two approaches to text processing: facilitating chart reviews of radiology reports in electronic medical records. ( 0,779799857799604 )
AMIA Annu Symp Proc - Combining corpus-derived sense profiles with estimated frequency information to disambiguate clinical abbreviations. ( 0,77918619731691 )
J Biomed Inform - Approaches to verb subcategorization for biomedicine. ( 0,777086584898285 )
AMIA Annu Symp Proc - Extracting patient demographics and personal medical information from online health forums. ( 0,775051143168893 )
J Am Med Inform Assoc - Hybrid methods for improving information access in clinical documents: concept, assertion, and relation identification. ( 0,771014920018883 )
J Biomed Inform - Degree centrality for semantic abstraction summarization of therapeutic studies. ( 0,769725779251963 )
J Am Med Inform Assoc - Temporal reasoning over clinical text: the state of the art. ( 0,765753355759759 )
J Biomed Inform - Common data model for natural language processing based on two existing standard information models: CDA+GrAF. ( 0,765311092736439 )
J Biomed Inform - Unsupervised biomedical named entity recognition: experiments with clinical and biological texts. ( 0,764885246088318 )
Neural Comput - Scaling laws of associative memory retrieval. ( 0,762554429926937 )
J Biomed Inform - Using an ensemble system to improve concept extraction from clinical records. ( 0,760969134926778 )
J Biomed Inform - Ontology-guided feature engineering for clinical text classification. ( 0,75833644801966 )