Brief. Bioinformatics - Caveats and pitfalls of ROC analysis in clinical microarray research (and how to avoid them).

Tópicos

{ model(2341) predict(2261) use(1141) }
{ research(1085) discuss(1038) issu(1018) }
{ studi(2440) review(1878) systemat(933) }
{ case(1353) use(1143) diagnosi(1136) }
{ take(945) account(800) differ(722) }
{ control(1307) perform(991) simul(935) }
{ perform(999) metric(946) measur(919) }
{ use(1733) differ(960) four(931) }
{ data(1737) use(1416) pattern(1282) }
{ method(1219) similar(1157) match(930) }
{ featur(3375) classif(2383) classifi(1994) }
{ framework(1458) process(801) describ(734) }
{ data(2317) use(1299) case(1017) }
{ can(774) often(719) complex(702) }
{ patient(2315) diseas(1263) diabet(1191) }
{ treatment(1704) effect(941) patient(846) }
{ learn(2355) train(1041) set(1003) }
{ clinic(1479) use(1117) guidelin(835) }
{ method(1557) propos(1049) approach(1037) }
{ data(1714) softwar(1251) tool(1186) }
{ general(901) number(790) one(736) }
{ studi(1410) differ(1259) use(1210) }
{ risk(3053) factor(974) diseas(938) }
{ import(1318) role(1303) understand(862) }
{ perform(1367) use(1326) method(1137) }
{ research(1218) medic(880) student(794) }
{ implement(1333) system(1263) develop(1122) }
{ estim(2440) model(1874) function(577) }
{ decis(3086) make(1611) patient(1517) }
{ detect(2391) sensit(1101) algorithm(908) }
{ model(3404) distribut(989) bayesian(671) }
{ imag(1947) propos(1133) code(1026) }
{ inform(2794) health(2639) internet(1427) }
{ system(1976) rule(880) can(841) }
{ measur(2081) correl(1212) valu(896) }
{ imag(1057) registr(996) error(939) }
{ bind(1733) structur(1185) ligand(1036) }
{ sequenc(1873) structur(1644) protein(1328) }
{ imag(2830) propos(1344) filter(1198) }
{ network(2748) neural(1063) input(814) }
{ imag(2675) segment(2577) method(1081) }
{ motion(1329) object(1292) video(1091) }
{ assess(1506) score(1403) qualiti(1306) }
{ surgeri(1148) surgic(1085) robot(1054) }
{ problem(2511) optim(1539) algorithm(950) }
{ error(1145) method(1030) estim(1020) }
{ chang(1828) time(1643) increas(1301) }
{ concept(1167) ontolog(924) domain(897) }
{ algorithm(1844) comput(1787) effici(935) }
{ extract(1171) text(1153) clinic(932) }
{ design(1359) user(1324) use(1319) }
{ model(2220) cell(1177) simul(1124) }
{ care(1570) inform(1187) nurs(1089) }
{ method(984) reconstruct(947) comput(926) }
{ search(2224) databas(1162) retriev(909) }
{ featur(1941) imag(1645) propos(1176) }
{ howev(809) still(633) remain(590) }
{ data(3963) clinic(1234) research(1004) }
{ system(1050) medic(1026) inform(1018) }
{ visual(1396) interact(850) tool(830) }
{ compound(1573) activ(1297) structur(1058) }
{ studi(1119) effect(1106) posit(819) }
{ blood(1257) pressur(1144) flow(957) }
{ spatial(1525) area(1432) region(1030) }
{ record(1888) medic(1808) patient(1693) }
{ health(3367) inform(1360) care(1135) }
{ model(3480) simul(1196) paramet(876) }
{ monitor(1329) mobil(1314) devic(1160) }
{ ehr(2073) health(1662) electron(1139) }
{ state(1844) use(1261) util(961) }
{ patient(2837) hospit(1953) medic(668) }
{ model(2656) set(1616) predict(1553) }
{ age(1611) year(1155) adult(843) }
{ medic(1828) order(1363) alert(1069) }
{ signal(2180) analysi(812) frequenc(800) }
{ cost(1906) reduc(1198) effect(832) }
{ group(2977) signific(1463) compar(1072) }
{ sampl(1606) size(1419) use(1276) }
{ gene(2352) biolog(1181) express(1162) }
{ data(3008) multipl(1320) sourc(1022) }
{ first(2504) two(1366) second(1323) }
{ intervent(3218) particip(2042) group(1664) }
{ activ(1138) subject(705) human(624) }
{ time(1939) patient(1703) rate(768) }
{ patient(1821) servic(1111) care(1106) }
{ use(2086) technolog(871) perceiv(783) }
{ can(981) present(881) function(850) }
{ analysi(2126) use(1163) compon(1037) }
{ health(1844) social(1437) communiti(874) }
{ structur(1116) can(940) graph(676) }
{ high(1669) rate(1365) level(1280) }
{ cancer(2502) breast(956) screen(824) }
{ use(976) code(926) identifi(902) }
{ drug(1928) target(777) effect(648) }
{ result(1111) use(1088) new(759) }
{ survey(1388) particip(1329) question(1065) }
{ process(1125) use(805) approach(778) }
{ activ(1452) weight(1219) physic(1104) }
{ method(1969) cluster(1462) data(1082) }
{ method(2212) result(1239) propos(1039) }

Resumo

The receiver operating characteristic (ROC) has emerged as the gold standard for assessing and comparing the performance of classifiers in a wide range of disciplines including the life sciences. ROC curves are frequently summarized in a single scalar, the area under the curve (AUC). This article discusses the caveats and pitfalls of ROC analysis in clinical microarray research, particularly in relation to (i) the interpretation of AUC (especially a value close to 0.5); (ii) model comparisons based on AUC; (iii) the differences between ranking and classification; (iv) effects due to multiple hypotheses testing; (v) the importance of confidence intervals for AUC; and (vi) the choice of the appropriate performance metric. With a discussion of illustrative examples and concrete real-world studies, this article highlights critical misconceptions that can profoundly impact the conclusions about the observed performance.

Resumo Limpo

receiv oper characterist roc emerg gold standard assess compar perform classifi wide rang disciplin includ life scienc roc curv frequent summar singl scalar area curv auc articl discuss caveat pitfal roc analysi clinic microarray research particular relat interpret auc especi valu close ii model comparison base auc iii differ rank classif iv effect due multipl hypothes test v import confid interv auc vi choic appropri perform metric discuss illustr exampl concret realworld studi articl highlight critic misconcept can profound impact conclus observ perform

Resumos Similares

Comput Math Methods Med - Variable selection in ROC regression. ( 0,77080205633134 )
BMC Med Inform Decis Mak - Use of outcomes to evaluate surveillance systems for bioterrorist attacks. ( 0,748097867837146 )
BMC Med Inform Decis Mak - Artificial neural network models for prediction of cardiovascular autonomic dysfunction in general Chinese population. ( 0,7232034414632 )
BMC Med Inform Decis Mak - A three-step approach for the derivation and validation of high-performing predictive models using an operational dataset: congestive heart failure readmission case study. ( 0,717772444545241 )
Appl Clin Inform - Comparing predictions made by a prediction model, clinical score, and physicians: pediatric asthma exacerbations in the emergency department. ( 0,697435637070187 )
Int J Med Inform - Application of data mining to the identification of critical factors in patient falls using a web-based reporting system. ( 0,696714493126048 )
J Am Med Inform Assoc - An improved model for predicting postoperative nausea and vomiting in ambulatory surgery patients using physician-modifiable risk factors. ( 0,695127699076179 )
Comput. Biol. Med. - A ternary model of decompression sickness in rats. ( 0,693517722504135 )
J Biomed Inform - Decision-making model for early diagnosis of congestive heart failure using rough set and decision tree approaches. ( 0,692285148333327 )
BMC Med Inform Decis Mak - Evaluation of prediction models for the staging of prostate cancer. ( 0,691544171324431 )
IEEE J Biomed Health Inform - The effect of sample age and prediction resolution on myocardial infarction risk prediction. ( 0,688359493418129 )
Med Decis Making - Application of an artificial neural network to predict postinduction hypotension during general anesthesia. ( 0,678309672425207 )
Lifetime Data Anal - Understanding increments in model performance metrics. ( 0,674879967551257 )
Comput. Biol. Med. - Pre-operative prediction of surgical morbidity in children: comparison of five statistical models. ( 0,673733737641014 )
Comput Methods Programs Biomed - Single stage and multistage classification models for the prediction of liver fibrosis degree in patients with chronic hepatitis C infection. ( 0,671817379726564 )
J Biomed Inform - Not just data: a method for improving prediction with knowledge. ( 0,667472002647914 )
J. Comput. Biol. - Prediction of siRNA potency using sparse logistic regression. ( 0,667456844554401 )
Brief. Bioinformatics - Adjusting confounders in ranking biomarkers: a model-based ROC approach. ( 0,662783466172686 )
J Chem Inf Model - Two new parameters based on distances in a receiver operating characteristic chart for the selection of classification models. ( 0,656356274004035 )
Comput Math Methods Med - Modified logistic regression models using gene coexpression and clinical features to predict prostate cancer progression. ( 0,655287809739806 )
AMIA Annu Symp Proc - Developing predictive models using electronic medical records: challenges and pitfalls. ( 0,652939217080305 )
Comput Methods Programs Biomed - Recurrence predictive models for patients with hepatocellular carcinoma after radiofrequency ablation using support vector machines with feature selection methods. ( 0,651568427153189 )
Int J Health Geogr - Prediction of high-risk areas for visceral leishmaniasis using socioeconomic indicators and remote sensing data. ( 0,650165221971119 )
J Clin Monit Comput - Use of genetic programming, logistic regression, and artificial neural nets to predict readmission after coronary artery bypass surgery. ( 0,64989689613719 )
Comput Methods Programs Biomed - Development of a daily mortality probability prediction model from Intensive Care Unit patients using a discrete-time event history analysis. ( 0,646002610986623 )
J Med Syst - Classifying hospitals as mortality outliers: logistic versus hierarchical logistic models. ( 0,645686851234818 )
Comput Methods Programs Biomed - Prediction of postprandial blood glucose under uncertainty and intra-patient variability in type 1 diabetes: a comparative study of three interval models. ( 0,645040048340853 )
Med Decis Making - A comparison of methods for converting DCE values onto the full health-dead QALY scale. ( 0,643892548760236 )
J Biomed Inform - Statistical process control for validating a classification tree model for predicting mortality--a novel approach towards temporal validation. ( 0,642965006536308 )
J Biomed Inform - Prediction of influenza vaccination outcome by neural networks and logistic regression. ( 0,642704936163254 )
J Med Syst - Effective automated prediction of vertebral column pathologies based on logistic model tree with SMOTE preprocessing. ( 0,642193971873683 )
Methods Inf Med - Limited sampling strategies to estimate the area under the concentration-time curve. Biases and a proposed more accurate method. ( 0,64127291959923 )
J Chem Inf Model - Are bigger data sets better for machine learning? Fusing single-point and dual-event dose response data for Mycobacterium tuberculosis. ( 0,634560041810715 )
Med Decis Making - Adaptation of clinical prediction models for application in local settings. ( 0,633761165825513 )
Methods Inf Med - Classification of postural profiles among mouth-breathing children by learning vector quantization. ( 0,630194185332169 )
BMC Med Inform Decis Mak - Prediction of axillary lymph node metastasis in primary breast cancer patients using a decision tree-based model. ( 0,62616280742118 )
Med Decis Making - Lehmann family of ROC curves. ( 0,625519593484292 )
Artif Intell Med - Machine learning for improved pathological staging of prostate cancer: a performance comparison on a range of classifiers. ( 0,622343502805619 )
Med Decis Making - Performance of a mathematical model to forecast lives saved from HIV treatment expansion in resource-limited settings. ( 0,620930244479576 )
Med Decis Making - Performance profiling in primary care: does the choice of statistical model matter? ( 0,62055223669911 )
Spat Spatiotemporal Epidemiol - Assessment of land use factors associated with dengue cases in Malaysia using Boosted Regression Trees. ( 0,618615816729141 )
J Biomed Inform - Partial least squares and logistic regression random-effects estimates for gene selection in supervised classification of gene expression data. ( 0,61861424173675 )
AMIA Annu Symp Proc - Predicting Surgical Risk: How Much Data is Enough? ( 0,61548641691898 )
Med Decis Making - Development of inpatient risk stratification models of acute kidney injury for use in electronic health records. ( 0,61512314588396 )
BMC Med Inform Decis Mak - Mining geriatric assessment data for in-patient fall prediction models and high-risk subgroups. ( 0,613611738928423 )
Neural Comput - An extension of the receiver operating characteristic curve and AUC-optimal classification. ( 0,613239256915898 )
Methods Inf Med - Extending statistical boosting. An overview of recent methodological developments. ( 0,612498103103039 )
BMC Med Inform Decis Mak - Bayesian predictors of very poor health related quality of life and mortality in patients with COPD. ( 0,612037160949591 )
J Biomed Inform - Towards probabilistic decision support in public health practice: predicting recent transmission of tuberculosis from patient attributes. ( 0,611298554387135 )
BMC Med Inform Decis Mak - Non-linear dynamical signal characterization for prediction of defibrillation success through machine learning. ( 0,611208746263382 )
Comput. Biol. Med. - A knowledge-driven probabilistic framework for the prediction of protein-protein interaction networks. ( 0,610139584691403 )
BMC Med Inform Decis Mak - Artificial neural network aided non-invasive grading evaluation of hepatic fibrosis by duplex ultrasonography. ( 0,604679838767513 )
Comput Math Methods Med - Iterative reweighted noninteger norm regularizing SVM for gene expression data classification. ( 0,604164407259552 )
Spat Spatiotemporal Epidemiol - Modeling habitat suitability for occurrence of highly pathogenic avian influenza virus H5N1 in domestic poultry in Asia: a spatial multicriteria decision analysis approach. ( 0,604076867704403 )
Artif Intell Med - Prediction of human major histocompatibility complex class II binding peptides by continuous kernel discrimination method. ( 0,603826065523496 )
IEEE Trans Image Process - Network-based H.264/AVC whole frame loss visibility model and frame dropping methods. ( 0,60333158459629 )
AMIA Annu Symp Proc - Decision path models for patient-specific modeling of patient outcomes. ( 0,601407667507549 )
Appl Clin Inform - Exploring the value of clinical data standards to predict hospitalization of home care patients. ( 0,599806354272491 )
BMC Med Inform Decis Mak - Prediction of adverse cardiac events in emergency department patients with chest pain using machine learning for variable selection. ( 0,598812263800745 )
Comput Methods Programs Biomed - ThyroScreen system: high resolution ultrasound thyroid image characterization into benign and malignant classes using novel combination of texture and discrete wavelet transform. ( 0,59823145459296 )
Med Decis Making - Constructing proper ROCs from ordinal response data using weighted power functions. ( 0,598052789447447 )
Comput Biol Chem - Using ensemble methods to deal with imbalanced data in predicting protein-protein interactions. ( 0,597864518697486 )
J Am Med Inform Assoc - Supervised embedding of textual predictors with applications in clinical diagnostics for pediatric cardiology. ( 0,596219495447873 )
Artif Intell Med - Predicting patient survival after liver transplantation using evolutionary multi-objective artificial neural networks. ( 0,595100322309239 )
Comput Math Methods Med - Prediction of BP reactivity to talking using hybrid soft computing approaches. ( 0,59194214216481 )
Lifetime Data Anal - Estimating improvement in prediction with matched case-control designs. ( 0,591152100804657 )
J Chem Inf Model - Predictive toxicology modeling: protocols for exploring hERG classification and Tetrahymena pyriformis end point predictions. ( 0,588234793864181 )
J Med Syst - A new approach: role of data mining in prediction of survival of burn patients. ( 0,586371629245766 )
Med Biol Eng Comput - Mortality prediction of rats in acute hemorrhagic shock using machine learning techniques. ( 0,585793133750617 )
J Biomed Inform - An empirical approach to model selection through validation for censored survival data. ( 0,585700917701727 )
Methods Inf Med - A probabilistic model to investigate the properties of prognostic tools for falls. ( 0,585136310928564 )
Comput Methods Programs Biomed - Tight glycemic control in critical care--the leading role of insulin sensitivity and patient variability: a review and model-based analysis. ( 0,585074925778403 )
Artif Intell Med - Predicting the need for CT imaging in children with minor head injury using an ensemble of Naive Bayes classifiers. ( 0,584973163449033 )
BMC Med Inform Decis Mak - Decision curve analysis revisited: overall net benefit, relationships to ROC curve analysis, and application to case-control studies. ( 0,584615979828879 )
J Biomed Inform - Private predictive analysis on encrypted medical data. ( 0,584604243167695 )
BMC Med Inform Decis Mak - Computerized prediction of intensive care unit discharge after cardiac surgery: development and validation of a Gaussian processes model. ( 0,584509688996108 )
Int J Health Geogr - Identifying malaria vector breeding habitats with remote sensing data and terrain-based landscape indices in Zambia. ( 0,583894028556904 )
Brief. Bioinformatics - Critical assessment of high-throughput standalone methods for secondary structure prediction. ( 0,58356812698785 )
Artif Intell Med - Machine learning of clinical performance in a pancreatic cancer database. ( 0,579438264331692 )
Comput Biol Chem - An ensemble method for prediction of conformational B-cell epitopes from antigen sequences. ( 0,575383816822074 )
Int J Health Geogr - Modeling larval malaria vector habitat locations using landscape features and cumulative precipitation measures. ( 0,573063889285483 )
J Chem Inf Model - Ligand efficiency-based support vector regression models for predicting bioactivities of ligands to drug target proteins. ( 0,571030250291173 )
Med Decis Making - Contrasting two frameworks for ROC analysis of ordinal ratings. ( 0,566703882673422 )
AMIA Annu Symp Proc - Development and implementation of a real-time 30-day readmission predictive model. ( 0,565686104819973 )
Med Biol Eng Comput - System identification of the mechanomyogram from single motor units during voluntary isometric contraction. ( 0,565657026973974 )
BMC Med Inform Decis Mak - Predicting the start week of respiratory syncytial virus outbreaks using real time weather variables. ( 0,564679103686604 )
Comput Methods Programs Biomed - Exploring an optimal vector autoregressive model for multi-channel pulmonary sound data. ( 0,563709005630716 )
J Med Syst - Comparison of artificial neural networks with logistic regression for detection of obesity. ( 0,559724987873811 )
J Am Med Inform Assoc - Machine learning for predicting the response of breast cancer to neoadjuvant chemotherapy. ( 0,557698075104265 )
J. Med. Internet Res. - Maximizing the value of mobile health monitoring by avoiding redundant patient reports: prediction of depression-related symptoms and adherence problems in automated health assessment services. ( 0,555975894950603 )
Methods Inf Med - Sensor-based fall risk assessment--an expert 'to go'. ( 0,555573478430263 )
Artif Intell Med - Artificial metaplasticity prediction model for cognitive rehabilitation outcome in acquired brain injury patients. ( 0,551386849734091 )
J Am Med Inform Assoc - Automating annotation of information-giving for analysis of clinical conversation. ( 0,550262841356454 )
Med Biol Eng Comput - Towards personalized clinical in-silico modeling of atrial anatomy and electrophysiology. ( 0,549465568781326 )
Artif Intell Med - White box radial basis function classifiers with component selection for clinical prediction models. ( 0,548798052650228 )
Comput. Biol. Med. - Statistical model based 3D shape prediction of postoperative trunks for non-invasive scoliosis surgery planning. ( 0,548724933839544 )
Brief. Bioinformatics - Added predictive value of high-throughput molecular data to clinical data and its validation. ( 0,548562592864281 )
J Am Med Inform Assoc - Design-phase prediction of potential cancer clinical trial accrual success using a research data mart. ( 0,54774460489659 )
J Clin Monit Comput - Predictive data mining on monitoring data from the intensive care unit. ( 0,54704369805562 )
Artif Intell Med - Operation room tool handling and miscommunication scenarios: an object-process methodology conceptual model. ( 0,544711597304126 )