J Chem Inf Model - Comparative studies on some metrics for external validation of QSPR models.


{ model(2656) set(1616) predict(1553) }
{ assess(1506) score(1403) qualiti(1306) }
{ model(3480) simul(1196) paramet(876) }
{ high(1669) rate(1365) level(1280) }
{ data(3008) multipl(1320) sourc(1022) }
{ method(1219) similar(1157) match(930) }
{ perform(999) metric(946) measur(919) }
{ import(1318) role(1303) understand(862) }
{ measur(2081) correl(1212) valu(896) }
{ general(901) number(790) one(736) }
{ process(1125) use(805) approach(778) }
{ studi(1410) differ(1259) use(1210) }
{ health(3367) inform(1360) care(1135) }
{ can(981) present(881) function(850) }
{ decis(3086) make(1611) patient(1517) }
{ data(1737) use(1416) pattern(1282) }
{ studi(2440) review(1878) systemat(933) }
{ framework(1458) process(801) describ(734) }
{ clinic(1479) use(1117) guidelin(835) }
{ howev(809) still(633) remain(590) }
{ compound(1573) activ(1297) structur(1058) }
{ gene(2352) biolog(1181) express(1162) }
{ structur(1116) can(940) graph(676) }
{ survey(1388) particip(1329) question(1065) }
{ estim(2440) model(1874) function(577) }
{ model(3404) distribut(989) bayesian(671) }
{ can(774) often(719) complex(702) }
{ system(1976) rule(880) can(841) }
{ sequenc(1873) structur(1644) protein(1328) }
{ take(945) account(800) differ(722) }
{ data(1714) softwar(1251) tool(1186) }
{ search(2224) databas(1162) retriev(909) }
{ featur(1941) imag(1645) propos(1176) }
{ data(3963) clinic(1234) research(1004) }
{ system(1050) medic(1026) inform(1018) }
{ ehr(2073) health(1662) electron(1139) }
{ state(1844) use(1261) util(961) }
{ sampl(1606) size(1419) use(1276) }
{ activ(1138) subject(705) human(624) }
{ use(2086) technolog(871) perceiv(783) }
{ analysi(2126) use(1163) compon(1037) }
{ health(1844) social(1437) communiti(874) }
{ use(976) code(926) identifi(902) }
{ use(1733) differ(960) four(931) }
{ method(1969) cluster(1462) data(1082) }
{ imag(1947) propos(1133) code(1026) }
{ inform(2794) health(2639) internet(1427) }
{ imag(1057) registr(996) error(939) }
{ bind(1733) structur(1185) ligand(1036) }
{ featur(3375) classif(2383) classifi(1994) }
{ imag(2830) propos(1344) filter(1198) }
{ network(2748) neural(1063) input(814) }
{ imag(2675) segment(2577) method(1081) }
{ patient(2315) diseas(1263) diabet(1191) }
{ motion(1329) object(1292) video(1091) }
{ treatment(1704) effect(941) patient(846) }
{ surgeri(1148) surgic(1085) robot(1054) }
{ problem(2511) optim(1539) algorithm(950) }
{ error(1145) method(1030) estim(1020) }
{ chang(1828) time(1643) increas(1301) }
{ learn(2355) train(1041) set(1003) }
{ concept(1167) ontolog(924) domain(897) }
{ algorithm(1844) comput(1787) effici(935) }
{ extract(1171) text(1153) clinic(932) }
{ method(1557) propos(1049) approach(1037) }
{ design(1359) user(1324) use(1319) }
{ control(1307) perform(991) simul(935) }
{ model(2220) cell(1177) simul(1124) }
{ care(1570) inform(1187) nurs(1089) }
{ method(984) reconstruct(947) comput(926) }
{ case(1353) use(1143) diagnosi(1136) }
{ risk(3053) factor(974) diseas(938) }
{ research(1085) discuss(1038) issu(1018) }
{ model(2341) predict(2261) use(1141) }
{ visual(1396) interact(850) tool(830) }
{ perform(1367) use(1326) method(1137) }
{ studi(1119) effect(1106) posit(819) }
{ blood(1257) pressur(1144) flow(957) }
{ spatial(1525) area(1432) region(1030) }
{ record(1888) medic(1808) patient(1693) }
{ monitor(1329) mobil(1314) devic(1160) }
{ research(1218) medic(880) student(794) }
{ patient(2837) hospit(1953) medic(668) }
{ data(2317) use(1299) case(1017) }
{ age(1611) year(1155) adult(843) }
{ medic(1828) order(1363) alert(1069) }
{ signal(2180) analysi(812) frequenc(800) }
{ cost(1906) reduc(1198) effect(832) }
{ group(2977) signific(1463) compar(1072) }
{ first(2504) two(1366) second(1323) }
{ intervent(3218) particip(2042) group(1664) }
{ time(1939) patient(1703) rate(768) }
{ patient(1821) servic(1111) care(1106) }
{ cancer(2502) breast(956) screen(824) }
{ drug(1928) target(777) effect(648) }
{ result(1111) use(1088) new(759) }
{ implement(1333) system(1263) develop(1122) }
{ activ(1452) weight(1219) physic(1104) }
{ method(2212) result(1239) propos(1039) }
{ detect(2391) sensit(1101) algorithm(908) }


Quantitative structure-property relationship (QSPR) models used for prediction of property of untested chemicals can be utilized for prioritization plan of synthesis and experimental testing of new compounds. Validation of QSPR models plays a crucial role for judgment of the reliability of predictions of such models. In the QSPR literature, serious attention is now given to external validation for checking reliability of QSPR models, and predictive quality is in the most cases judged based on the quality of predictions of property of a single test set as reflected in one or more external validation metrics. Here, we have shown that a single QSPR model may show a variable degree of prediction quality as reflected in some variants of external validation metrics like Q?(F1), Q?(F2), Q?(F3), CCC, and r?(m) (all of which are differently modified forms of predicted variance, which theoretically may attain a maximum value of 1), depending on the test set composition and test set size. Thus, this report questions the appropriateness of the common practice of the "classic" approach of external validation based on a single test set and thereby derives a conclusion about predictive quality of a model on the basis of a particular validation metric. The present work further demonstrates that among the considered external validation metrics, r?(m) shows statistically significantly different numerical values from others among which CCC is the most optimistic or less stringent. Furthermore, at a given level of threshold value of acceptance for external validation metrics, r?(m) provides the most stringent criterion (especially with r?(m) at highest tolerated value of 0.2) of external validation, which may be adopted in the case of regulatory decision support processes.

Resumo Limpo

quantit structureproperti relationship qspr model use predict properti untest chemic can util priorit plan synthesi experiment test new compound valid qspr model play crucial role judgment reliabl predict model qspr literatur serious attent now given extern valid check reliabl qspr model predict qualiti case judg base qualiti predict properti singl test set reflect one extern valid metric shown singl qspr model may show variabl degre predict qualiti reflect variant extern valid metric like qf qf qf ccc rm differ modifi form predict varianc theoret may attain maximum valu depend test set composit test set size thus report question appropri common practic classic approach extern valid base singl test set therebi deriv conclus predict qualiti model basi particular valid metric present work demonstr among consid extern valid metric rm show statist signific differ numer valu other among ccc optimist less stringent furthermor given level threshold valu accept extern valid metric rm provid stringent criterion especi rm highest toler valu extern valid may adopt case regulatori decis support process

Resumos Similares

Artif Intell Med - Training artificial neural networks directly on the concordance index for censored data using genetic algorithms. ( 0,785476007869668 )
J Chem Inf Model - Real external predictivity of QSAR models. Part 2. New intercomparable thresholds for different validation criteria and the need for scatter plot inspection. ( 0,783475796226387 )
J Chem Inf Model - iLOGP: a simple, robust, and efficient description of n-octanol/water partition coefficient for drug design using the GB/SA approach. ( 0,778185659000705 )
J Chem Inf Model - Time-split cross-validation as a method for estimating the goodness of prospective prediction. ( 0,762404365956182 )
BMC Med Inform Decis Mak - Concordance and predictive value of two adverse drug event data sets. ( 0,753077773765169 )
J Chem Inf Model - Does rational selection of training and test sets improve the outcome of QSAR modeling? ( 0,752581721369186 )
J Chem Inf Model - Study of chromatographic retention of natural terpenoids by chemoinformatic tools. ( 0,73979449663335 )
J Chem Inf Model - Three useful dimensions for domain applicability in QSAR models using random forest. ( 0,736913166587864 )
J Chem Inf Model - GRID-based three-dimensional pharmacophores II: PharmBench, a benchmark data set for evaluating pharmacophore elucidation methods. ( 0,718338751048553 )
J Chem Inf Model - Rank order entropy: why one metric is not enough. ( 0,71696351769645 )
J Chem Inf Model - Beyond the scope of Free-Wilson analysis: building interpretable QSAR models with machine learning algorithms. ( 0,713476878540357 )
AMIA Annu Symp Proc - Effect of data combination on predictive modeling: a study using gene expression data. ( 0,710808261596835 )
AMIA Annu Symp Proc - Predicting the dengue incidence in Singapore using univariate time series models. ( 0,692364054658577 )
J Chem Inf Model - RS-Predictor models augmented with SMARTCyp reactivities: robust metabolic regioselectivity predictions for nine CYP isozymes. ( 0,689374989765057 )
Int J Health Geogr - Incorporating geographical factors with artificial neural networks to predict reference values of erythrocyte sedimentation rate. ( 0,684086826906495 )
J Chem Inf Model - Predicting pK(a) values of substituted phenols from atomic charges: comparison of different quantum mechanical methods and charge distribution schemes. ( 0,677571034351057 )
J Chem Inf Model - Leave-cluster-out cross-validation is appropriate for scoring functions derived from diverse protein data sets. ( 0,674881562927068 )
J Chem Inf Model - Prediction of linear cationic antimicrobial peptides based on characteristics responsible for their interaction with the membranes. ( 0,674268445912937 )
AMIA Annu Symp Proc - Advanced proficiency EHR training: effect on physicians' EHR efficiency, EHR satisfaction and job satisfaction. ( 0,670505787471428 )
Int J Comput Assist Radiol Surg - Assessing performance in brain tumor resection using a novel virtual reality simulator. ( 0,669409230134047 )
J Chem Inf Model - Pharmacophore assessment through 3-D QSAR: evaluation of the predictive ability on new derivatives by the application on a series of antitubercular agents. ( 0,665678689210899 )
AMIA Annu Symp Proc - Motivating the additional use of external validity: examining transportability in a model of glioblastoma multiforme. ( 0,655739540615366 )
J. Comput. Biol. - The complexity of the dirichlet model for multiple alignment data. ( 0,653131889446542 )
Comput. Biol. Med. - A prediction model of substrates and non-substrates of breast cancer resistance protein (BCRP) developed by GA-CG-SVM method. ( 0,653005136915673 )
J Chem Inf Model - Applicability Domain ANalysis (ADAN): a robust method for assessing the reliability of drug property predictions. ( 0,652076161618547 )
J Biomed Inform - MysiRNA: improving siRNA efficacy prediction using a machine-learning model combining multi-tools and whole stacking energy (G). ( 0,646769147072898 )
Comput. Biol. Med. - Quantification of contributions of molecular fragments for eye irritation of organic chemicals using QSAR study. ( 0,64626239738076 )
BMC Med Inform Decis Mak - Regression tree construction by bootstrap: model search for DRG-systems applied to Austrian health-data. ( 0,641583373245648 )
Comput Methods Programs Biomed - Kinetic modelling of haemodialysis removal of myoglobin in rhabdomyolysis patients. ( 0,638400204288098 )
J Chem Inf Model - Applicability domains for classification problems: Benchmarking of distance to models for Ames mutagenicity set. ( 0,637321427081908 )
J Chem Inf Model - Applicability domain based on ensemble learning in classification and regression analyses. ( 0,634936546082624 )
J Chem Inf Model - CSAR data set release 2012: ligands, affinities, complexes, and docking decoys. ( 0,633718802275131 )
J Chem Inf Model - A new approach to radial basis function approximation and its application to QSAR. ( 0,632709926511654 )
J Chem Inf Model - In silico prediction of aqueous solubility using simple QSPR models: the importance of phenol and phenol-like moieties. ( 0,632414371772741 )
BMC Med Inform Decis Mak - Measuring preferences for analgesic treatment for cancer pain: how do African-Americans and Whites perform on choice-based conjoint (CBC) analysis experiments? ( 0,631693968552028 )
J Chem Inf Model - Criterion for evaluating the predictive ability of nonlinear regression models without cross-validation. ( 0,628962356055014 )
J Chem Inf Model - Statistical analysis and compound selection of combinatorial libraries for soluble epoxide hydrolase. ( 0,627056116509666 )
Comput. Aided Surg. - Evaluation of a computational model to predict elbow range of motion. ( 0,624699160667224 )
J Chem Inf Model - Robust scoring functions for protein-ligand interactions with quantum chemical charge models. ( 0,62301812242289 )
J Chem Inf Model - Design of novel FLT-3 inhibitors based on dual-layer 3D-QSAR model and fragment-based compounds in silico. ( 0,622587245128298 )
J. Med. Internet Res. - A case study of the New York City 2012-2013 influenza season with daily geocoded Twitter data from temporal and spatiotemporal perspectives. ( 0,621345595518111 )
Comput Methods Programs Biomed - A predictive model of longitudinal, patient-specific colonoscopy results. ( 0,621113877257461 )
J Chem Inf Model - Impact of template choice on homology model efficiency in virtual screening. ( 0,618337584151995 )
J Chem Inf Model - Template CoMFA: the 3D-QSAR Grail? ( 0,616044167932879 )
Neural Comput - Molecular diffusion model of neurotransmitter homeostasis around synapses supporting gradients. ( 0,615929310630126 )
Med Decis Making - Developing a tuberculosis transmission model that accounts for changes in population health. ( 0,615262913555925 )
J Chem Inf Model - Development of novel 3D-QSAR combination approach for screening and optimizing B-Raf inhibitors in silico. ( 0,609949918527467 )
J Am Med Inform Assoc - Harvest: an open platform for developing web-based biomedical data discovery and reporting applications. ( 0,607794907321995 )
Artif Intell Med - Fuzzy model identification of dengue epidemic in Colombia based on multiresolution analysis. ( 0,60664753386972 )
J Biomed Inform - Transfer learning based clinical concept extraction on data from multiple sources. ( 0,604594610437758 )
J Chem Inf Model - Oversampling to overcome overfitting: exploring the relationship between data set composition, molecular descriptors, and predictive modeling methods. ( 0,602510558382602 )
Artif Intell Med - Improving predictive models of glaucoma severity by incorporating quality indicators. ( 0,602459510715226 )
Med Biol Eng Comput - Optimal design of clinical tests for the identification of physiological models of type 1 diabetes in the presence of model mismatch. ( 0,601582049194992 )
Int J Health Geogr - Comparative analysis of remotely-sensed data products via ecological niche modeling of avian influenza case occurrences in Middle Eastern poultry. ( 0,601108790564157 )
J Chem Inf Model - Best of both worlds: combining pharma data and state of the art modeling technology to improve in Silico pKa prediction. ( 0,600627026334123 )
J Chem Inf Model - Building a three-dimensional model of CYP2C9 inhibition using the Autocorrelator: an autonomous model generator. ( 0,597755376569559 )
J Chem Inf Model - Combined 3D-QSAR, molecular docking, and molecular dynamics study on piperazinyl-glutamate-pyridines/pyrimidines as potent P2Y12 antagonists for inhibition of platelet aggregation. ( 0,597422884321265 )
J Chem Inf Model - In silico prediction of total human plasma clearance. ( 0,596321591242567 )
Comput Methods Programs Biomed - Interstitial insulin kinetic parameters for a 2-compartment insulin model with saturable clearance. ( 0,595189593309204 )
Med Biol Eng Comput - Application of the RIMARC algorithm to a large data set of action potentials and clinical parameters for risk prediction of atrial fibrillation. ( 0,593628463408799 )
J Am Med Inform Assoc - Choosing blindly but wisely: differentially private solicitation of DNA datasets for disease marker discovery. ( 0,593308708742859 )
IEEE Trans Image Process - Neighborhood Supported Model Level Fuzzy Aggregation for Moving Object Segmentation. ( 0,590981791531951 )
J Chem Inf Model - Coping with unbalanced class data sets in oral absorption models. ( 0,588734885821078 )
Comput Methods Programs Biomed - Bayesian bivariate generalized Lindley model for survival data with a cure fraction. ( 0,58749129332136 )
J Chem Inf Model - Real external predictivity of QSAR models: how to evaluate it? Comparison of different validation criteria and proposal of using the concordance correlation coefficient. ( 0,586364699835911 )
J Chem Inf Model - Estimation of carcinogenicity using molecular fragments tree. ( 0,58269862295532 )
J. Comput. Biol. - Rich parameterization improves RNA structure prediction. ( 0,581133354652372 )
J Chem Inf Model - Classification of compounds with distinct or overlapping multi-target activities and diverse molecular mechanisms using emerging chemical patterns. ( 0,580591644568868 )
Med Decis Making - Prediction of health preference values from CD4 counts in individuals with HIV. ( 0,575947691170598 )
Comput Methods Programs Biomed - Predicting body fat percentage based on gender, age and BMI by using artificial neural networks. ( 0,575764886184744 )
Comput Math Methods Med - Multiscale autoregressive identification of neuroelectrophysiological systems. ( 0,574710958684164 )
J. Med. Internet Res. - The Ume? University Database of Facial Expressions: a validation study. ( 0,574214210365514 )
J Chem Inf Model - Introducing conformal prediction in predictive modeling. A transparent and flexible alternative to applicability domain determination. ( 0,570413627783718 )
Comput Methods Programs Biomed - Modeling the glucose regulatory system in extreme preterm infants. ( 0,56964383251752 )
J Chem Inf Model - In silico prediction of chemical Ames mutagenicity. ( 0,56914753676867 )
Med Biol Eng Comput - Accelerometry-based prediction of movement dynamics for balance monitoring. ( 0,56682060646072 )
J Chem Inf Model - Four-dimensional structure-activity relationship model to predict HIV-1 integrase strand transfer inhibition using LQTA-QSAR methodology. ( 0,564940997765794 )
J Chem Inf Model - Design and synthesis of new antioxidants predicted by the model developed on a set of pulvinic acid derivatives. ( 0,563989854643226 )
IEEE Trans Image Process - Incremental N-mode SVD for large-scale multilinear generative models. ( 0,563791621705022 )
Brief. Bioinformatics - Rediscovery rate estimation for assessing the validation of significant findings in high-throughput studies. ( 0,563033623253923 )
Med Decis Making - Predicting EQ-5D utility scores from the Seattle Angina Questionnaire in coronary artery disease: a mapping algorithm using a Bayesian framework. ( 0,561792966141505 )
Comput. Biol. Med. - Artificial neural network modelling of the results of tympanoplasty in chronic suppurative otitis media patients. ( 0,561510852419408 )
J Chem Inf Model - Analysis and study of molecule data sets using snowflake diagrams of weighted maximum common subgraph trees. ( 0,558389009193984 )
Comput Methods Programs Biomed - A therapy parameter-based model for predicting blood glucose concentrations in patients with type 1 diabetes. ( 0,556567066920076 )
Artif Intell Med - A machine learning-based approach to prognostic analysis of thoracic transplantations. ( 0,556036412554109 )
AMIA Annu Symp Proc - Ontology-based federated data access to human studies information. ( 0,554523902846742 )
Comput Methods Programs Biomed - A 5-component mathematical model for salt-induced hypertension in Dahl-S and Dahl-R rats. ( 0,554516470205825 )
J. Med. Internet Res. - Outsourcing medical data analyses: can technology overcome legal, privacy, and confidentiality issues? ( 0,5536069555627 )
J Chem Inf Model - Optimizing predictive performance of CASE Ultra expert system models using the applicability domains of individual toxicity alerts. ( 0,551319628790579 )
Spat Spatiotemporal Epidemiol - Spatial modelling of disease using data- and knowledge-driven approaches. ( 0,550287987786458 )
J Chem Inf Model - Quantitative structure-activity relationship models for ready biodegradability of chemicals. ( 0,549329708717991 )
J Am Med Inform Assoc - Examining construct and predictive validity of the Health-IT Usability Evaluation Scale: confirmatory factor analysis and structural equation modeling results. ( 0,547627894186386 )
J Chem Inf Model - A multiscale simulation system for the prediction of drug-induced cardiotoxicity. ( 0,547467677372185 )
Int J Health Geogr - A linear programming model for preserving privacy when disclosing patient spatial information for secondary purposes. ( 0,546941474344783 )
Neural Comput - A compartmental model of linear resonance and signal transfer in dendrites. ( 0,545711224543232 )
J Chem Inf Model - How accurately can we predict the melting points of drug-like compounds? ( 0,542928853278102 )
J Biomed Inform - Hospital information systems: measuring end user computing satisfaction (EUCS). ( 0,541153192194167 )
Lifetime Data Anal - Bayesian inference of the fully specified subdistribution model for survival data with competing risks. ( 0,539895050234806 )
J Chem Inf Model - Template CoMFA applied to 116 biological targets. ( 0,539197813739195 )
Med Biol Eng Comput - Use of a comprehensive numerical model to improve biventricular pacemaker temporization in patients affected by heart failure undergoing to CRT-D therapy. ( 0,537706422657859 )