Jump to content

Evidence-based assessment/ROC Party/Ready to ROC

Add links
From Wikiversity
Click Here for Landing Page
Click Here for Landing Page
HGAPS New for Fall 2022: HGAPS and Psychology Conferences
Click Here for Landing Page
Click Here for Landing Page

HGAPS is finding new ways to make psychological science conferences more accessible!

Here are examples from APA 2022 and the JCCAP Future Directions Forum. Coming soon... ABCT!
~ More at HGAPS.org ~


This set of pages provides a practical introduction to Receiver Operating Characteristic (ROC) analyses, including sample code and examples generated with multiple popular statistical programs.

Example ROC Party page

[edit | edit source]

ROC sample code

[edit | edit source]

ROC Paper examples

[edit | edit source]

The following papers are a selection that use ROC in their analyses, or illustrate application of DLRs to cases and decision-making, or both.

In press

[edit | edit source]
  • Cohen, J.R., Adams, Z.W., Menon, S.V., Youngstrom, E.A., Bunnell, B.E., Acierno, R., Ruggiero, K.J., & Danielson, C.K. (in press). How should we assess for depression following a natural disaster? An ROC approach to post-disaster assessments in adolescents and adults. Journal of Affective Disorders.
  • Danielson, C.K. Cohen, J., Adams, Z., Youngstrom, E.A., Soltis, K., Amstadter, A., & Ruggiero, K. (in press). Clinical decision making following disasters: efficient identification of PTSD risk in adolescents. Journal of Abnormal Child Psychology.
  • Jarrett, M., Van Meter, A.R., Hilton, D., Youngstrom, E.A., & Ollendick, T. (in press). Evidence-based assessment of ADHD in youth using a Receiver Operating Characteristic (ROC) approach. Journal of Clinical Child and Adolescent Psychology.
  • Jo, B., Findling, R.L., Hastie, T., Youngstrom, E.A., Wang, C.P., Arnold, L.E., Fristad, M.A., Frazier, T.W., Birmaher, B., Gill, M.K., & Horwitz, S.M. (in press). Construction of longitudinal prediction targets using semi-supervised learning. Statistical Methods in Medical Research.
  • Mesman, E., Youngstrom, E.A., Juliana, N.K., Nolen, W.A., & Hillegers, M.H.J. (in press). Validation of the Seven Up Seven Down (7U7D) in bipolar offspring: screening and prediction of mood disorders. Findings from the Dutch Bipolar Offspring Study. Journal of Affective Disorders.
  • Ong, M. L., Youngstrom, E. A., **Chua, J. J. X., *Halverson, T. F., Horwitz, S. M., Storfer-Isser, A., . . . Findling, R. L. (2016). Comparing the CASI-4R and the PGBI-10M for differentiating bipolar spectrum disorders from other outpatient diagnoses in youth. Journal of Abnormal Child Psychology. doi:10.1007/s10802-016-0182-4
  • Ransom, D.M., Burns, A.R., Youngstrom, E.A., Vaughan, C.G., Sady, M.D., & Gioia, G.A. (in press). Applying an evidence-based assessment model to identify students at risk for perceived academic problems following concussion. Journal of the International Neuropsychological Society.
  • Salcedo, S., Fristad, M., Birmaher, B., Kowatch, R.A., Gadow, K.D., Phillips, M.L., Frazier, T.W., **Chen, Y-L., Arnold, L.E., Horwitz, S.M., Findling, R.L., & The LAMS Team (in press). Diagnostic efficiency of the Child and Adolescent Symptom Inventory (CASI-4R) depressive subscale for youth mood disorders. Journal of Clinical Child and Adolescent Psychology. doi: 10.1080/15374416.2017.1280807.
  • You, D.S., Youngstrom, E.A., Feeny, N.C., Youngstrom, J.K., & Findling, R.L. (in press). Comparing the diagnostic accuracy of five instruments for detecting posttraumatic stress disorder in youth. Journal of Clinical Child and Adolescent Psychology. doi:10.1080/15374416.2015.1030754

2016

[edit | edit source]
  • Algorta, G. P., Dodd, A. L., Stringaris, A., & Youngstrom, E. A. (2016). Diagnostic efficiency of the SDQ for parents to identify ADHD in the UK: a ROC analysis. European child & adolescent psychiatry, 25(9), 949–957. https://doi.org/10.1007/s00787-015-0815-0
  • Youngstrom, E. A., & Van Meter, A. (2016). Empirically supported assessment of children and adolescents. Clinical Psychology: Science and Practice, 23(4), 327–347. https://doi.org/10.1037/h0101738
  • Frazier TW, Klingemier EW, Beukemann M, Speer L, Markowitz L, Parikh S, Wexberg S, Giuliano K, Schulte E, Delahunty C, Ahuja V, Eng C, Manos MJ, Hardan AY, Youngstrom EA, & Strauss MS (2016). Development of an objective autism risk index using remote eye tracking. Journal of the American Academy of Child and Adolescent Psychiatry, 55, 301-309.
  • Jenkins, M.M., & Youngstrom, E.A. (2016). A randomized controlled trial of cognitive debiasing improves assessment and treatment selection for pediatric bipolar disorder. Journal of Consulting and Clinical Psychology, 84, 323-333.
  • Van Meter, A. R., You, D. S., *Halverson, T., Youngstrom, E. A., Birmaher, B., Fristad, M. A., Kowatch, R.A., Storfer-Isser, A., Horwitz, S.M., Frazier, T.W., Arnold, L.E., Findling, R.L., The LAMS Group. (2016). Diagnostic Efficiency of Caregiver Report on the SCARED for Identifying Youth Anxiety Disorders in Outpatient Settings. Journal of Clinical Child & Adolescent Psychology, 1-15. doi:10.1080/15374416.2016.1188698

2015

[edit | edit source]
  • Youngstrom, E.A., Hameed, A., Mitchell, M., Van Meter, A.R., Freeman, A.J., Perez Algorta, G., White, A.M., Clayton, P., Gelenberg, A., & Meyer, R.E. (2015). Direct comparison of the psychometric properties of multiple interview and patient-rated assessments of suicidal ideation and behavior in an adult psychiatric inpatient sample. Journal of Clinical Psychiatry, 76, 1676-82. doi:10.4088/JCP.14m09353
  • Youngstrom, E.A., & De Los Reyes, A. (2015). Moving towards cost-effectiveness in using psychophysiological measures in clinical assessment: Validity, decision-making, and adding value. Journal of Clinical Child and Adolescent Psychology, 44, 352-361. doi:10.1080/15374416.2014.913252
  • Youngstrom, E.A., *Choukas-Bradley, S., *Calhoun, C.D., & Jensen-Doss, A. (2015). Clinical guide to the evidence-based assessment approach to diagnosis and treatment. Cognitive and Behavioral Practice, 22, 20-35. doi: 10.1016/j.cbpra.2013.12.005
  • Lindhiem, O., Yu, L., Grasso, D.J., Kolko, D.J., & Youngstrom, E.A. (2015). Adapting the Posterior Probability of Diagnosis (PPOD) index to enhance evidence-based screening: An application to ADHD in primary care. Assessment, 22, 198-207. doi: 10.1177/1073191114540748
  • Prieto, M.L., Youngstrom, E.A., Ozerdem, A., Altinbas, K., Quiroz, D., Aydemir, O., Yalin, N., Geske, J.R., Feeder, S.E., Angst, J., & Frye, M.A. (2015). Different patterns of manic/hypomanic symptoms in depression: A pilot modification of the Hypomania Checklist - 32 to assess mixed depression. Journal of Affective Disorders, 172, 355-360. doi: 10.1016/j.jad.2014.09.047

2014

[edit | edit source]
  • Youngstrom, E.A. (2014). A primer on Receiver Operating Characteristic analysis and diagnostic efficiency statistics for pediatric psychology: We are ready to ROC. Journal of Pediatric Psychology, 39, 204-221. Special Issue: Quantitative Methodology (Guest Editors: Bryan Karaszia & Kristoffer Berlin); doi: 10.1093/jpepsy/jst062
  • Frazier, T., Youngstrom, E.A., Fristad, M., Demeter, C., Birmaher, B., Kowatch, R., . . . Findling, R. (2014). Improving clinical prediction of bipolar spectrum disorders in youth. Journal of Clinical Medicine, 3, 218-232. doi: 10.3390/jcm3010218
  • Lee, H.J., Joo, Y., Youngstrom, E.A., Yum, S.Y., Findling, R.L., & Kim, H.W. (2014). Diagnostic validity and reliability of a Korean version of the Parent and Adolescent General Behavior Inventories. Comprehensive Psychiatry, 55, 1730-1737. doi:10.1016/j.comppsych.2014.05.008
  • Pendergast, L. L., Youngstrom, E. A., Merkitch, K. G., Moore, K. A., Black, C. L., Abramson, L. Y., & Alloy, L. B. (2014). Differentiating bipolar disorder from unipolar depression and ADHD: the utility of the General Behavior Inventory. Psychological Assessment, 26, 195-206. doi: 10.1037/a0035138
  • Van Meter, A. R., Youngstrom, E. A., Youngstrom, J. K., Ollendick, T., Demeter, C., & Findling, R. L. (2014). Clinical decision-making about child and adolescent anxiety disorders using the Achenbach System of Empirically Based Assessment. Journal of Clinical Child & Adolescent Psychology.

2013

[edit | edit source]
  • Youngstrom, E.A. (2013). Future directions in psychological assessment: Combining evidence based medicine innovations with psychology’s historical strengths to enhance utility. Journal of Clinical Child and Adolescent Psychology, 42, 139-159. doi: 10.1080/15374416.2012.736358.
  • Algorta, G.P., Youngstrom, E.A., Phelps, J., *Jenkins, M.M., Youngstrom, J.K., & Findling, R.L. (2013). An inexpensive family index of risk for mood issues improves identification of pediatric bipolar disorder. Psychological Assessment, 25, 12-22. doi: 10.1037/a0029225

2012

[edit | edit source]
  • Freeman, A.J., Youngstrom, E.A., Demeter, C., Youngstrom, J.K., & Findling, R.L. (2012). Portability of a screener for pediatric bipolar disorder to a diverse setting. Psychological Assessment, 24, 341-351. doi: 10.1037/a0025617
  • Jenkins, M.M., Youngstrom, E.A., Youngstrom, J.K., Feeny, N.C. & Findling, R.L. (2012). Generalizability of evidence-based assessment recommendations for pediatric bipolar disorder. Psychological Assessment, 24, 269-281. doi: 10.1037/a0025775

2011

[edit | edit source]
  • Jenkins, M.M., Youngstrom, E.A., Washburn, J.J., & Youngstrom, J.K. (2011). Evidence-based strategies improve assessment of pediatric bipolar disorder by community practitioners. Professional Psychology: Research and Practice, 42, 121-129. doi: 10.1037/a0022506

2010

[edit | edit source]
  • Horwitz, S.M., Demeter, C., Pagano, M.E., Youngstrom, E.A., Fristad, M.A., Arnold, L.E., Birmaher, B., Gill, M.K., Axelson, D., Kowatch, R.A. & Findling, R.L. (2010). Identification of a childhood bipolar phenotype-the Longitudinal Assessment of Symptoms of Mania Study: Background, design and initial screening. Journal of Clinical Psychiatry, 71, 1511-1517. doi: 10.4088/JCP.09m05835yel2009
  • Youngstrom, E.A., & Kendall, P.C. (2009). Psychological science and bipolar disorder. Clinical Psychology: Science and Practice, 16, 93-97. doi: 10.1111/j.1468-2850.2009.01149.x
  • Youngstrom, E.A., *Freeman, A.J., & *Jenkins, M.M. (2009). The assessment of bipolar disorder in children and adolescents. Child and Adolescent Psychiatric Clinics of North America, 18, 353-390. doi: 10.1016/j.chc.2008.12.002
  • Meyer, S. E., Carlson, G. A., Youngstrom, E., Ronsaville, D. S., Martinez, P. E., Gold, P. W., Hakak, R., & Radke-Yarrow, M. (2009). Long-term outcomes of youth who manifested the CBCL-Pediatric Bipolar Disorder phenotype during childhood and/or adolescence. Journal of Affective Disorders, 113, 227-235. doi: 10.1016/j.jad.2008.05.024

2008

[edit | edit source]
  • Youngstrom, E.A., *Frazier, T.W., Demeter, C., Calabrese, J.R., & Findling, R.L. (2008). Developing a ten-item mania scale from the Parent General Behavior Inventory for children and adolescents. Journal of Clinical Psychiatry, 69, 831-839. doi:10.4088/JCP.v69n0517
  • Youngstrom, E.A., *Greene, J., & *Joseph, M. (2008). Comparing the psychometric properties of multiple teacher report instruments as predictors of bipolar disorder in children and adolescents. Journal of Clinical Psychology, 64, 382-401. doi: 10.1002/jclp.20462
  • Henry, D., Pavuluri, M., Youngstrom, E.A., & Birmaher, B. (2008). Accuracy of brief and full forms of a mania rating scale. Journal of Clinical Psychology, 64, 368-381. doi: 10.1002/jclp.20464
  • Meyers, O.I., & Youngstrom, E.A. (2008). A Parent General Behavior Inventory subscale to measure sleep disturbance in pediatric bipolar disorder. Journal of Clinical Psychiatry, 69, 840-843. doi: 10.1016/j.jad.20009.07.020

2007

[edit | edit source]
  • Patel, N.C., Patrick, D.M., Youngstrom, E.A., Strakowski, S.M., & DelBello, M.P. (2007). Response and remission in adolescent mania: Signal detection analyses of the Young Mania Rating Scale. Journal of the American Academy of Child and Adolescent Psychiatry, 46, 628-635. doi: 10.1097/chi.0b013e3180335ae4

2006

[edit | edit source]
  • Youngstrom, E.A., *Meyers, O.I., Youngstrom, J.K., Calabrese, J.R., & Findling, R.L. (2006). Comparing the effects of sampling designs on the diagnostic accuracy of eight promising screening algorithms for pediatric bipolar disorder. Biological Psychiatry, 60, 1013-1019. doi: 10.1016/j.biopsych.2006.06.023.
  • Frazier, T.W., & Youngstrom, E.A. (2006). Evidence based assessment of attention-deficit/hyperactivity disorder: Using multiple sources of information. Journal of the American Academy of Child and Adolescent Psychiatry, 45, 614-620. doi: 10.1097/01.chi.0000196597.09103.25

2005

[edit | edit source]
  • Youngstrom, E. A., *Meyers, O. I., Demeter, C., Kogos Youngstrom, J., Morello, L., Piiparinen, R., Feeny, N. C., Findling, R. L., & Calabrese, J. R. (2005). Comparing diagnostic checklists for pediatric bipolar disorder in academic and community mental health settings. Bipolar Disorders, 7, 507-517. doi: j.1399-5618.2005.00269.x
  • Youngstrom, E. A., Findling, R. L., Youngstrom, J. K., & Calabrese, J. R., (2005). Towards an evidence-based assessment of pediatric bipolar disorder. Journal of Clinical Child and Adolescent Psychology, 34, 433-448. Special Issue: Evidence-Based Assessment. doi: 10.1207/s15374424jccp3403_4
  • Youngstrom, E. A., & *Duax, J. (2005). Evidence Based Assessment of Pediatric Bipolar Disorder, Part 1: Base Rate and Family History. Journal of the American Academy of Child and Adolescent Psychiatry, 44, 712-717. doi: 10.1097/01.chi.0000162581.87710.bd
  • Youngstrom, E. A., & Kogos Youngstrom, J. (2005). Evidence Based Assessment of Pediatric Bipolar Disorder, Part 2: Incorporating Information from Behavior Checklists. Journal of the American Academy of Child and Adolescent Psychiatry, 44(8), 823-828. doi: 10.1097/01.chi.0000164589.10200.a4

2004

[edit | edit source]
  • Youngstrom, E. A., Findling, R. L., Calabrese, J. R., Gracious, B. L., Demeter, C., DelPorto Bedoya, D., & *Price, M.E. (2004). Comparing the diagnostic accuracy of six potential screening instruments for bipolar disorder in youths aged 5 to 17 years. Journal of the American Academy of Child & Adolescent Psychiatry, 43, 847-858. doi: 10.1097/01.chi.0000125091.35109.1e

2003

[edit | edit source]
  • Danielson, C. K., Youngstrom, E. A., Findling, R. L., & Calabrese, J. R. (2003). Discriminative validity of the General Behavior Inventory using youth report. Journal of Abnormal Child Psychology, 31, 29-39. doi: 10.1023/A:1021717231272

2002

[edit | edit source]
  • Findling, R. L., Youngstrom, E. A., DelPorto, D., Papish-David, R., Townsend, L., *Danielson, C. K., & Calabrese, J. R. (2002). Clinical decision-making using the General Behavior Inventory. Bipolar Disorders, 4, 34-42. doi: 10.1034/j.1399-5618.2002.40102.

2001

[edit | edit source]
  • Youngstrom, E.A., Findling, R.L., *Danielson, C.K., & Calabrese, J.R. (2001). Discriminative validity of parent report of hypomanic and depressive symptoms on the General Behavior Inventory. Psychological Assessment, 13, 267-276. doi: 10.1037//1040-3590.13.2.267
Retrieved from "https://en.wikiversity.org/w/index.php?title=Evidence-based_assessment/ROC_Party/Ready_to_ROC&oldid=2622889"