Original article| Volume 60, ISSUE 9, P1013-1019, November 01, 2006

Download started.


Comparing the Effects of Sampling Designs on the Diagnostic Accuracy of Eight Promising Screening Algorithms for Pediatric Bipolar Disorder


      Past studies have used markedly different inclusion and exclusion criteria to form samples used to evaluate diagnostic tests, making it difficult to compare results across studies. The present investigation compared eight screening algorithms in the same sample but under two different design strategies.


      The DSM-IV diagnoses were based on a semi-structured diagnostic interview (KSADS) with the parent and youth sequentially. Raters were blind to index test scores. Participants were 216 youths with bipolar spectrum diagnoses and 284 youths with other Axis I diagnoses or no diagnosis. T-tests evaluated whether areas under the curve (AUC) from receiver operating characteristic analyses differed under the two design conditions.


      All of the instruments discriminated bipolar cases better when inclusion and exclusion criteria duplicated those used in phenomenological research studies selecting narrow phenotypic cases (AUCs ranging from .90 to .81). The measures performed less well when more heterogeneous clinical presentations were included [AUCs ranging from .86 to .69, t(8) = 4.99, p = 001].


      Results indicate that checklists perform less well discriminating pediatric bipolar disorder under conditions that more closely resemble clinical practice. Test users must consider whether the sampling strategies and participant characteristics used to evaluate tests match the characteristics of their own patients.

      Key Words

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Biological Psychiatry
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Achenbach T.M.
        Manual for the Child Behavior Checklist/4-18 and 1991 Profile. University of Vermont, Department of Psychiatry, Burlington1991
        • Allison P.D.
        Missing Data. Sage, Thousand Oaks, California2002
        • American Psychiatric Association
        Diagnostic and Statistical Manual of Mental Disorders. 4th ed.–Revised. Author, Washington DC2001
        • Anderson J.C.
        DSM-III disorders in preadolescent children: Prevalence in a large sample from the general population.
        Arch Gen Psychiatry. 1987; 44: 69-76
        • Biederman J.
        • Mick E.
        • Faraone S.V.
        Depression in attention deficit hyperactivity disorder (ADHD) children: “True” depression or demoralization?.
        J Affect Disord. 1998; 47: 113-122
        • Biederman J.
        • Wozniak J.
        • Kiely K.
        • Ablon S.
        • Faraone S.
        • Mick E.
        • et al.
        CBCL clinical scales discriminate prepubertal children with structured interview-derived diagnosis of mania from those with ADHD.
        J Am Acad Child Adolesc Psychiatry. 1995; 34: 464-471
        • Bird H.R.
        • Gould M.S.
        • Staghezza B.
        Aggregating data from multiple informants in child psychiatry epidemiological research.
        J Am Acad Child Adolesc Psychiatry. 1992; 31: 78-85
        • Bossuyt P.M.
        • Reitsma J.B.
        • Bruns D.E.
        • Gatsonis C.A.
        • Glasziou P.P.
        • Irwig L.M.
        • et al.
        Towards complete and accurate reporting of studies of diagnostic accuracy: The STARD initiative.
        BMJ. 2003; 326: 41-44
        • Bowring M.A.
        • Kovacs M.
        Difficulties in diagnosing manic disorders among children and adolescents.
        J Am Acad Child Adolesc Psychiatry. 1992; 31: 611-614
        • Dunn L.M.
        • Dunn L.M.
        The Peabody Picture Vocabulary Test – Revised: Manual for Forms L and M. American Guidance Service, Circle Pines, Minnesota1981
        • Faraone S.V.
        • Althoff R.R.
        • Hudziak J.J.
        • Monuteaux M.
        • Biederman J.
        The CBCL predicts DSM bipolar disorder in children: A receiver operating characteristic curve analysis.
        Bipolar Disord. 2005; 7: 518-524
        • Findling R.L.
        • Gracious B.L.
        • McNamara N.K.
        • Youngstrom E.A.
        • Demeter C.
        • Calabrese J.R.
        Rapid, continuous cycling and psychiatric co-morbidity in pediatric bipolar I disorder.
        Bipolar Disord. 2001; 3: 202-210
        • Findling R.L.
        • Youngstrom E.A.
        • Danielson C.K.
        • DelPorto-Bedoya D.
        • Papish-David R.
        • Townsend L.
        • Calabrese J.
        Clinical decision-making using the General Behavior Inventory in juvenile bipolarity.
        Bipolar Disord. 2002; 4: 34-42
        • Geller B.
        • Craney J.L.
        • Bolhofner K.
        • DelBello M.P.
        • Axelson D.
        • Luby J.
        • et al.
        Phenomenology and longitudinal course of children with a prepubertal and early adolescent bipolar disorder phenotype.
        in: Geller B. DelBello M.P. Bipolar Disorder in Childhood and Early Adolescence. Guilford, New York2003: 25-50
        • Geller B.
        • Warner K.
        • Williams M.
        • Zimerman B.
        Prepubertal and young adolescent bipolarity versus ADHD: Assessment and validity using the WASH-U-KSADS, CBCL and TRF.
        J Affect Disord. 1998; 51: 93-100
        • Geller B.
        • Zimerman B.
        • Williams M.
        • Delbello M.P.
        • Bolhofner K.
        • Craney J.L.
        • et al.
        DSM-IV mania symptoms in a prepubertal and early adolescent bipolar disorder phenotype compared to attention-deficit hyperactive and normal controls.
        J Child Adolesc Psychopharmacol. 2002; 12: 11-25
      1. Guyatt G.H. Rennie D. Users’ Guides to the Medical Literature. AMA Press, Chicago2002
        • Hanley J.A.
        • McNeil B.J.
        A method of comparing the areas under receiver operating characteristic curves derived from the same cases.
        Radiology. 1983; 148: 839-843
        • Hazell P.L.
        • Lewin T.J.
        • Carr V.J.
        Confirmation that Child Behavior Checklist clinical scales discriminate juvenile mania from attention deficit hyperactivity disorder.
        J Paediatr Child Health. 1999; 35: 199-203
        • Kahana S.Y.
        • Youngstrom E.A.
        • Findling R.L.
        • Calabrese J.R.
        Employing parent, teacher, and youth self-report checklists in identifying pediatric bipolar spectrum disorders: An examination of diagnostic accuracy and clinical utility.
        J Child Adolesc Psychopharmacol. 2003; 13: 471-488
        • Kaufman J.
        • Birmaher B.
        • Brent D.
        • Rao U.
        • Flynn C.
        • Moreci P.
        • et al.
        Schedule for Affective Disorders and Schizophrenia for School-Age Children-Present and Lifetime version (K-SADS-PL): Initial reliability and validity data.
        J Am Acad Child Adolesc Psychiatry. 1997; 36: 980-988
        • Kim E.Y.
        • Miklowitz D.J.
        Childhood mania, attention deficit hyperactivity disorder and conduct disorder: A critical review of diagnostic dilemmas.
        Bipolar Disord. 2002; 4: 215-225
        • Kraemer H.C.
        Evaluating Medical Tests: Objective and Quantitative Guidelines. Sage Publications, Newbury Park, California1992
        • Leibenluft E.
        • Charney D.S.
        • Towbin K.E.
        • Bhangoo R.K.
        • Pine D.S.
        Defining clinical phenotypes of juvenile mania.
        Am J Psychiatry. 2003; 160: 430-437
        • Lengua L.J.
        • Sadowski C.A.
        • Friedrich W.N.
        • Fisher J.
        Rationally and empirically derived dimensions of children’s symptomatology: Expert ratings and confirmatory factor analyses of the CBCL.
        J Consult Clin Psychol. 2001; 69: 683-698
        • Mick E.
        • Biederman J.
        • Pandina G.
        • Faraone S.V.
        A preliminary meta-analysis of the child behavior checklist in pediatric bipolar disorder.
        Biol Psychiatry. 2003; 53: 1021-1027
        • Nottelmann E.
        • Biederman J.
        • Birmaher B.
        • Carlson G.A.
        • Chang K.D.
        • Fenton W.S.
        • et al.
        National Institute of Mental Health research roundtable on prepubertal bipolar disorder.
        J Am Acad Child Adolesc Psychiatry. 2001; 40: 871-878
        • Orvaschel H.
        Schizophrenia and Affective Disorders Schedule for children - Epidemiological Version (KSADS-E). Nova University, Ft. Lauderdale, Florida1995
        • Papolos D.F.
        • Papolos J.
        The Bipolar Child: The Definitive and Reassuring Guide to Childhood’s Most Misunderstood Disorder. 2nd ed. Broadway Books, New York2002
        • Poznanski E.O.
        • Miller E.
        • Salguero C.
        • Kelsh R.C.
        Preliminary studies of the reliability and validity of the Children’s Depression Rating Scale.
        J Am Acad Child Psychiatry. 1984; 23: 191-197
        • Quinn C.A.
        • Fristad M.A.
        Defining and identifying early onset bipolar spectrum disorder.
        Curr Psychiatry Rep. 2004; 6: 101-107
        • Sackett D.L.
        • Straus S.E.
        • Richardson W.S.
        • Rosenberg W.
        • Haynes R.B.
        Evidence-Based Medicine: How to Practice and Teach EBM. 2nd ed. Churchill Livingstone, New York2000
        • Shaffer D.
        • Gould M.S.
        • Brasic J.
        • Ambrosini P.
        • Fisher P.
        • Bird H.
        • Aluwahlia S.
        A children’s global assessment scale (CGAS).
        Arch Gen Psychiatry. 1983; 40: 1228-1231
        • Teplin L.A.
        • Abram K.M.
        • McClelland G.M.
        • Dulcan M.K.
        • Mericle A.A.
        Psychiatric disorders in youth in juvenile detention.
        Arch Gen Psychiatry. 2002; 59: 1133-1143
        • Tillman R.
        • Geller B.
        A brief screening tool for a prepubertal and early adolescent bipolar disorder phenotype.
        Am J Psychiatry. 2005; 162: 1214-1216
        • Tramontina S.
        • Schmitz M.
        • Polanczyk G.
        • Rohde L.A.
        Juvenile bipolar disorder in Brazil: Clinical and treatment findings.
        Biol Psychiatry. 2003; 53: 1043-1049
        • Young R.C.
        • Biggs J.T.
        • Ziegler V.E.
        • Meyer D.A.
        A rating scale for mania: Reliability, validity, and sensitivity.
        Br J Psychiatry. 1978; 133: 429-435
        • Youngstrom E.A.
        • Findling R.L.
        • Calabrese J.R.
        • Gracious B.L.
        • Demeter C.
        • DelPorto-Bedoya D.
        • Price M.
        Comparing the diagnostic accuracy of six potential screening instruments for bipolar disorder in youths aged 5 to 17 years.
        J Am Acad Child Adolesc Psychiatry. 2004; 43: 847-858
        • Youngstrom E.A.
        • Findling R.L.
        • Danielson C.K.
        • Calabrese J.R.
        Discriminative validity of parent report of hypomanic and depressive symptoms on the General Behavior Inventory.
        Psychol Assess. 2001; 13: 267-276
        • Youngstrom E.A.
        • Findling R.L.
        • Youngstrom J.K.
        • Calabrese J.R.
        Toward an evidence-based assessment of pediatric bipolar disorder.
        J Clin Child Adolesc Psychol. 2005; 34: 433-448
        • Youngstrom E.A.
        • Meyers O.I.
        • Demeter C.
        • Kogos Youngstrom J.
        • Morello L.
        • et al.
        Comparing diagnostic checklists for pediatric bipolar disorder in academic and community mental health settings.
        Bipolar Disord. 2005; 7: 507-517
        • Youngstrom E.A.
        • Youngstrom J.K.
        • Starr M.
        Bipolar diagnoses in community mental health: Achenbach CBCL profiles and patterns of comorbidity.
        Biol Psychiatry. 2005; 58: 569-575
        • Zhou X.-H.
        • Obuchowski N.A.
        • McClish D.K.
        Statistical Methods in Diagnostic Medicine. Wiley, New York2002