Advertisement

Data-Driven Phenotypic Categorization for Neurobiological Analyses: Beyond DSM-5 Labels

Open AccessPublished:July 19, 2016DOI:https://doi.org/10.1016/j.biopsych.2016.06.027

      Abstract

      Background

      Data-driven approaches can capture behavioral and biological variation currently unaccounted for by contemporary diagnostic categories, thereby enhancing the ability of neurobiological studies to characterize brain-behavior relationships.

      Methods

      A community-ascertained sample of individuals (N = 347, 18–59 years of age) completed a battery of behavioral measures, psychiatric assessment, and resting-state functional magnetic resonance imaging in a cross-sectional design. Bootstrap-based exploratory factor analysis was applied to 49 phenotypic subscales from 10 measures. Hybrid hierarchical clustering was applied to resultant factor scores to identify nested groups. Adjacent groups were compared via independent samples t tests and chi-square tests of factor scores, syndrome scores, and psychiatric prevalence. Multivariate distance matrix regression examined functional connectome differences between adjacent groups.

      Results

      Reduction yielded six factors, which explained 77.8% and 65.4% of the variance in exploratory and constrained exploratory models, respectively. Hybrid hierarchical clustering of these six factors identified two, four, and eight nested groups (i.e., phenotypic communities). At the highest clustering level, the algorithm differentiated functionally adaptive and maladaptive groups. At the middle clustering level, groups were separated by problem type (maladaptive groups; internalizing vs. externalizing problems) and behavioral type (adaptive groups; sensation-seeking vs. extraverted/emotionally stable). Unique phenotypic profiles were also evident at the lowest clustering level. Group comparisons exhibited significant differences in intrinsic functional connectivity at the highest clustering level in somatomotor, thalamic, basal ganglia, and limbic networks.

      Conclusions

      Data-driven approaches for identifying homogenous subgroups, spanning typical function to dysfunction, not only yielded clinically meaningful groups, but also captured behavioral and neurobiological variation among healthy individuals.

      Keywords

      The limitations of categorical definitions of psychiatric illness for clinical practice (
      • Wakefield J.C.
      Misdiagnosing normality: Psychiatry׳s failure to address the problem of false positive diagnoses of mental disorder in a changing professional environment.
      ) and psychiatric research (
      • Cuthbert B.
      • Insel T.
      Toward the future of psychiatric diagnosis: The seven pillars of RDoC.
      ) are increasingly apparent. Although diagnostic labels defined in nosological systems such as the DSM-5 (
      American Psychiatric Association
      ) are needed for clinical practice, these systems impede the search for pathophysiological markers using epidemiologic, genetic, and neuroimaging approaches (
      • Hyman S.E.
      The diagnosis of mental disorders: The problem of reification.
      ). Given growing recognition of these limitations, the Research Domain Criteria (RDoC) project has called for the development of a new nosology (
      • Insel T.R.
      The NIMH Research Domain Criteria (RDoC) Project: Precision medicine for psychiatry.
      ). In response, empirical data are being used to identify target phenotypic domains and constructs to characterize psychopathology and guide psychological and neurobiological investigations.
      Not surprisingly, how to best delineate phenotypic domains or constructs to guide a nonsyndromal research framework is uncertain (
      • Casey B.J.
      • Craddock N.
      • Cuthbert B.N.
      • Hyman S.E.
      • Lee F.S.
      • Ressler K.J.
      DSM-5 and RDoC: Progress in psychiatry research?.
      ). Inherent to this pursuit is the varying utility of categorical and dimensional frameworks. Although dimensional models of psychopathology are widely supported (
      • Kraemer H.C.
      • Noda A.
      • O׳Hara R.
      Categorical versus dimensional approaches to diagnosis: methodological challenges.
      ,
      • Widiger T.A.
      • Samuel D.B.
      Diagnostic categories or dimensions? A question for the Diagnostic And Statistical Manual Of Mental Disorders--fifth edition.
      ,
      • Yee C.M.
      • Javitt D.C.
      • Miller G.A.
      Replacing dsm categorical analyses with dimensional analyses in psychiatry research: The research domain criteria initiative.
      ), fully dimensional perspectives have limitations with respect to clinical decision making (
      • First M.B.
      Clinical utility: A prerequisite for the adoption of a dimensional approach in DSM.
      ,
      • Weinberger D.R.
      • Glick I.D.
      • Klein D.F.
      Whither Research Domain Criteria (RDoC)?: The good, the bad, and the ugly.
      ). Further, it is unclear whether it is more expedient to derive phenotypic targets from existing models (based on existing data and theory), data-driven analytic approaches (
      • Biswal B.B.
      • Mennes M.
      • Zuo X.-N.
      • Gohel S.
      • Kelly C.
      • Smith S.M.
      • et al.
      Toward discovery science of human brain function.
      ,
      • Akil H.
      • Martone M.E.
      • Van Essen D.C.
      Challenges and Opportunities in Mining Neuroscience Data.
      ), or some combination. Psychiatric classification systems have variable derivations spanning clinical and research observations (e.g., DSM) as well as empirical assessment (e.g., Achenbach System of Empirically Based Assessment), with other entities exhibiting a combination (e.g., RDoC). Consensus-driven methods [e.g., DSM; also arguably RDoC (
      • Kraemer H.
      REsearch Domain Criteria (RDoC) and the dsm--Two methodological approaches to mental health diagnosis.
      )] can certainly provide valuable insights; however, data-driven approaches may be crucial for identifying more behaviorally refined biological phenotypes (
      • Insel T.R.
      • Cuthbert B.N.
      Brain disorders? Precisely.
      ) to address the profound heterogeneity evident in health and illness (
      • McGorry P.
      • Keshavan M.
      • Goldstone S.
      • Amminger P.
      • Allott K.
      • Berk M.
      • et al.
      Biomarkers and clinical staging in psychiatry.
      ,
      • Wardenaar K.J.
      • de Jonge P.
      Diagnostic heterogeneity in psychiatry: Toward an empirical solution.
      ). Fair et al. (
      • Fair D.A.
      • Bathula D.
      • Nikolas M.A.
      • Nigg J.T.
      Distinct neuropsychological subgroups in typically developing youth inform heterogeneity in children with ADHD.
      ) and Karalunas et al. (
      • Karalunas S.L.
      • Fair D.
      • Musser E.D.
      • Aykes K.
      • Iyer S.P.
      • Nigg J.T.
      Subtyping attention-deficit/hyperactivity disorder using temperament dimensions: Toward biologically based nosologic criteria.
      ) recently demonstrated the potential value of delineating groups by similarity and dissimilarity of individual phenotypic profiles (e.g., neuropsychological profiles, temperament profiles). Adopting community detection methodologies from graph theory, they successfully identified 1) six distinct neuropsychological profiles that capture normal variation and are modified by attention-deficit/hyperactivity disorder (
      • Fair D.A.
      • Bathula D.
      • Nikolas M.A.
      • Nigg J.T.
      Distinct neuropsychological subgroups in typically developing youth inform heterogeneity in children with ADHD.
      ) and 2) three temperamental phenotypes that showed intriguing biological differences as well as differential clinical outcomes (
      • Karalunas S.L.
      • Fair D.
      • Musser E.D.
      • Aykes K.
      • Iyer S.P.
      • Nigg J.T.
      Subtyping attention-deficit/hyperactivity disorder using temperament dimensions: Toward biologically based nosologic criteria.
      ).
      Here, we used the Nathan Kline Institute-Rockland Sample (NKI-RS) (
      • Nooner K.B.
      • Colcombe S.
      • Tobe R.
      • Mennes M.
      • Benedict M.
      • Moreno A.
      • et al.
      The NKI-Rockland Sample: A Model for Accelerating the Pace of Discovery Science in Psychiatry.
      ), a deeply phenotyped, community-ascertained multimodal imaging sample. Using data from adult participants (18–59 years of age) in the NKI-RS, we aimed to identify data-driven phenotypes, based on core behavioral features representing several domains of function (including personality or temperament, symptom features, interpersonal functioning, and behavioral tendencies). Our first aim was to identify phenotypic dimensions that accurately represent meaningful variation across multiple domains of behavior. Accordingly, we conducted a bootstrap-based exploratory factor analysis (EFA) on 49 subscales derived from 10 measures obtained for 347 participants. The second aim was to identify a nested hierarchy of homogenous participant groups via hybrid hierarchical clustering (HHC) of participants, based on the factor profiles that we previously identified. To provide a phenotypic characterization of the participant groupings identified, we used DSM-IV labels and the Achenbach Adult Self-Report (ASR) (
      • Achenbach T.M.
      • Rescorla L.A.
      Manual for the ASEBA Adult Forms & Profiles.
      ), neither of which were included in the factor analysis. The third aim was to examine multivariate intrinsic brain functional connectivity differences among adjacent clusters and groups (derived from the first two aims).

      Methods and Materials

      Participants

      Participants were recruited as part of the NKI-RS (
      • Nooner K.B.
      • Colcombe S.
      • Tobe R.
      • Mennes M.
      • Benedict M.
      • Moreno A.
      • et al.
      The NKI-Rockland Sample: A Model for Accelerating the Pace of Discovery Science in Psychiatry.
      ), a community-based sample of approximately 1000 participants 6–85 years of age. To ascertain a cohort approximating a representative sample, exclusion criteria were minimal. Notably, comorbid medical conditions and medications (including psychotropics) were permitted. Written informed consent was obtained from all participants in accordance with local institutional review board oversight. The following inclusion criteria were applied: 1) 18–59 years of age; 2) absence of serious head injury or major neurological disorder; 3) negative history of bipolar disorder or psychosis; 4) negative drug test for commonly used illicit drugs with no therapeutic analogs or applications; and 5) at least 95% completion of each self-report measure examined.

      Subject Phenotyping

      All participants completed the Structured Clinical Interview for DSM-IV Axis I Disorders (SCID) (
      • First M.B.
      • Spitzer R.L.
      • Gibbon M.
      • Williams J.B.W.
      Structured Clinical Interview for DSM-IV Axis I Disorders.
      ), and the Edinburgh Handedness Inventory (
      • Oldfield R.C.
      The assessment and analysis of handedness: The Edinburgh inventory.
      ), in addition to measures reflecting clinical symptom domains, personality or temperament, and broad behavioral characteristics (see Supplemental Table S1) (
      • Nooner K.B.
      • Colcombe S.
      • Tobe R.
      • Mennes M.
      • Benedict M.
      • Moreno A.
      • et al.
      The NKI-Rockland Sample: A Model for Accelerating the Pace of Discovery Science in Psychiatry.
      ). We selected subscales rather than individual items or full-scale scores to balance depth of assessment and amount of data per subject.

      Magnetic Resonance Imaging Acquisition

      Imaging data were collected on a 3T Siemens TIM Trio system (Siemens Healthcare GmbH, Erlangen, Germany) equipped with 32-channel head coil. Both structural and resting state functional magnetic resonance imaging (R-fMRI) data were acquired. The structural image was a T1-weighted magnetization prepared gradient echo sequence: repetition time = 1900 ms, echo time = 2.52 ms, flip angle = 9°, 176 slices, 1 mm3 isotropic voxels. The R-fMRI data were acquired via multiband echo-planar imaging (
      • Xu J.
      • Moeller S.
      • Auerbach E.J.
      • Strupp J.
      • Smith S.M.
      • Feinberg D.A.
      • et al.
      Evaluation of slice accelerations using multiband echo planar imaging at 3 T.
      ) with the following parameters: volumes = 900, repetition time = 645 ms, echo time = 30 ms, flip angle = 60°, 3 mm3 isotropic voxels.

      Phenotypic Analysis

      Data Screening

      All self-report data were checked for univariate and multivariate outliers. We also tested the assumption of missingness at random (
      • Potthoff R.F.
      • Tudor G.E.
      • Pieper K.S.
      • Hasselblad V.
      Can one assess whether missing data are missing at random in medical studies?.
      ). Missing data were imputed using an expectation-maximization algorithm (
      • Gold M.S.
      • Bentler P.M.
      Treatments of missing data: A Monte Carlo comparison of RBHDI, iterative stochastic regression imputation, and expectation-maximization.
      ).

      Dimension Reduction

      For each participant, we created a multidimensional phenotypic profile using 49 subscale scores obtained from 10 questionnaires. Given modest intercorrelations among the questionnaires, we next performed an EFA to obtain a reduced set of dimensions. Analyses were done on age- and gender-regressed residuals of the 49 subscale scores to minimize the impact of these demographic variables on clustering (
      • Hudziak J.J.
      • Achenbach T.M.
      • Althoff R.R.
      • Pine D.S.
      A dimensional approach to developmental psychopathology.
      ). Parallel analysis (
      • Horn J.L.
      A rationale and test for the number of factors in factor analysis.
      ) of 10,000 permutations of the raw data (
      • Castellan N.J.
      Shuffling arrays: Appearances may be deceiving.
      ) was used to determine number of factors, comparing eigenvalues for each factor from the raw data with the 95th percentile of eigenvalues from the permutations. Maximum likelihood factor estimation with varimax rotation was used to estimate six factor loadings for each subscale score. Confidence intervals for factor loadings were estimated from 10,000 bootstrapped resamplings (
      • Zientek L.
      • Thompson B.
      Applying the bootstrap to the multivariate case: Bootstrap component/factor analysis.
      ). To minimize factor intercorrelations, factor loadings overlapping zero (95% confidence interval) or exhibiting values less than |0.25| were set to zero in a restricted model. Factor scores (i.e., equivalent of latent value per factor) were computed using regression estimation (
      • Grice J.W.
      Computing and evaluating factor scores.
      ).

      Clustering Analysis

      Categorical approaches (e.g., DSM) confer the ability to delineate unique combinations of above-threshold impairments in subsets of individuals. Building on this strength, a central goal of the present work is to identify phenotypically distinct groupings of participants among the larger sample (akin to categories), based on their dimensional six-factor phenotypic profiles. Specifically, we implemented HHC (
      • Chipman H.
      • Tibshirani R.
      Hybrid hierarchical clustering with applications to microarray data.
      ) using tree-structured vector quantization (
      • Gersho A.
      • Gray R.M.
      ) to identify nested participant groups based on Euclidean distances between participant factor score profiles. HHC combines agglomerative and divisive clustering by 1) identifying mutual clusters (i.e., groups of data that are exceptionally close to one another and as a group, distant from all others) via agglomerative clustering, 2) implementing constrained divisive clustering (retaining mutual clusters), and 3) applying additional divisive clustering, which explores the division of mutual clusters. In combination with visual examination of the dendrogram, we used the Calinski-Harabasz criterion (CHC) (
      • Caliński T.
      • Harabasz J.
      A dendrite method for cluster analysis. Commun.
      ) at each cluster number to inform cut decisions (i.e., where to divide into subgroups).

      Cluster Comparisons

      Pairwise comparisons were made among adjacent clusters at each level using EFA, ASR (
      • Achenbach T.M.
      • Rescorla L.A.
      Manual for the ASEBA Adult Forms & Profiles.
      ), and SCID profiles. EFA scores were fully dimensional, whereas the ASR and SCID diagnoses were categorical. A categorical version of ASR scores was achieved by mimicking common clinical strategies for its use, which apply cut-scores to identify meaningful psychopathology. For each grouping of participants identified via cluster analysis, we calculated the percentage of individuals exhibiting standardized scores (T scores) greater than or equal to 60 within each ASR domain. The cutoff of 60 was chosen to increase sensitivity to subthreshold symptoms of potential relevance. EFA scores were compared using independent-sample t tests, whereas ASR proportions and psychiatric diagnoses were compared using chi-square tests. We also conducted t tests on continuous ASR scores, reported in the Supplemental Results.

      MRI Data Processing

      R-fMRI Preprocessing

      Data were preprocessed using the Configurable Pipeline for Analysis of Connectomes (http://fcp-indi.github.io), which combines tools from AFNI (http://afni.nimh.nih.gov/afni), FSL (http://fsl.fmrib.ox.ac.uk/fsl/fslwiki/), and Advanced Normalization Tools (http://stnava.github.io/ANTs), using Nipype (
      • Gorgolewski K.
      • Burns C.D.
      • Madison C.
      • Clark D.
      • Halchenko Y.O.
      • Waskom M.L.
      • et al.
      Nipype: A flexible, lightweight and extensible neuroimaging data processing framework.
      ). Preprocessing included 1) motion correction, 2) mean-based intensity normalization, 3) nuisance signal regression, 4) temporal band-pass filtering (0.01–0.1 Hz), 5) coregistration of functional to structural images using boundary-based registration (
      • Greve D.N.
      • Fischl B.
      Accurate and robust brain image alignment using boundary-based registration.
      ) using FSL’s FLIRT (
      • Jenkinson M.
      • Bannister P.
      • Brady M.
      • Smith S.
      Improved optimization for the robust and accurate linear registration and motion correction of brain images.
      ), 6) normalization of functional image to Montreal Neurological Institute 152 template (
      • Avants B.B.
      • Epstein C.L.
      • Grossman M.
      • Gee J.C.
      Symmetric diffeomorphic image registration with cross-correlation: Evaluating automated labeling of elderly and neurodegenerative brain.
      ) by applying a nonlinear transform from Advanced Normalization Tools, and 7) smoothing with a full width at half maximum 6 mm Gaussian kernel. Nuisance regression removed linear and quadratic trends to account for scanner drift, 24 motion parameters, and five nuisance signals, identified via the component correction approach (CompCor) (
      • Behzadi Y.
      • Restom K.
      • Liau J.
      • Liu T.T.
      A Component Based Noise Correction Method (CompCor) for BOLD and perfusion based fMRI.
      ).

      Multivariate Distance Matrix Regression

      At each level of the hierarchy identified via HHC, we used multivariate distance matrix regression (MDMR) (
      • Shehzad A.
      • Kelly C.
      • Reiss P.T.
      • Craddock R.C.
      • Emerson J.W.
      • McMahon K.
      • et al.
      A multivariate distance-based analytic framework for connectome-wide association studies.
      ) to compare voxelwise functional connectivity profiles between adjacent phenotypic groups (e.g., cluster 1 [C1] vs. cluster 2 [C2], C1a vs. C1b, C2a vs. C2b).
      MDMR was performed on a voxel-by-voxel basis. At each voxel, the following three steps were carried out. First, for each participant, Pearson’s correlations were computed between the target voxel and all other voxels within a specified brain mask; this step generated, for each participant, a whole-brain functional connectivity map for the target voxel. Second, a between-participant distance matrix was computed, reflecting the distance between the connectivity maps obtained for the target voxel in two different participants. Distance was defined as √(2 * (1 – r)), where r is the spatial correlation of the connectivity maps obtained at the target voxel in two different individuals; the range of this distance metric is 0 (perfectly correlated) to 2 (perfectly negatively correlated), where 1 reflects no correlation. Importantly, these distances are calculated independent of any phenotypic relationships. Third, a pseudo-F statistic was computed to provide mathematical evaluation of the relationship between the variability in the distance matrix [computed in the second step (
      • Reiss P.T.
      • Stevens M.H.H.
      • Shehzad Z.
      • Petkova E.
      • Milham M.P.
      On distance-based permutation tests for between-group comparisons.
      )] and the variable of interest (i.e., group membership). Thus, the pseudo-F value at each voxel tells us whether the functional connectivity profiles for that voxel varied among individuals as a function of phenotypic group membership (e.g., C1 vs. C2, C1a vs. C1b, C2a vs. C2b).
      MDMR was applied using the Connectir package in R (http://czarrar.github.io/connectir) on resampled, 4 mm3 isotropic voxels. Computations were constrained to a study-specific group mask, including only voxels present across all participants and contained in a 25% probability gray matter Montreal Neurological Institute mask. The MDMR model (at each level of the dendrogram) specified cluster membership (categorical) and age, sex, hand laterality, and mean framewise displacement as covariates.
      Consistent with prior work, voxelwise significance of the pseudo-F statistic was determined via estimation of the null distribution with random permutation (n = 10,000) for each cluster–community comparison (
      • Shehzad A.
      • Kelly C.
      • Reiss P.T.
      • Craddock R.C.
      • Emerson J.W.
      • McMahon K.
      • et al.
      A multivariate distance-based analytic framework for connectome-wide association studies.
      ). Recent work has raised concerns about potentially inflated type I error rates with random field theory (RFT) cluster thresholding approaches (
      • Woo C.-W.
      • Krishnan A.
      • Wager T.D.
      Cluster-extent based thresholding in fMRI analyses: Pitfalls and recommendations.
      ,
      • Eklund A.
      • Nichols T.
      • Knutsson H.
      Can parametric statistical methods be trusted for fMRI based group studies?.
      ). These concerns are primarily applicable to parametric tests; nonparametric approaches are likely less prone to such inflations. However, we chose a more conservative thresholding approach than those used in prior studies implementing MDMR. Specifically, we corrected for multiple comparisons using cluster-based permutation (n = 5000) with a height threshold of Z greater than or equal to 2.33 (p < .01) and cluster extent probability of p less than .05.
      Given that lower levels of the hierarchy had few participants per group, statistical power in these comparisons would be expected to be notably lower. For illustrative purposes, we repeated analyses using less stringent criteria (voxelwise p < .05; RFT-corrected p < .05) and included these in the Supplemental Results.

      Results

      Demographic and diagnostic information is provided in Table 1. Details about data screening are provided in the Supplemental Results.
      Table 1Demographic and Diagnostic Characteristics
      CharacteristicFull Sample (N = 347)
      Age, Years37.5 ± 13.6
      Female66.0 (229)
      Right-Handed91.9 (319)
      Ethnicity
       Hispanic/Latino13.8 (47)
      Racial Background
       White/Caucasian67.6 (230)
       Black/African American20.3 (69)
       Asian7.6 (26)
       Native American/Pacific Islander1.2 (4)
       Other3.2 (11)
      Lifetime Psychiatric History
      n = 340; diagnostic information missing for 7.
       Any disorder49.1 (167)
       Depression21.2 (72)
       Anxiety
      Excluding obsessive-compulsive disorder and posttraumatic stress disorder.
      9.4 (32)
       Substance use disorder26.2 (89)
       ADHD1.8 (6)
      Current Psychiatric History
      n = 340; diagnostic information missing for 7.
       Any disorder10.9 (37)
       Depression2.6 (9)
       Anxiety
      Excluding obsessive-compulsive disorder and posttraumatic stress disorder.
      4.7 (16)
       Substance use disorder4.1 (14)
       ADHD0.9 (3)
      Number of Lifetime Psychiatry Diagnoses
       052.3 (178)
       125.3 (86)
       212.4 (42)
       3 or more10.0 (34)
      Values are mean ± SD or % (n).
      ADHD, attention-deficit/hyperactivity disorder.
      a n = 340; diagnostic information missing for 7.
      b Excluding obsessive-compulsive disorder and posttraumatic stress disorder.

      Dimension Reduction

      Parallel analysis suggested seven factors (Supplemental Figure S1), though the seventh factor exhibited factor loadings whose bootstrap-based 95% confidence intervals overlapped zero. Thus a six-factor solution was estimated with maximum likelihood–based EFA, accounting for 77.8% of the variance. The constrained model (eliminating low-loading items) accounted for 65.4% of the total variance; standardized factor loadings are presented in Supplemental Table S3. Correlations between latent factor scores and regression estimates are displayed in Supplemental Table S4. Primary factor loadings and example items for each subscale are provided in Figure 1. The six retained factors were interpreted as follows (see Figure 1): 1) general distress and impairment, 2) conscientiousness, 3) sensation and risk seeking, 4) frustration intolerance, 5) contextual sensitivity, and 6) neuroticism and negative affect. Details about each of the factors are provided in the Supplemental Results.
      Figure 1.
      Figure 1Factors and their corresponding subscales (along with example items) identified by exploratory analysis with 10,000 bootstrap resamplings. Subscale names are provided in the middle column. Sample items for each subscale are provided in the far right column. Color bar on the left provides an index to the shading of each subscale relative to its standardized loading on the factor. ATQ, Adult Temperament Questionnaire; BDI, Beck Depression Inventory-Second Edition; CAARS, Conners’ Adult ADHD Rating Scale; DOSP, Domain-Specific Risk-Taking Scale; ICU, Inventory of Callous and Unemotional Traits; IRI, Interpersonal Reactivity Index; NEO, NEO-Five Factor Inventory; (R), reverse-scored item in scale; STAI, Spielberger State-Trait Anxiety Inventory; TSC40, Trauma Symptom Checklist; UPPS, Impulsive Behavior Scale. aThese sample items were selected from the response option corresponding to 2 on a 0–3 scale.

      Cluster Analysis

      Visual inspection of the dendrogram suggested three clear cutpoints, yielding two, four, and eight groups, respectively. The largest CHC value was observed at k = 2 and stable subgroups (operationalized as values of k wherein CHC did not change appreciably from the prior solution [i.e., local minima]) at k = 4 and k = 8. Visual examination of participant-by-participant squared Euclidean distance matrices supported the face validity of these cutpoints (see Figure 2).
      Figure 2.
      Figure 2Results of hierarchical clustering. Panels in this figure depict the decision criteria used to ascertain clustering levels and resultant groups, as well as the similarity and dissimilarity of the groups as a function of correlation and squared Euclidean distance between factor scores (subject by subject). Panel (A) shows the dendrogram resulting from hybrid hierarchical clustering. It also shows the various levels at which the dendrogram was cut and the resultant groups. Panel (B) shows the Calinski-Harabasz criterion (CHC; black line), used as a decision aid for dendrogram cutting, as a function of cluster (C) number. The red line depicts percent change in the CHC value from one cluster to the next. Because the CHC did not exhibit a typical pattern (i.e., elevation at some cluster level), we defined stability (i.e., minimal change from one cluster number to the next) as our goal in deciding where to cut the dendrogram. Panel (C) again depicts the dendrogram, but relative to the squared Euclidean distance matrix. Groups and subgroups are outlined with dashed lines to help visualize group membership and increased similarity or decreased dissimilarity.

      Phenotypic Cluster Differences

      At all three levels of the dendrogram (i.e., two-, four-, and eight-cluster solutions), the participant groups differed from one another with respect to their phenotypic profiles. Due to space limitations, we limit reported findings beyond the first level (C1 vs. C2) to those along the C2 arm (which exhibited more psychopathology-like patterns). Phenotypic results along the C1 arm as well as for both groups together are provided in the Supplemental Results.

      Level 1

      Results at level 1 were robust, reflecting broad-reaching group differences that spanned nearly all domains. More specifically, C1 participants exhibited higher levels of adaptive functionality and C2 higher levels of maladaptive functionality; significant differences were noted in nearly all measures included in the three phenotypic profiles examined (EFA, ASR, psychiatric diagnosis [SCID]; see Figure 3, Supplemental Figure S2). To facilitate comparison, we also depicted phenotypic findings as heatmaps (and using a continuous score on the ASR) in Supplemental Figure S3.
      Figure 3.
      Figure 3Factor, clinical symptom, and lifetime psychiatric profiles visualized as radar plots by cluster (C) or group at three levels of hierarchical clustering, showing expansion within C2. All panels in this figure represent different measures pertaining to the same clusters [i.e., C2 is the same group of individuals, showing (A) variation in factor profiles, (B) Achenbach Adult Self-Report (ASR) clinical symptom profiles, and (C) lifetime psychiatric diagnosis]. Panel (A) represents mean values by cluster for each of the six factors from the exploratory factor analysis (EFA). Plots represent a standard loading of –1.5 at the origin and 1.5 at the maximum for each of the six factors. Panel (B) represents percent of individuals within a cluster exhibiting T scores ≥60 (1 SD above the mean; approaching clinical importance) for eight domains from the ASR. Plots represent zero at the center and 25% at the periphery (unless otherwise denoted) for each of the eight domains. Panel (C) represents percent of individuals within a cluster exhibiting a lifetime (i.e., past or current) psychiatric diagnosis (any diagnosis [ANY]; depressive disorder [DEPR]; anxiety disorder (ANX), excluding obsessive-compulsive disorder and posttraumatic stress disorder; substance use disorder [SUD]; attention-deficit/hyperactivity disorder [ADHD]). Where significant differences in current psychiatric diagnosis were observed, group percentages and significance are represented next to the diagnosis in italics. Plots represent zero at the center and 60% at the periphery (unless otherwise denoted) for each of the five diagnostic categories. Note that diagnoses are not mutually exclusive. Significant group differences are represented by asterisks: *p < .05, **p < .01, ***p < .001. Aggr Bx, aggressive behavior; Anx Depr, anxiety and depression; Atten Prob, attention problems; Consient, Conscientiousness; Context Sens., contextual sensitivity; Dx, diagnosis; Frust Intol, frustration intolerance; Intrus Bx, intrusive behavior; Rule Brk, rule breaking behavior; SCID, Structured Clinical Interview for DSM-IV Axis I Disorders; Sens Seeking, sensation seeking.

      Level 2

      The second level (k = 4) subdivided C1 (C1a and C1b) and C2 (C2a and C2b). C2 (functionally maladaptive group) was further divided into internalizing (C2a) and externalizing (C2b) problem characteristics. C2a had significantly higher ASR scores across all internalizing domains (Supplemental Figure S3). C2a also exhibited significantly higher rates of any lifetime psychiatric diagnosis and lifetime depression. C2b exhibited significantly higher levels of sensation and risk seeking on the EFA factor profile and significantly higher levels of ASR externalizing problems (see Figure 3).

      Level 3

      The third level (k = 8) divided the four clusters from level 2 into eight total subclusters (two clusters at level 3 for each cluster at level 2). Significant pairwise differences in ASR domains were largely a difference of magnitude (see Figure 3, Supplemental Figures S2, S3). Overall there were few significant pairwise differences between subclusters in DSM diagnoses, though notably C2a2 exhibited more current psychopathology than C2a1.

      Multivariate Intrinsic Connectivity Differences Among Clusters

      With permutation-based cluster correction, only MDMR findings from the first level (C1 vs. C2) survived multiple comparison correction (see Figure 4; see Supplemental Table S8 for functional peaks). This is not surprising given the larger number of participants in each group at the first level (C1-functionally adaptive: n = 165; C2-functionally maladaptive: n = 115), compared to the lower levels, which subdivided the sample into four and eight groups, respectively.
      Figure 4.
      Figure 4Results from multivariate distance matrix analysis of the functional connectome between cluster 1 (C1) and cluster 2 (C2). Adjacent groups at the highest level of hierarchical clustering (level 1: nC1 = 165, nC2 = 115) are displayed. Rendered brains and axial slices reflect multivariate distance matrix regression comparing intrinsic connectivity between groups; findings represent conversion of pseudo-F test results to Z values via permutation testing (10,000 resamplings of data) and permutation-based cluster correction (5000 resamplings of data) with cluster formation set at p less than .01 and extent threshold set at p less than .05. Note that images are presented in neurological convention (left [L] = right [R], R = L). The bottom row of axial slices represents significant findings at Montreal Neurological Institute axis Z values of –30, –20, –10, 0, and 10. From left to right, top to bottom, the top two thirds of the figure depict the lateral and medial surface of the right hemisphere, the dorsal and ventral surface of the right, then left, hemisphere, and finally, the dorsal and medial surface of the left hemisphere. Only level 1 results survived cluster permutation testing correction for multiple comparisons.
      Three clusters were identified for the first-level comparison of C1 versus C2. The largest cluster (k = 133,248 mm3, p = .0005) included the bilateral primary and secondary somatosensory cortices, as well as premotor, motor, and supplementary motor regions and was approximately centered on the midline near the supplementary motor area (X = 0, Y = –20, Z = 54); it also extended bilaterally to the lateral temporal lobes. These regions have been implicated in bodily self-awareness (
      • Blanke O.
      Multisensory brain mechanisms of bodily self-consciousness.
      ) and interoception (
      • Craig A.D.
      How do you feel? Interoception: The sense of the physiological condition of the body.
      ), and identified as comprising a major hub (
      • Tomasi D.
      • Shokri-Kojori E.
      • Volkow N.D.
      High-resolution functional connectivity density: Hub locations, sensitivity, specificity, reproducibility, and reliability.
      ) and critical network (
      • Smith S.M.
      • Fox P.T.
      • Miller K.L.
      • Glahn D.C.
      • Fox P.M.
      • Mackay C.E.
      • et al.
      Correspondence of the brain׳s functional architecture during activation and rest.
      ) of the functional brain. Additionally, emerging evidence suggests an important role for the somatosensory–somatomotor hub in high prevalence psychopathology (
      • Gabbay V.
      • Ely B.A.
      • Li Q.
      • Bangaru S.D.
      • Panzer A.M.
      • Alonso C.M.
      • et al.
      Striatum-based circuitry of adolescent depression and anhedonia.
      ,
      • Cui H.
      • Zhang J.
      • Liu Y.
      • Li Q.
      • Li H.
      • Zhang L.
      • et al.
      Differential alterations of resting-state functional connectivity in generalized anxiety disorder and panic disorder.
      ).
      The second largest cluster (k = 27,323 mm3, p = .0061) was approximately centered on the left thalamus (X = –16, Y = –20, Z = 14). In addition to a large thalamic contribution, the cluster included limbic regions (e.g., hippocampus, amygdala), decision-making regions (e.g., caudate, putamen), and various language (e.g., lingual gyrus) and vision regions (e.g., fusiform gyrus). It also extended from the left to right thalamus into the right caudate and putamen. The cluster comprised thalamic and basal ganglia regions commonly implicated in models of mental illness that emphasize thalamocortical and frontostriatal contributions (
      • Cui H.
      • Zhang J.
      • Liu Y.
      • Li Q.
      • Li H.
      • Zhang L.
      • et al.
      Differential alterations of resting-state functional connectivity in generalized anxiety disorder and panic disorder.
      ,
      • Buckholtz Joshua W.
      • Meyer-Lindenberg A.
      Psychopathology and the human connectome: Toward a transdiagnostic model of risk for mental illness.
      ).
      Finally, the third cluster (k = 14,528 mm3, p = .0167) was approximately centered on the right hippocampus (X = 32, Y = –24, Z = –18) and largely comprised right limbic regions (e.g., hippocampus and amygdala), as well as the caudate and putamen, fusiform gyrus, and middle and posterior insula. The regions implicated in this cluster, especially the right hippocampus and amygdala, are commonly associated with automated emotional processing (
      • Dyck M.
      • Loughead J.
      • Kellermann T.
      • Boers F.
      • Gur R.C.
      • Mathiak K.
      Cognitive versus automatic mechanisms of mood induction differentially activate left and right amygdala.
      ), particularly of the kind related to high prevalence psychopathological alterations (
      • Fairhall S.L.
      • Sharma S.
      • Magnusson J.
      • Murphy B.
      Memory related dysregulation of hippocampal function in major depressive disorder.
      ). Regions herein have also been identified as part of the medial temporal lobe subsystem of the default mode network (
      • Andrews-Hanna J.R.
      • Reidler J.S.
      • Sepulcre J.
      • Poulin R.
      • Buckner R.L.
      Functional-anatomic fractionation of the brain׳s default network.
      ).
      See the Supplemental Results for MDMR findings obtained at the second and third levels of the dendrogram using a more liberal thresholding strategy.

      Discussion

      Traditional psychiatric nosology comprises heterogeneous categories with few meaningful neurobiological correlates. The present findings illustrate the utility of data-driven approaches to 1) derive relevant phenotypic dimensions from diverse measures, 2) identify interpretable groups from hierarchical clustering of dimensional variables, and 3) use nested groups to examine potential neurobiological differences.
      Our findings suggest that inclusion of instruments with normal distributions (i.e., not truncated due to an assessment floor, such as an absence of symptoms [e.g., NEO Five Factor Inventory, Adult Temperament Questionnaire]) can be critical to defining relevant groups in population-based classification and among high-prevalence conditions (e.g., anxiety, depression, substance use). Although most individuals in the functionally adaptive group had truncated scores on syndrome-focused or problem-focused measures, our inclusion of bipolar scales allowed us to delineate further subgroups. The present work also highlights the value of including assessment tools that capture positive or protective factors rather than focusing solely on syndrome characteristics or problems. For example, conscientiousness, a personality characteristic associated with health and well-being (
      • Bogg T.
      • Roberts B.W.
      Conscientiousness and health-related behaviors: A meta-analysis of the leading behavioral contributors to mortality.
      ), was a large contributor to differentiating groups at the highest and lowest levels of the nested hierarchical classification. Protective factors are rarely assessed in syndrome- or problem-focused assessments, though they can have important implications for presentation and prognosis among neuropsychiatric conditions (
      • Belcher A.M.
      • Volkow N.D.
      • Moeller F.G.
      • Ferré S.
      Personality traits and vulnerability or resilience to substance use disorders.
      ).
      Within the limitations of the current sample size, the present work demonstrated the value of pursuing nested subgroups within both the functionally maladaptive (C2) and functionally adaptive (C1) participant groupings. Of concern, it is possible that the groups merely recapitulated the theories on which some of the measures were predicated. It is important to note, however, that no single scale had its subscores distributed in a manner that could explain all components derived from the factor analysis (arguably, the NEO Five Factor Inventory came the closest). We observed novel combinations of subscales in the factors derived, reflecting a range of psychiatric symptoms, personality features, and protective or risk factors.
      The present data also show the value of using broad behavioral characteristics for group identification to examine potential neurobiological differences. Multivariate comparisons of intrinsic connectivity were implemented at three different clustering levels, though the findings only passed stringent multiple comparisons correction at the first level, where power was the largest. The connectome differences observed between the functionally maladaptive and adaptive groups (C1 vs. C2) at the first level were evident within the 1) somatomotor network, 2) thalamic and basal ganglia regions, and 3) amygdala and extended hippocampal complex.
      The somatomotor network is a somewhat novel functional target in the context of identifying potential imaging biomarkers of the tendency toward psychiatric illness. Although known as a key hub (
      • Tomasi D.
      • Shokri-Kojori E.
      • Volkow N.D.
      High-resolution functional connectivity density: Hub locations, sensitivity, specificity, reproducibility, and reliability.
      ,
      • Smith S.M.
      • Fox P.T.
      • Miller K.L.
      • Glahn D.C.
      • Fox P.M.
      • Mackay C.E.
      • et al.
      Correspondence of the brain׳s functional architecture during activation and rest.
      ) of the functional brain, the potential role in psychopathology of the somatomotor network is only beginning to be recognized (
      • Gabbay V.
      • Ely B.A.
      • Li Q.
      • Bangaru S.D.
      • Panzer A.M.
      • Alonso C.M.
      • et al.
      Striatum-based circuitry of adolescent depression and anhedonia.
      ,
      • Cui H.
      • Zhang J.
      • Liu Y.
      • Li Q.
      • Li H.
      • Zhang L.
      • et al.
      Differential alterations of resting-state functional connectivity in generalized anxiety disorder and panic disorder.
      ). The somatosensory–motor network is intimately involved in bodily self-consciousness and interoception (
      • Blanke O.
      Multisensory brain mechanisms of bodily self-consciousness.
      ,
      • Craig A.D.
      How do you feel? Interoception: The sense of the physiological condition of the body.
      ), processes that are increasingly implicated in predictive outcome models (
      • Barrett L.F.
      • Simmons W.K.
      Interoceptive predictions in the brain.
      ), especially for neuropsychiatric illness (
      • Seth A.K.
      Interoceptive inference, emotion, and the embodied self.
      ). At a coarse level of group differentiation (e.g., functionally maladaptive vs. functionally adaptive), these results underscore the potential importance of subjective valuation and bodily states in making interactional predictions that may be fundamentally altered as part of the pathophysiology of psychiatric illness.
      Somewhat less novel, connectomic alterations in the thalamus and basal ganglia, as well as the amygdala and extended hippocampal complex, underscore the importance of basic cognitive functions (e.g., attention, working memory), as well as reward and emotion or saliency in delineating adaptive from maladaptive function. This latter finding lends support to the RDoC-style approach to examining domains rather than syndromes or diseases. On the whole, the intrinsic connectivity findings indicated potential new targets associated with adaptive and maladaptive function, while affirming existing targets, which suggests neurobiological validity of our data-driven group assignments.

      Limitations

      A number of steps were taken to protect against over- or underfitting the number of factors and provide robust estimates of which subscales loaded on which dimension. Factor number was determined via parallel analysis, factor structure was based on bootstrapping the raw data, and bootstrap-based confidence intervals were computed for estimates of standardized factor loadings. Despite all this, a separate confirmatory sample will be necessary to validate the replicability of the factor structure and the groups identified.
      The different response formats of the questionnaires could potentially have led to clustering of items by comparable response format and hence, recapitulation of the original scale factor structure. There was little evidence of such a result, though. To overcome clustering as a function of similar response formats, all questions would have to be administered with the same response format. However, such an approach would require extremely large samples and known psychometric properties of the scales would be forfeited.
      Significant connectome-wide differences decreased dramatically beyond the first level of hierarchical classification. Results at lower levels were only significant with less stringent, RFT multiple comparisons correction (see the Supplement). There are at least three plausible reasons for this decrease in significant findings: sample size, group homogeneity, and demographic differences. Average sample size was halved for each incremental level of the hierarchy. Thus, one contributor may merely be decreased power. Another potential factor is increasing subgroup homogeneity: group factor profiles became progressively more similar to one another at lower levels of the hierarchy. Demographic differences may also contribute to smaller connectome-wide differences at more refined levels of subgroup detection; groups may become more demographically similar as they become more behaviorally homogenous. However, cluster analyses were conducted on age and gender regression-residuals, mitigating some concerns regarding the influence of these demographic variables.
      Although biomarker differences may diminish as subgroups become more similar, it is exactly these kinds of comparisons that will ultimately permit differential classification based on differences in neurobiology (
      • Kapur S.
      • Phillips A.G.
      • Insel T.R.
      Why has it taken so long for biological psychiatry to develop clinical tests and what to do about it?.
      ). Larger samples within specific diagnostic categories or problem domains will likely provide more power to detect differences among more similar groups (
      • Fair D.A.
      • Bathula D.
      • Nikolas M.A.
      • Nigg J.T.
      Distinct neuropsychological subgroups in typically developing youth inform heterogeneity in children with ADHD.
      ,
      • Karalunas S.L.
      • Fair D.
      • Musser E.D.
      • Aykes K.
      • Iyer S.P.
      • Nigg J.T.
      Subtyping attention-deficit/hyperactivity disorder using temperament dimensions: Toward biologically based nosologic criteria.
      ,
      • Clementz B.A.
      • Sweeney J.A.
      • Hamm J.P.
      • Ivleva E.I.
      • Ethridge L.E.
      • Pearlson G.D.
      • et al.
      Identification of distinct psychosis biotypes using brain-based biomarkers.
      ) and can also remedy some of the problems that arise in exploratory analyses requiring complex multiple comparisons correction. By combining population-based, data-driven categorization and diagnostically focused pattern assessments, we can begin to compare the value of different classification methods. Currently, diagnostic heterogeneity is so marked that only extremely large samples (
      • Clementz B.A.
      • Sweeney J.A.
      • Hamm J.P.
      • Ivleva E.I.
      • Ethridge L.E.
      • Pearlson G.D.
      • et al.
      Identification of distinct psychosis biotypes using brain-based biomarkers.
      ) or time-consuming separation into individual diagnostic criteria permit head-to-head comparison of both methods (
      • Borsboom D.
      • Cramer A.O.J.
      Network analysis: An integrative approach to the structure of psychopathology.
      ).
      Finally, although the present work focused on phenotypic information alone for group classification or detection, alternative approaches might permit classification based entirely on neurobiology or on a combination of neurobiology and behavioral features. We opted to avoid classification purely on neurobiology as such approaches are complex (
      • Yang Z.
      • Xu Y.
      • Xu T.
      • Hoy C.W.
      • Handwerker D.A.
      • Chen G.
      • et al.
      Brain network informed subject community detection in early-onset schizophrenia.
      ), and interpretation of behavioral characteristics can be tricky when groups are clustered entirely on neurobiology. A potentially valuable alternative is analytic approaches that simultaneously consider phenotypic and neurobiological information (
      • Chen G.
      • Sullivan P.F.
      • Kosorok M.R.
      Biclustering with heterogeneous variance.
      ).

      Conclusions

      Psychiatry is limited by a lack of validated biobehavioral tests and extensive heterogeneity within diagnostic categories. In addition to consensus-based approaches to reframing the current nosology (e.g., RDoC), data-driven approaches to delineating homogenous subgroups, spanning adaptive to maladaptive function can yield clinically meaningful groups with potentially important neurobiological differences. In doing so, it will be important to consider not only symptoms of psychiatric illness (i.e., deficits or problems), but also features of psychiatric health (i.e., strengths or protective factors). Examination of a broad array of phenotypic characteristics, in combination with neurobiological differences, may improve our understanding of the pathophysiology of mental illness and provide new preventative and treatment strategies. Although these analyses will require large samples and advanced analytic approaches, the present study is evidence that such efforts can yield new hypotheses, as well as support existing theories, helping to focus biological psychiatry on those areas that may yield the highest return on investment.

      Acknowledgments and Disclosures

      This study was supported by National Institute of Mental Health Grant Nos. R01MH094639 (to MPM), R01MH081218 (to FXC), R01MH083246 (to FXC), and R21MH084126, with additional funding for personnel and administrative support provided by a grant from the Child Mind Institute (1FDN2012-1 to MPM) and funds from the New York State Office of Mental Health, the Research Foundation for Mental Hygiene, the Brain Research Foundation, and the Stavros Niarchos Foundation. Additional funds were provided by gifts from Phyllis Green, Randolph Cowen, and Joseph P. Healey.
      All data are publicly available via the International Neuroimaging Data-Sharing Initiative. Data can be accessed at http://fcon_1000.projects.nitrc.org. A data usage agreement is required for access to detailed phenotypic information.
      All authors report no biomedical financial interests or potential conflicts of interest.

      Appendix A. Supplementary material

      References

        • Wakefield J.C.
        Misdiagnosing normality: Psychiatry׳s failure to address the problem of false positive diagnoses of mental disorder in a changing professional environment.
        J Ment Health. 2010; 19: 337-351
        • Cuthbert B.
        • Insel T.
        Toward the future of psychiatric diagnosis: The seven pillars of RDoC.
        BMC Med. 2013; 11: 126
        • American Psychiatric Association
        Diagnostic and Statistical Manual of Mental Disorders. 5th ed. American Psychiatric Press, Washington, DC2013
        • Hyman S.E.
        The diagnosis of mental disorders: The problem of reification.
        Annu Rev Clin Psychol. 2010; 6: 155-179
        • Insel T.R.
        The NIMH Research Domain Criteria (RDoC) Project: Precision medicine for psychiatry.
        Am J Psychiatry. 2014; 171: 395-397
        • Casey B.J.
        • Craddock N.
        • Cuthbert B.N.
        • Hyman S.E.
        • Lee F.S.
        • Ressler K.J.
        DSM-5 and RDoC: Progress in psychiatry research?.
        Nat Rev Neurosci. 2013; 14: 810-814
        • Kraemer H.C.
        • Noda A.
        • O׳Hara R.
        Categorical versus dimensional approaches to diagnosis: methodological challenges.
        J Psychiatr Res. 2004; 38: 17-25
        • Widiger T.A.
        • Samuel D.B.
        Diagnostic categories or dimensions? A question for the Diagnostic And Statistical Manual Of Mental Disorders--fifth edition.
        J Abnorm Psychol. 2005; 114: 494-504
        • Yee C.M.
        • Javitt D.C.
        • Miller G.A.
        Replacing dsm categorical analyses with dimensional analyses in psychiatry research: The research domain criteria initiative.
        JAMA Psychiatry. 2015; 72: 1159-1160
        • First M.B.
        Clinical utility: A prerequisite for the adoption of a dimensional approach in DSM.
        J Abnorm Psychol. 2005; 114: 560-564
        • Weinberger D.R.
        • Glick I.D.
        • Klein D.F.
        Whither Research Domain Criteria (RDoC)?: The good, the bad, and the ugly.
        JAMA Psychiatry. 2015; 72: 1161-1162
        • Biswal B.B.
        • Mennes M.
        • Zuo X.-N.
        • Gohel S.
        • Kelly C.
        • Smith S.M.
        • et al.
        Toward discovery science of human brain function.
        Proc Natl Acad Sci U S A. 2010; 107: 4734-4739
        • Akil H.
        • Martone M.E.
        • Van Essen D.C.
        Challenges and Opportunities in Mining Neuroscience Data.
        Science. 2011; 331: 708-712
        • Kraemer H.
        REsearch Domain Criteria (RDoC) and the dsm--Two methodological approaches to mental health diagnosis.
        JAMA Psychiatry. 2015; 72: 1163-1164
        • Insel T.R.
        • Cuthbert B.N.
        Brain disorders? Precisely.
        Science. 2015; 348: 499-500
        • McGorry P.
        • Keshavan M.
        • Goldstone S.
        • Amminger P.
        • Allott K.
        • Berk M.
        • et al.
        Biomarkers and clinical staging in psychiatry.
        World Psychiatry. 2014; 13: 211-223
        • Wardenaar K.J.
        • de Jonge P.
        Diagnostic heterogeneity in psychiatry: Toward an empirical solution.
        BMC Med. 2013; 11: 201
        • Fair D.A.
        • Bathula D.
        • Nikolas M.A.
        • Nigg J.T.
        Distinct neuropsychological subgroups in typically developing youth inform heterogeneity in children with ADHD.
        Proc Natl Acad Sci U S A. 2012; 109: 6769-6774
        • Karalunas S.L.
        • Fair D.
        • Musser E.D.
        • Aykes K.
        • Iyer S.P.
        • Nigg J.T.
        Subtyping attention-deficit/hyperactivity disorder using temperament dimensions: Toward biologically based nosologic criteria.
        JAMA Psychiatry. 2014; 71: 1015-1024
        • Nooner K.B.
        • Colcombe S.
        • Tobe R.
        • Mennes M.
        • Benedict M.
        • Moreno A.
        • et al.
        The NKI-Rockland Sample: A Model for Accelerating the Pace of Discovery Science in Psychiatry.
        Front Neurosci. 2012; 6: 152
        • Achenbach T.M.
        • Rescorla L.A.
        Manual for the ASEBA Adult Forms & Profiles.
        Research Center for Children, Youth, & Familes, University of Vermont, Burlington, VT2003
        • First M.B.
        • Spitzer R.L.
        • Gibbon M.
        • Williams J.B.W.
        Structured Clinical Interview for DSM-IV Axis I Disorders.
        Biometrics Research, New York State Psychiatric Institute, New York, NY1996
        • Oldfield R.C.
        The assessment and analysis of handedness: The Edinburgh inventory.
        Neuropsychologia. 1971; 9: 97-113
        • Xu J.
        • Moeller S.
        • Auerbach E.J.
        • Strupp J.
        • Smith S.M.
        • Feinberg D.A.
        • et al.
        Evaluation of slice accelerations using multiband echo planar imaging at 3 T.
        Neuroimage. 2013; 83: 991-1001
        • Potthoff R.F.
        • Tudor G.E.
        • Pieper K.S.
        • Hasselblad V.
        Can one assess whether missing data are missing at random in medical studies?.
        Stat Methods Med Res. 2006; 15: 213-234
        • Gold M.S.
        • Bentler P.M.
        Treatments of missing data: A Monte Carlo comparison of RBHDI, iterative stochastic regression imputation, and expectation-maximization.
        Struct Equat Model. 2000; 7: 319-355
        • Hudziak J.J.
        • Achenbach T.M.
        • Althoff R.R.
        • Pine D.S.
        A dimensional approach to developmental psychopathology.
        Int J Methods Psychiatr Res. 2007; 16: S16-S23
        • Horn J.L.
        A rationale and test for the number of factors in factor analysis.
        Psychometrika. 1965; 30: 179-185
        • Castellan N.J.
        Shuffling arrays: Appearances may be deceiving.
        Behav Res Methods Instrum Comput. 1992; 24: 72-77
        • Zientek L.
        • Thompson B.
        Applying the bootstrap to the multivariate case: Bootstrap component/factor analysis.
        Behav Res Methods. 2007; 39: 318-325
        • Grice J.W.
        Computing and evaluating factor scores.
        Psychol Methods. 2001; 6: 430-450
        • Chipman H.
        • Tibshirani R.
        Hybrid hierarchical clustering with applications to microarray data.
        Biostatistics. 2006; 7: 286-301
        • Gersho A.
        • Gray R.M.
        Vector Quantization and Signal Compression. Kluwer Academic, Dordrecht, the Netherlands1992
        • Caliński T.
        • Harabasz J.
        A dendrite method for cluster analysis. Commun.
        Stat. 1974; 3: 1-27
        • Gorgolewski K.
        • Burns C.D.
        • Madison C.
        • Clark D.
        • Halchenko Y.O.
        • Waskom M.L.
        • et al.
        Nipype: A flexible, lightweight and extensible neuroimaging data processing framework.
        Front Neuroinform. 2011; 5: 13
        • Greve D.N.
        • Fischl B.
        Accurate and robust brain image alignment using boundary-based registration.
        Neuroimage. 2009; 48: 63-72
        • Jenkinson M.
        • Bannister P.
        • Brady M.
        • Smith S.
        Improved optimization for the robust and accurate linear registration and motion correction of brain images.
        Neuroimage. 2002; 17: 825-841
        • Avants B.B.
        • Epstein C.L.
        • Grossman M.
        • Gee J.C.
        Symmetric diffeomorphic image registration with cross-correlation: Evaluating automated labeling of elderly and neurodegenerative brain.
        Med Image Anal. 2008; 12: 26-41
        • Behzadi Y.
        • Restom K.
        • Liau J.
        • Liu T.T.
        A Component Based Noise Correction Method (CompCor) for BOLD and perfusion based fMRI.
        Neuroimage. 2007; 37: 90-101
        • Shehzad A.
        • Kelly C.
        • Reiss P.T.
        • Craddock R.C.
        • Emerson J.W.
        • McMahon K.
        • et al.
        A multivariate distance-based analytic framework for connectome-wide association studies.
        Neuroimage. 2014; 93: 74-94
        • Reiss P.T.
        • Stevens M.H.H.
        • Shehzad Z.
        • Petkova E.
        • Milham M.P.
        On distance-based permutation tests for between-group comparisons.
        Biometrics. 2010; 66: 636-643
        • Woo C.-W.
        • Krishnan A.
        • Wager T.D.
        Cluster-extent based thresholding in fMRI analyses: Pitfalls and recommendations.
        Neuroimage. 2014; 91: 412-419
        • Eklund A.
        • Nichols T.
        • Knutsson H.
        Can parametric statistical methods be trusted for fMRI based group studies?.
        Proc Natl Acad Sci U S A. 2016; 113: 7900-7905
        • Blanke O.
        Multisensory brain mechanisms of bodily self-consciousness.
        Nat Rev Neurosci. 2012; 13: 556-571
        • Craig A.D.
        How do you feel? Interoception: The sense of the physiological condition of the body.
        Nat Rev Neurosci. 2002; 3: 655-666
        • Tomasi D.
        • Shokri-Kojori E.
        • Volkow N.D.
        High-resolution functional connectivity density: Hub locations, sensitivity, specificity, reproducibility, and reliability.
        Cereb Cortex. 2016; 26: 3249-3259
        • Smith S.M.
        • Fox P.T.
        • Miller K.L.
        • Glahn D.C.
        • Fox P.M.
        • Mackay C.E.
        • et al.
        Correspondence of the brain׳s functional architecture during activation and rest.
        Proc Natl Acad Sci U S A. 2009; 106: 13040-13045
        • Gabbay V.
        • Ely B.A.
        • Li Q.
        • Bangaru S.D.
        • Panzer A.M.
        • Alonso C.M.
        • et al.
        Striatum-based circuitry of adolescent depression and anhedonia.
        J Am Acad Child Adolesc Psychiatry. 2013; 52: e613
        • Cui H.
        • Zhang J.
        • Liu Y.
        • Li Q.
        • Li H.
        • Zhang L.
        • et al.
        Differential alterations of resting-state functional connectivity in generalized anxiety disorder and panic disorder.
        Hum Brain Mapp. 2016; 37: 1459-1473
        • Buckholtz Joshua W.
        • Meyer-Lindenberg A.
        Psychopathology and the human connectome: Toward a transdiagnostic model of risk for mental illness.
        Neuron. 2012; 74: 990-1004
        • Dyck M.
        • Loughead J.
        • Kellermann T.
        • Boers F.
        • Gur R.C.
        • Mathiak K.
        Cognitive versus automatic mechanisms of mood induction differentially activate left and right amygdala.
        Neuroimage. 2011; 54: 2503-2513
        • Fairhall S.L.
        • Sharma S.
        • Magnusson J.
        • Murphy B.
        Memory related dysregulation of hippocampal function in major depressive disorder.
        Biol Psychol. 2010; 85: 499-503
        • Andrews-Hanna J.R.
        • Reidler J.S.
        • Sepulcre J.
        • Poulin R.
        • Buckner R.L.
        Functional-anatomic fractionation of the brain׳s default network.
        Neuron. 2010; 65: 550-562
        • Bogg T.
        • Roberts B.W.
        Conscientiousness and health-related behaviors: A meta-analysis of the leading behavioral contributors to mortality.
        Psychol Bull. 2004; 130: 887-919
        • Belcher A.M.
        • Volkow N.D.
        • Moeller F.G.
        • Ferré S.
        Personality traits and vulnerability or resilience to substance use disorders.
        Trends Cogn Sci. 2014; 18: 211-217
        • Barrett L.F.
        • Simmons W.K.
        Interoceptive predictions in the brain.
        Nat Rev Neurosci. 2015; 16: 419-429
        • Seth A.K.
        Interoceptive inference, emotion, and the embodied self.
        Trends Cogn Sci. 2013; 17: 565-573
        • Kapur S.
        • Phillips A.G.
        • Insel T.R.
        Why has it taken so long for biological psychiatry to develop clinical tests and what to do about it?.
        Mol Psychiatry. 2012; 17: 1174-1179
        • Clementz B.A.
        • Sweeney J.A.
        • Hamm J.P.
        • Ivleva E.I.
        • Ethridge L.E.
        • Pearlson G.D.
        • et al.
        Identification of distinct psychosis biotypes using brain-based biomarkers.
        Am J Psychiatry. 2016; 173: 373-384
        • Borsboom D.
        • Cramer A.O.J.
        Network analysis: An integrative approach to the structure of psychopathology.
        Annu Rev Clin Psychol. 2013; 9: 91-121
        • Yang Z.
        • Xu Y.
        • Xu T.
        • Hoy C.W.
        • Handwerker D.A.
        • Chen G.
        • et al.
        Brain network informed subject community detection in early-onset schizophrenia.
        Sci Rep. 2014; 4: 5549
        • Chen G.
        • Sullivan P.F.
        • Kosorok M.R.
        Biclustering with heterogeneous variance.
        Proc Natl Acad Sci U S A. 2013; 110: 12253-12258

      Linked Article

      • Using Hybrid Modeling to Determine the Latent Structure of Psychopathology
        Biological PsychiatryVol. 81Issue 6
        • Preview
          Van Dam et al. (1) took an innovative, data-driven (as opposed to top-down, diagnostically driven) approach to elucidate psychiatric phenotypes and related differences in functional brain connectivity in a large sample of adults. This work attempted to clarify one of the major problems in both clinical practice and research—namely, high comorbidity of DSM diagnoses. The authors’ intention was to move away from DSM-5 labels, and their results using a hierarchical clustering approach showed distinctions between individuals with internalizing and externalizing symptoms, similar to the approach advocated by the Hierarchical Taxonomy of Psychopathology consortium (R.
        • Full-Text
        • PDF