Advertisement

Pharmacogenomic Study of Clozapine-Induced Agranulocytosis/Granulocytopenia in a Japanese Population

Open AccessPublished:February 11, 2016DOI:https://doi.org/10.1016/j.biopsych.2015.12.006

      Abstract

      Background

      Clozapine-induced agranulocytosis (CIA)/clozapine-induced granulocytopenia (CIG) (CIAG) is a life-threatening event for schizophrenic subjects treated with clozapine.

      Methods

      To examine the genetic factor for CIAG, a genome-wide pharmacogenomic analysis was conducted using 50 subjects with CIAG and 2905 control subjects.

      Results

      We identified a significant association in the human leukocyte antigen (HLA) region (rs1800625, p = 3.46 × 10−9, odds ratio [OR] = 3.8); therefore, subsequent HLA typing was performed. We detected a significant association of HLA-B*59:01 with CIAG (p = 3.81 × 10−8, OR = 10.7) and confirmed this association by comparing with an independent clozapine-tolerant control group (n = 380, p = 2.97 × 10−5, OR = 6.3). As we observed that the OR of CIA (OR: 9.3~15.8) was approximately double that in CIG (OR: 4.4~7.4), we hypothesized that the CIG subjects were a mixed population of those who potentially would develop CIA and those who would not develop CIA (non-CIA). This hypothesis allowed the proportion of the CIG who were non-CIA to be calculated, enabling us to estimate the positive predictive value of the nonrisk allele on non-CIA in CIG subjects. Assuming this model, we estimated that 1) ~50% of CIG subjects would be non-CIA; and 2) ~60% of the CIG subjects without the risk allele would be non-CIA and therefore not expected to develop CIA.

      Conclusions

      Our results suggest that HLA-B*59:01 is a risk factor for CIAG in the Japanese population. Furthermore, if our model is true, the results suggest that rechallenging certain CIG subjects with clozapine may not be always contraindicated.

      Keywords

      Schizophrenia is a chronic, serious, and disabling mental disorder with a lifetime prevalence of approximately 1% of the world population (
      • Picchioni M.M.
      • Murray R.M.
      Schizophrenia.
      ). Antipsychotics are the most useful therapeutic option for schizophrenia; however, approximately one third of patients do not respond adequately to first-line antipsychotics, resulting in treatment-resistant schizophrenia (TRS) (
      • Dold M.
      • Leucht S.
      Pharmacotherapy of treatment-resistant schizophrenia: A clinical perspective.
      ,
      • Hasan A.
      • Falkai P.
      • Wobrock T.
      • Lieberman J.
      • Glenthoj B.
      • Gattaz W.F.
      • et al.
      World Federation of Societies of Biological Psychiatry (WFSBP) Guidelines for Biological Treatment of Schizophrenia, part 1: Update 2012 on the acute treatment of schizophrenia and the management of treatment resistance.
      ).
      Clozapine (CLZ) is a gold standard drug for managing TRS, with a lower incidence of movement abnormality and efficacy superior to that of other antipsychotics in the management of TRS (
      • Dold M.
      • Leucht S.
      Pharmacotherapy of treatment-resistant schizophrenia: A clinical perspective.
      ,
      • Hasan A.
      • Falkai P.
      • Wobrock T.
      • Lieberman J.
      • Glenthoj B.
      • Gattaz W.F.
      • et al.
      World Federation of Societies of Biological Psychiatry (WFSBP) Guidelines for Biological Treatment of Schizophrenia, part 1: Update 2012 on the acute treatment of schizophrenia and the management of treatment resistance.
      ,
      • Essali A.
      • Al-Haj Haasan N.
      • Li C.
      • Rathbone J.
      Clozapine versus typical neuroleptic medication for schizophrenia.
      ,
      • Lewis S.W.
      • Barnes T.R.
      • Davies L.
      • Murray R.M.
      • Dunn G.
      • Hayhurst K.P.
      • et al.
      Randomized controlled trial of effect of prescription of clozapine versus other second-generation antipsychotic drugs in resistant schizophrenia.
      ). Despite its efficacy with TRS, the use of CLZ is significantly restricted by severe side effects such as CLZ-induced agranulocytosis (CIA)/CLZ-induced granulocytopenia (CIG) (CIAG), which is rare but potentially life threatening. CIAG events are observed in approximately 1% (for CIA) and 3% (for CIG) of CLZ-treatment patients, although the prevalence of CIA differs between populations (Caucasian: .3%~.9%; Asian: greater than 1%; Japanese: 1.1%) (
      • Cohen D.
      • Bogers J.P.
      • van Dijk D.
      • Bakker B.
      • Schulte P.F.
      Beyond white blood cell monitoring: Screening in the initial phase of clozapine therapy.
      ,
      • Copolov D.L.
      • Bell W.R.
      • Benson W.J.
      • Keks N.A.
      • Strazzeri D.C.
      • Johnson G.F.
      Clozapine treatment in Australia: A review of haematological monitoring.
      ,
      • Munro J.
      • O’Sullivan D.
      • Andrews C.
      • Arana A.
      • Mortimer A.
      • Kerwin R.
      Active monitoring of 12,760 clozapine recipients in the UK and Ireland. Beyond pharmacovigilance.
      ,
      • Alvir J.M.
      • Lieberman J.A.
      • Safferman A.Z.
      • Schwimmer J.L.
      • Schaaf J.A.
      Clozapine-induced agranulocytosis. Incidence and risk factors in the United States.
      ,
      • Atkin K.
      • Kendall F.
      • Gould D.
      • Freeman H.
      • Liberman J.
      • O’Sullivan D.
      Neutropenia and agranulocytosis in patients receiving clozapine in the UK and Ireland.
      ,
      • Inada K.
      • Ishigooka J.
      [Clozapine].
      ).
      In the clinical setting, patients treated with CLZ require frequent blood monitoring for reduced neutrophil count, an indication of CIG and CIA. One of the main reasons for this is that, in general, CIG and CIA are considered to be a continuous phenotype; psychiatrists expect to control the development of CIA by detecting patients at the less serious CIG stage (
      • Whiskey E.
      • Taylor D.
      Restarting clozapine after neutropenia: Evaluating the possibilities and practicalities.
      ). However, as psychiatrists recognize based on experience, and as Hummer et al. (
      • Hummer M.
      • Kurz M.
      • Barnas C.
      • Saria A.
      • Fleischhacker W.W.
      Clozapine-induced transient white blood count disorders.
      ) reported, some patients with CIG may not go on to experience a serious outcome such as immediate development of CIA (i.e., benign or transient neutropenia) (
      • Whiskey E.
      • Taylor D.
      Restarting clozapine after neutropenia: Evaluating the possibilities and practicalities.
      ,
      • Hummer M.
      • Kurz M.
      • Barnas C.
      • Saria A.
      • Fleischhacker W.W.
      Clozapine-induced transient white blood count disorders.
      ). For example, a routine blood examination showing an absolute neutrophil count (ANC) of 1400 cells/mm3 may result in discontinuation of CLZ (as this is below the Japanese cutoff level of 1500 cells/mm3 at which discontinuing CLZ treatment is mandatory), but the ANC may then recover quickly to an acceptable value. This may be because an ANC of around 1500 cells/mm3 can fluctuate; for example, it may increase with exercise (
      • Phillips D.
      • Rezvani K.
      • Bain B.J.
      Exercise induced mobilisation of the marginated granulocyte pool in the investigation of ethnic neutropenia.
      ,
      • Nooijen P.M.
      • Carvalho F.
      • Flanagan R.J.
      Haematological toxicity of clozapine and some other drugs used in psychiatry.
      ) or decrease with viral infection (
      • Dunk L.R.
      • Annan L.J.
      • Andrews C.D.
      Rechallenge with clozapine following leucopenia or neutropenia during previous therapy.
      ).
      In such clinical situations, a perfect genetic test to screen for the risk of CIAG before treatment by CLZ is not always mandatory. As described, frequent blood monitoring can be used to detect CIA before CIG develops. Also, CLZ is one of the last treatment options for TRS; the psychiatrist may have little choice other than CLZ treatment. In this regard, a screening test may be useful for reducing the frequency of blood monitoring, thus relieving the burden on quality of life and reducing cost. More importantly, however, psychiatrists need more information on whether or not rechallenging is possible or need a screening test to select CIAG patients who can be treated again with CLZ as a rechallenge.
      To date, several pharmacogenetic/pharmacogenomic (PGx) studies have been carried out, mainly in Caucasian populations; however, there is no report to clarify the difference between CIA and CIG. For example, classical candidate gene-based pharmacogenetic studies suggested that a specific sequence variant in human leukocyte antigen (HLA) (HLA-DQB1, 6672G > C) was associated with CIA, but the significance was modest (p ~ .001) (
      • Zhang J.P.
      • Malhotra A.K.
      Pharmacogenetics of antipsychotics: Recent progress and methodological issues.
      ,
      • Athanasiou M.C.
      • Dettling M.
      • Cascorbi I.
      • Mosyagin I.
      • Salisbury B.A.
      • Pierz K.A.
      • et al.
      Candidate gene analysis identifies a polymorphism in HLA-DQB1 associated with clozapine-induced agranulocytosis.
      ). A recent PGx study using a genome-wide association approach and Exome sequencing analysis reported a large impact on CIAG genetics: it showed that CIAG was associated with two independent amino acid changes in HLA-B (158T: p = 6.4 × 10−10) and HLA-DQB1 (126Q: p = 4.7 × 10−14) (
      • Goldstein J.I.
      • Jarskog L.F.
      • Hilliard C.
      • Alfirevic A.
      • Duncan L.
      • Fourches D.
      • et al.
      Clozapine-induced agranulocytosis is associated with rare HLA-DQB1 and HLA-B alleles.
      ). It is of note that the p values surpassed the genome-wide significance threshold (5 × 10−8), although these associations were detected by imputation of nonsynonymous single nucleotide polymorphisms (SNPs) of HLA genes and not by the genotyping of classical HLA alleles.
      In this study, we aimed to explore the genetic risk for CIAG in the Japanese population by a genome-wide SNP survey. Although the sample size of CIAG subjects was small, we detected significant SNPs in the HLA region compared with healthy control subjects drawn from the general population. To detect the responsible risk allele, we conducted classical HLA typing for the CIAG subjects and for CLZ-tolerant control subjects, which showed that a specific allele in HLA-B was significantly associated with CIAG. We investigated the clinical utility of this risk allele and whether it could increase posterior probability in the screening test for CIAG or could provide information regarding the possibility of CLZ rechallenging for CIG subjects.

      Methods and Materials

      Ethical Statement

      After providing a complete description of the study to the subjects, written informed consent was obtained. The ethics committees of each university, institute, and hospital participating in this project approved this study.

      Participants

      CIAG Subjects

      Fifty-two patients with CIAG were included (28 male and 24 female patients, age 44.0 ± 15.6 years old). Of these, 23 were diagnosed with CIA, defined as a decrease in ANC to less than 500 cells/mm3, and 29 were diagnosed with CIG, defined as an ANC between 500 cells/mm3 and 1500 cells/mm3 or for three of these subjects as leukopenia (white blood cell count less than 3000 cells/mm3). These criteria were followed by the Clozaril Patient Monitoring Service in Japan and many other countries, which can reflect a real-world clinical setting. All subjects experienced CIAG within 180 days of first being prescribed CLZ, in accordance with the definition of CIAG in a previous paper (
      • Pirmohamed M.
      • Park K.
      Mechanism of clozapine-induced agranulocytosis: Current status of research and implications for drug development.
      ). All of the subjects were diagnosed as TRS, and all identified themselves as Japanese.

      Healthy Comparison Subjects

      A total of 2948 subjects (1120 male and 1828 female subjects; age 37.0 ± 15.2 years old) were genotyped as healthy control subjects; they had no personal history of mental disorders and were Japanese descent by self-report. Among them, 1108 subjects were nurses at Fujita Health University Hospital (
      • Ikeda M.
      • Shimasaki A.
      • Takahashi A.
      • Kondo K.
      • Saito T.
      • Kawase K.
      • et al.
      Genome-wide environment interaction between depressive state and stressful life events.
      ), while the others were recruited from the general population.

      CLZ-Tolerant Control Subjects

      For the classical HLA association analysis, we recruited 380 CLZ-tolerant individuals (210 male and 170 female individuals, age 42.7 ± 11.8 years old) who had been treated with CLZ for more than 180 days without suffering from CIAG. Again, all of the subjects were diagnosed as having TRS and self-reported as Japanese.

      Genotyping and Quality Control

      In the PGx analysis, we genotyped the CIAG cases and the healthy comparison subjects using the Illumina HumanOmniExpressExome v1.0 (for 825 healthy comparison subjects) or v1.2 (for the rest of the subjects) (Illumina, San Diego, California).
      The following quality control (QC) procedure was applied: 1) we extracted overlapping SNPs between the v1.0 and v1.2 chips; 2) we ensured gender consistency by investigating the SNPs on chromosome X (one CIAG subject); 3) we removed the subjects with a low call rate < .99 (null subjects); and 4) we removed subjects with two degrees or less of relatedness using an identity-by-state analysis (22 healthy comparison subjects). After this first QC filtering, 51 CIAG and 2926 healthy comparison subjects were still eligible, with 643,234 SNPs with a minor allele frequency of more than 1%. To investigate the population structures as following QC, we carried out principal component analysis (
      • Price A.L.
      • Patterson N.J.
      • Plenge R.M.
      • Weinblatt M.E.
      • Shadick N.A.
      • Reich D.
      Principal components analysis corrects for stratification in genome-wide association studies.
      ). Using HapMap datasets for four populations (Japanese in Tokyo; Han Chinese in Beijing; Yoruba in Ibadan, Nigeria; and Utah residents with ancestry from northern and western Europe), we confirmed that samples in the East Asian population were grouped in one cluster (Supplemental Figure S1). Then, using only the East Asian HapMap populations (Japanese in Tokyo and Han Chinese in Beijing), we classified the population clusters, including Chinese, Japanese-Mainland, and Japanese-Ryukyu clusters (Supplemental Figure S1) (
      • Yamaguchi-Kabata Y.
      • Nakazono K.
      • Takahashi A.
      • Saito S.
      • Hosono N.
      • Kubo M.
      • et al.
      Japanese population structure, based on SNP genotypes from 7003 individuals compared to other ethnic groups: Effects on population-based association studies.
      ). We confirmed that the 11 CIAG subjects recruited from Ryukyu Hospital were classified in the Japanese-Ryukyu clusters. The subjects in these two Japanese clusters were selected by visual inspection, with 22 subjects (one CIAG and 21 healthy comparison subjects) being excluded as outliers. To calculate more precise eigenvectors for association analysis, we carried out further principal component analysis for the selected CIAG and healthy comparison subjects. The SNPs with a call rate <.99 and with large deviations from the Hardy Weinberg equilibrium (p value of the exact test <1 × 10−5) in the control group were also removed. Finally, we filtered SNPs by minor allele frequency of more than 1% in control subjects, leaving 522,694 SNPs, 50 CIAG subjects (22 CIA and 28 CIG), and 2905 healthy comparison subjects.
      In the HLA analysis, genotyping of the HLA alleles of HLA-A, HLA-C, HLA-B, and HLA-DRB1 genes was performed using WAKFlow HLA typing kit (Wakunaga, Hiroshima, Japan) and a Luminex Multi-Analyte Profiling system (xMAP; Luminex, Austin, Texas).

      Statistical Analysis in the PGx

      The association of the direct genotyped SNPs was assessed by a logistic regression analysis with adjustment of sex and the first two eigenvectors as covariates using PLINK ver1.9 (https://www.cog-genomics.org/plink2/) (
      • Chang C.C.
      • Chow C.C.
      • Tellier L.C.
      • Vattikuti S.
      • Purcell S.M.
      • Lee J.J.
      Second-generation PLINK: Rising to the challenge of larger and richer datasets.
      ). Regional plots were generated using LocusZoom (
      • Pruim R.J.
      • Welch R.P.
      • Sanna S.
      • Teslovich T.M.
      • Chines P.S.
      • Gliedt T.P.
      • et al.
      LocusZoom: Regional visualization of genome-wide association scan results.
      ). The association of HLA alleles (dominant model) between CIAG and healthy comparison/CLZ-tolerant control subjects was evaluated using a 2 × 2 Fisher’s exact test in R (www.r-project.org). The statistical level of significance was set at p < 5 × 10−8 for the genome-wide SNP analysis and the HLA analysis.

      Clinical Performance of HLA-B*59:01 as a Genetic Test

      To assess the clinical utility of the detected risk in HLA-B (HLA-B*59:01), we calculated sensitivity and specificity, a standard statistical measure of the performance of the test. We also estimated positive predictive value (PPV) and negative predictive value (NPV) assuming 1%, 3%, and 4% prevalence for CIA, CIG, and CIAG, respectively, as prior probabilities.
      Based on the PGx results, described later in Results, we hypothesized that CIG subjects were a mixed population of those with the potential to go on to develop CIA (CIAestimated) and those whose neutrophil count had simply decreased below the recommended cutoff line and who would not develop CIA (non-CIAestimated) (
      • Whiskey E.
      • Taylor D.
      Restarting clozapine after neutropenia: Evaluating the possibilities and practicalities.
      ,
      • Hummer M.
      • Kurz M.
      • Barnas C.
      • Saria A.
      • Fleischhacker W.W.
      Clozapine-induced transient white blood count disorders.
      ). We therefore focused on the CIG subjects to estimate the proportion who were non-CIAestimated.
      Assume that ϕ is the proportion of the CIG who are non-CIAestimated (i.e., ϕ = non-CIAestimated/CIG) and that P(CIGobserved), P(CIAobserved), and P(Controlobserved) are the proportions of patients observed to have the risk allele in the CIG, CIA, and control (i.e., healthy comparison subjects and CLZ-tolerant control subjects) groups, respectively. In line with this assumption, we hypothesized that the proportion between the probability of CIAestimated with the risk allele in the CIG subjects and that without the risk allele in the CIG subjects corresponds with the proportion between the CIA subjects observed to have the risk allele and those without the risk allele. This would also be the case with the non-CIAestimated subjects, which is compatible with the healthy comparison subjects and CLZ-tolerant control subjects (Supplemental Figure S2). To calculate ϕ, we used the following formulas:
      P(CIGobserved)=ϕ×P(non-CIAestimated)+(1ϕ)×P(CIAestimated)ϕ×P(Controlobserved)+(1ϕ)×P(CIAobserved)


      After solving for ϕ, we could estimate the number of subjects with the risk allele (under the dominant model) for non-CIAestimated [=number of CIGobserved × ϕ × P(non-CIAobserved)] and CIAestimated [=number of CIGobserved × (1−ϕ) × P(CIAobserved)] in the CIG group. Then, we examined the statistical measure of the performance of the test to assess the utility of the nonrisk allele on the non-CIAestimated, who are the safest subjects for rechallenging: sensitivity, specificity, PPV, and NPV. It is of note that sensitivity and specificity are assumed in the theoretical model (we could not divide subjects into non-CIA and CIA in the CIG group in the real world), but PPV (the proportion of non-CIA in the CIG without the risk allele) is the most informative value because we can speculate the validity of CLZ rechallenging to the CIG.

      Results

      Genome-wide PGx Analysis

      A quantile-quantile plot was generated using the logistic regression test (Supplemental Figure S3) and the genomic inflation factor (λGC) based on median chi-square statistics was 1, which indicates that population stratification was controlled by the eigenvectors.
      Figure 1 shows a Manhattan plot of the data. The top SNPs of the association are listed in Table 1. SNPs in the major histocompatibility complex (MHC) were redundant due to linkage disequilibrium, so we listed only SNPs with genome-wide significance. For regions other than the MHC, SNPs with p < 1 × 10−5 were listed; all of the top SNPs with p < 1 × 10−5 are listed in Supplemental Table S1.
      Figure thumbnail gr1
      Figure 1Manhattan plot for the genome-wide pharmacogenomic results. Red line indicates genome-wide significance (5 × 10−8). The y axis is −log10(p values) of the single nucleotide polymorphisms and the x axis is chromosomal position (hg19).
      Table 1Results for the Association Between CIAG and Control Subjects
      CHRSNPBPA1A2F_AF_UOR (95% CIs)pClosest Gene
      6rs180062532152442GA.30.0843.78 (2.43–5.87)3.46E-09PBX2 (MHC)
      6rs226941832180120GT.30.0943.60 (2.29–5.66)2.69E-08NOTCH4 (MHC)
      6rs321346832183158AG.30.0943.58 (2.28–5.62)2.89E-08NOTCH4 (MHC)
      6rs207128232188943TC.30.0953.56 (2.27–5.60)3.45E-08NOTCH4 (MHC)
      3rs37494487188116AG.37.202.96 (1.90–4.61)1.60E-06GRM7
      12rs64678250482211GA.14.0513.94 (2.15–7.23)9.59E-06SMARCD1
      Due to tight LD in MHC, only SNPs with p less than 5E-8 for MHC region were listed. For other region, SNPs with p less than 1E-5 were listed.A1, nonreference allele; A2, reference allele; BP, base position based on hg19; CHR, chromosome; CI, confidence interval; CIAG, clozapine-induced agranulocytosis/granulocytopenia; F_A, frequency of A1 in cases; F_U, frequency of A1 in healthy comparison subjects; LD, linkage disequilibrium; MHC, major histocompatibility complex; OR, odds ratio for A1; SNP, single nucleotide polymorphism.
      The SNPs in the MHC region showed genome-wide significance (i.e., p < 5 × 10−8), and the top hit in this region was rs1800625 located around 32 megabase in chromosome 6 (p = 3.5 × 10−9, odds ratio [OR] = 3.8; Supplemental Figure S4). The other regions of top association were seen at rs3749448 in the glutamate receptor, metabotropic 7 (GRM7, p = 1.6 × 10−6, OR = 3.0; Supplemental Figure S5), and rs646782 in SWI/SNF related, matrix associated, action dependent regulator of chromatin, subfamily D, member 1 (SMARCD1) (p = 9.1 × 10−6, OR = 3.9; Supplemental Figure S5).
      To detect the responsible HLA allele, a classical HLA typing (HLA-A, HLA-C, HLA-B, and HLA-DRB1) was conducted to assess the association of HLA alleles between CIAG (n = 50) and a subset of healthy comparison subjects (n = 1891) of which most were from the general population (n = 1739) and the rest were nurses (n = 152). We detected that a specific allele of HLA-B*59:01 showed significant association with CIAG in the Japanese population (dominant model: p = 3.8 × 10−8, OR = 10.7; Table 2; all of the association results are listed in Supplemental Table S2). Interestingly, the narrow phenotype subjects with CIA (n = 22) showed a greater effect size (OR = 15.8), indicating that the core phenotype might extract a purer risk for CIAG despite the lower statistical power due to the small sample size (p = 2.8 × 10−6; Table 2). We also found a trend for significant association of HLA-DRB1*04:05 (p = 5.2 × 10−5, OR = 3.4; Supplemental Table S2); however, this allele is in linkage disequilibrium with HLA-B*59:01 (D′ ~ .95 and r2 ~ .1). The conditional analysis with adjustment of HLA-B*59:01 revealed only a nominal significant association between HLA-DRB1*04:05 and CIAG (p = .034), and so this association was correlated with that of HLA-B*59:01.
      Table 2Association of HLA-B*59:01 With CIAG
      Allele Count for Case
      2 allele, samples with 2 alleles of HLA-B*59:01; 1 allele, samples with 1 allele of HLA-B*59:01; 0 allele, samples with null allele of HLA-B*59:01.
      Allele Count for Control
      2 allele, samples with 2 alleles of HLA-B*59:01; 1 allele, samples with 1 allele of HLA-B*59:01; 0 allele, samples with null allele of HLA-B*59:01.
      Comparison2 Allele + 1 Allele0 Allele2 Allele + 1 Allele0 AllelepOR95% CIs
      CIAG (n = 50) vs. Healthy Comparison Subjects (n = 1891)12385418373.81E-0810.74.8–22.4
      CIA (n = 22) vs. Healthy Comparison Subjects (n = 1891)7152.78E-0615.85.2–43.2
      CIG (n = 28) vs. Healthy Comparison Subjects (n = 1891)5231.32E-037.42.1–20.9
      CIAG (n = 50) vs. Tolerant Control Subjects (n = 380)1238183622.97E-056.32.6–15.1
      CIA (n = 22) vs. Tolerant Control Subjects (n = 380)7151.39E-049.32.8–28.1
      CIG (n = 28) vs. Tolerant Control Subjects (n = 380)523.0154.41.2–13.7
      CI, confidence interval; CIA, clozapine-induced agranulocytosis; CIAG, clozapine-induced agranulocytosis/granulocytopenia; CIG, clozapine-induced granulocytopenia; OR, odds ratio.
      a 2 allele, samples with 2 alleles of HLA-B*59:01; 1 allele, samples with 1 allele of HLA-B*59:01; 0 allele, samples with null allele of HLA-B*59:01.
      To validate the significant association of HLA-B*59:01 between the CIAG and healthy comparison subjects and to find novel risk alleles, we genotyped a total of 380 CLZ-tolerant control subjects as an independent comparison dataset (HLA-A, HLA-C, HLA-B, and HLA-DRB1). Again, a significant association was detected between the CIAG group and the CLZ-tolerant control subjects in HLA-B*59:01 (p = 3.0 × 10−5, OR = 6.3; Table 2; all of the association results are listed in Supplemental Table S3). The narrow phenotype, CIA, again showed a greater effect size (OR = 9.3) than that for CIG (OR = 4.4).
      To assess the clinical utility for the risk allele of HLA-B*59:01, standard statistical measures of test performance were calculated (Table 3). We obtained low sensitivity and PPV but high specificity and NPV; however, the acceptable line for clinical use was not reached.
      Table 3Diagnostic Performance of HLA-B*59:01 on CIAG
      ComparisonSensitivitySpecificityPPV
      Assuming 1%, 3%, and 4% prevalence for CIA, CIG, and CIAG, respectively.
      NPV
      Assuming 1%, 3%, and 4% prevalence for CIA, CIG, and CIAG, respectively.
      PAR
      Assuming 1%, 3%, and 4% prevalence for CIA, CIG, and CIAG, respectively.
      PARP
      Assuming 1%, 3%, and 4% prevalence for CIA, CIG, and CIAG, respectively.
      (%)
      CIAG (n = 50) vs. Healthy Comparison Subjects (n = 1891).240.971.259.968.0084321.1
      CIA (n = 22) vs. Healthy Comparison Subjects (n = 1891).318.971.101.993.0029629.6
      CIG (n = 28) vs. Healthy Comparison Subjects (n = 1891).179.971.162.975.0045115.0
      CIAG (n = 50) vs. Tolerant Control Subjects (n = 380).240.953.174.968.0078319.6
      CIA (n = 22) vs. Tolerant Control Subjects (n = 380).318.953.064.993.0028228.2
      CIG (n = 28) vs. Tolerant Control Subjects (n = 380).179.953.104.974.0040213.4
      CIA, clozapine-induced agranulocytosis; CIAG, clozapine-induced agranulocytosis/granulocytopenia; CIG, clozapine-induced granulocytopenia; NPV, negative predictive value; PAR, population attributable risk; PARP, population attributable risk percent; PPV, positive predictive value.
      a Assuming 1%, 3%, and 4% prevalence for CIA, CIG, and CIAG, respectively.

      Simulated Performance of HLA-B*59:01 for CLZ Rechallenging

      The PGx results showed a large difference in effect size between CIA and CIG (CIA > CIG). We therefore assumed a simple model where CIG consists of a mixture of subjects on the way to developing CIA (CIAestimated) and subjects whose ANC had simply decreased to less than 1500 cell/mm3 by chance or due to an unknown risk factor and who would not develop CIA (non-CIAestimated) (
      • Whiskey E.
      • Taylor D.
      Restarting clozapine after neutropenia: Evaluating the possibilities and practicalities.
      ,
      • Hummer M.
      • Kurz M.
      • Barnas C.
      • Saria A.
      • Fleischhacker W.W.
      Clozapine-induced transient white blood count disorders.
      ). Using this model, we examined as an explorative analysis its clinical utility, i.e., whether this information can be a reasonable predictor for identifying among CIG subjects those that are non-CIAestimated and those that are CIAestimated. The values for ϕ (non-CIAestimated/CIG) were around ~50% (48.2% based on the calculation for healthy comparison subjects and 51.6% for the CLZ-tolerant control subjects), indicating that half of the subjects in the CIG group had minimal risk of developing CIA. The estimated allelic distributions of HLA-B*59:01 are listed in Table 4, calculated based on P(CIAobserved) and P(non-CIAobserved). In this 2 × 2 table, it is of note that 1) 82% of the CIG subjects did not have the risk allele; and 2) the PPV for detecting non-CIAestimated subjects in the CIG group without the risk allele was moderate (~60%).
      Table 4Diagnostic Performance of Nonrisk Allele (Alleles Except for HLA-B*59:01) on non-CIA Among CIG Subjects
      Calculated Based onSamplesnon-CIAestimated
      Proportion of the non-CIAestimated and CIAestimated samples were calculated by ϕ.
      (%)
      CIAestimated
      Proportion of the non-CIAestimated and CIAestimated samples were calculated by ϕ.
      (%)
      Total (%)SensitivitySpecificityPPV
      Assuming 48.2% and 51.6% incidence based on healthy comparison and tolerant controls subjects, respectively, as prior probability.
      NPV
      Assuming 48.2% and 51.6% incidence based on healthy comparison and tolerant controls subjects, respectively, as prior probability.
      CIA vs. Healthy Comparison SubjectsRisk allele (−)46.835.382.1.971.318.570.923
      (ϕ = 48.2%)Risk allele (+)1.416.517.9
      Total48.251.8100
      CIA vs. Tolerant Control SubjectsRisk allele (−)49.133.082.1.953.318.598.863
      (ϕ = 51.6%)Risk allele (+)2.415.417.8
      Total51.648.4100
      CIA, clozapine-induced agranulocytosis; CIG, clozapine-induced granulocytopenia; NPV, negative predictive value; PPV, positive predictive value.
      a Proportion of the non-CIAestimated and CIAestimated samples were calculated by ϕ.
      b Assuming 48.2% and 51.6% incidence based on healthy comparison and tolerant controls subjects, respectively, as prior probability.

      Discussion

      Through the genome-wide SNP and classical HLA analyses, we identified a risk allele for CIAG at HLA-B in people of Japanese ancestry. It is of note that HLA-B was reported as a risk factor for CIAG in Caucasian subjects in a recent PGx study (
      • Goldstein J.I.
      • Jarskog L.F.
      • Hilliard C.
      • Alfirevic A.
      • Duncan L.
      • Fourches D.
      • et al.
      Clozapine-induced agranulocytosis is associated with rare HLA-DQB1 and HLA-B alleles.
      ); our results therefore support this association. However, SNP 158T in HLA-B, the risk allele detected by Goldstein et al. (
      • Goldstein J.I.
      • Jarskog L.F.
      • Hilliard C.
      • Alfirevic A.
      • Duncan L.
      • Fourches D.
      • et al.
      Clozapine-induced agranulocytosis is associated with rare HLA-DQB1 and HLA-B alleles.
      ), is not compatible with HLA-B*59:01 detected in this study.
      HLA-B*59:01, which has not been reported as a risk for side effects induced by CLZ or other drugs, is not common in the Japanese population (the allele frequency is ~1.5% in our control subjects and 2.0% in the Japanese population based on the HLA laboratory dataset URL: http://www.hla.or.jp/haplo/haplonavi.php?type=haplo&lang=en). Interestingly, based on the publicly available database (http://www.allelefrequencies.net/), the allele is virtually nonexistent in other populations such as Caucasian and African, the exception being East Asian populations (e.g., in the Chinese and Korean populations the allele frequency is .1 ~ 2%). This worldwide allelic distribution might be the cause of the different prevalences of CIAG (for example, the East Asian population demonstrates a higher prevalence than the Caucasian), if this allele poses a genuine risk not only for those of Japanese ancestry but also for other populations.
      Although risk allele HLA-B*59:01 showed a significant association with CIAG, its clinical utility as a screening test is limited with regard to predicting future CIAG in patients being treated by CLZ for the first time. Sensitivity and PPV were low, indicating this allele is not suitable for a screening test to target CIAG, especially for CIG. Specificity and NPV were high, indicating that the test in general could be useful for making a definitive diagnosis. However, the information provided by this risk allele is not necessary for the diagnosis of CIAG, as definitive diagnosis is made by blood count. Taken together, the predictive power estimated by our results indicates this would not be useful for the clinical diagnosis of CIAG (and/or CIA) when first treating with CLZ.
      However, our results based on explorative estimates may provide evidence that CLZ rechallenging is a possible option for patients when they respond only to CLZ. Using the estimated proportion of non-CIAestimated/CIAestimated, we observed the high sensitivity (to assess the non-CIAestimated patients with the nonrisk allele) that can provide a key to personalized medicine for CIG patients. However, in the real world, we do not, of course, have a predictor to divide CIG patients into non-CIA and CIA but only have allele information of the risk with HLA-B*59:01. Therefore, the PPV is the most informative value for CIG patients: if a patient has no risk allele, indicating a lower risk for developing subsequent CIA (as with approximately 80% of the CIG subjects [n = 23/28]), they would be a potent candidate for rechallenging with CLZ treatment. Among these, it is of note that 60% of the CIG subjects without HLA-B*59:01 have the potential to avoid future CIA. Conversely, this means that 40% of the CIG subjects without the risk allele may develop CIA. It is a matter of clinical judgment whether or not this has a high value, but given that there is no treatment strategy other than CLZ to control the patient’s symptoms (30% to 40 % of patients with CLZ treatment showed a very good response) (
      • Butcher N.J.
      • Fung W.L.
      • Fitzpatrick L.
      • Guna A.
      • Andrade D.M.
      • Lang A.E.
      • et al.
      Response to clozapine in a clinically identifiable subtype of schizophrenia.
      ,
      • Naber D.
      • Riedel M.
      • Klimke A.
      • Vorbach E.U.
      • Lambert M.
      • Kuhn K.U.
      • et al.
      Randomized double blind comparison of olanzapine vs. clozapine on subjective well-being and clinical outcome in patients with schizophrenia.
      ), and that mortality from CIG is not high under the current monitoring system (2% to 4% even among CIA subjects) (
      • Cohen D.
      • Bogers J.P.
      • van Dijk D.
      • Bakker B.
      • Schulte P.F.
      Beyond white blood cell monitoring: Screening in the initial phase of clozapine therapy.
      ), rechallenging will be one of the options to control the severe symptoms for TRS. We stress, of course, that frequent monitoring of the white blood cells and ANC is essential regardless.
      Although we replicated the association in the HLA-B gene detected in the report by Goldstein et al. (
      • Goldstein J.I.
      • Jarskog L.F.
      • Hilliard C.
      • Alfirevic A.
      • Duncan L.
      • Fourches D.
      • et al.
      Clozapine-induced agranulocytosis is associated with rare HLA-DQB1 and HLA-B alleles.
      ), several inclusion criteria were not identical. For example, Goldstein et al. (
      • Goldstein J.I.
      • Jarskog L.F.
      • Hilliard C.
      • Alfirevic A.
      • Duncan L.
      • Fourches D.
      • et al.
      Clozapine-induced agranulocytosis is associated with rare HLA-DQB1 and HLA-B alleles.
      ) used a more stringent definition of CIG, the upper limit of the ANC being less than 1000 cells/mm3. Whereas the definition of the ANC we used was less than 1500 cells/mm3, which is in accord with the Clozaril Patient Monitoring Service, it thus could reflect the real-world clinical setting. To validate the findings by Goldstein et al. (
      • Goldstein J.I.
      • Jarskog L.F.
      • Hilliard C.
      • Alfirevic A.
      • Duncan L.
      • Fourches D.
      • et al.
      Clozapine-induced agranulocytosis is associated with rare HLA-DQB1 and HLA-B alleles.
      ), we conducted an explorative analysis using the stringent ANC cutoff level (less than 1000 cells/mm3); we could replicate the significant association between HLA-B*59:01 and the CIAG subjects (CIA + stringent CIG) with the ANC less than 1000 cells/mm3 (Supplemental Table S4). Also, the effect size showed similar magnitude between CIA (OR: 15.8/9.3 for comparison with healthy control subjects/tolerant control subjects, respectively) and the stringent CIG (OR: 13.5/7.9 for comparison with healthy control subjects/tolerant control subjects, respectively; further analysis by stratifying the subjects based on ANC ranges is described in the Supplement text and Supplemental Table S4), supporting HLA-B*59:01 as a genuine risk factor for CIA.
      Our study has several limitations. First, the definition of the tolerant control was not perfect: epidemiological surveys suggested that CIAG was observed at 80% in the first 18 weeks and 90% by 1 year (
      • Munro J.
      • O’Sullivan D.
      • Andrews C.
      • Arana A.
      • Mortimer A.
      • Kerwin R.
      Active monitoring of 12,760 clozapine recipients in the UK and Ireland. Beyond pharmacovigilance.
      ,
      • Atkin K.
      • Kendall F.
      • Gould D.
      • Freeman H.
      • Liberman J.
      • O’Sullivan D.
      Neutropenia and agranulocytosis in patients receiving clozapine in the UK and Ireland.
      ). Therefore, we would have to monitor our subjects again to validate them as genuine tolerant subjects; theoretically, however, approximately three subjects out of 380 subjects (380 × .04 × .2 ≈ 3) would be the misclassified subjects under the assumptions of the prevalence of CIAG as 4% and of the 20% (in a worst-case scenario) of the subjects undetected at the earlier stage, expecting low possibility to have a large impact on our results. Lastly, our sample size was limited; therefore, further replication with greater power is essential for clinical application.
      In summary, we identified the HLA risk for CIAG in the Japanese population. The effect size is relatively large, but due to the modest predictive power and population frequency of the allele, it is difficult to use this result as a predictive factor for patients exposed to CLZ for the first time. However, our results indicate that rechallenging is a possible option for CIG patients who have no risk allele when they respond only to CLZ. Future identification of risk factors other than HLA-B*59:01 could provide a more accurate indicator to predict CIAG, including environment factors, such as lower ANC in the stable condition, drug interaction, or gene-environment interplay.

      Acknowledgments and Disclosures

      This study was supported by grants of part of the BioBank Japan Project from the Ministry of Education, Culture, Sports, and Technology (MEXT) of Japan; the Health and Labour Sciences Research Grants for Comprehensive Research on Persons with Disabilities from Japan Agency for Medical Research and Development (AMD); Integrated Research on Neuropsychiatric Disorders carried out under the Strategic Research Program for Brain Sciences from the MEXT of Japan and Japan Agency for Medical Research and Development (AMED); Grant-in-Aid for Scientific Research on Innovative Areas (Comprehensive Brain Science Network, Glia assembly) from the MEXT of Japan; Grant-in-Aid for Scientific Research (B) from the MEXT of Japan; SENSHIN Medical Research Foundation, Japan.
      We thank the following doctors for their participation in this study: Drs. Kentaro Mizuno, M.D., and Toshiaki Shiratsuchi, M.D. (Wakakusa Hospital); Dr. Keishi Ueda, M.D. (Kumamotoseimei Hospital); Drs. Yasuhide Fukuji, M.D., and Minori Nakai, M.D. (National Hospital Organization Ryukyu Hospital); attending psychiatrists of Tosa Hospital; attending psychiatrists of Baba Hospital; attending psychiatrists of Matsuyama Kinen Hospital; attending psychiatrists of Kochi Medical School Hospital; Drs. Yutaka Sawa, M.D., and Haruo Watanabe, M.D. (Sawa Hospital); Drs. Sadakazu Tanaka, M.D., and Yoshiki Kishi, M.D. (Okayama Psychiatric Medical Center); attending psychiatrists of Manyo Clinic; Dr. Toshiya Murai, M.D., Ph.D. (Department of Psychiatry, Graduate School of Medicine, Kyoto University); Drs. Akiko Mamoto, M.D., Ph.D., and Norio Taniguchi, M.D., Ph.D. (Asakayama General Hospital); Dr. Kyuryoku Hirakawa, M.D. (Matsubara Hospital); Dr. Hideaki Tabuse, M.D. (Holy Cross Hospital); Dr. Yuuhei Amano, M.D. (Kakamigahara Hospital); Dr. Katsushi Kon, M.D. (Gakuji-kai Kimura Hospital); Drs. Yasuo Fujii, Ph.D., M.D., and Ryoji Miyata, M.D. (Yamanashi Prefectural Kita Hospital); attending psychiatrists of Okinawa Prefectural Seiwa Hospital; Dr. Nobutoshi Kawai, M.D., Ph.D. (Department of Psychiatry, Mitsukaido Kosei Hospital); Ms. Masami Miyata (Center for Research Promotion and Support, Fujita Health University); Dr. Ichiro Kusumi, M.D., Ph.D. (Department of Psychiatry, Hokkaido University Graduate School of Medicine); Dr. Hitoshi Kiyoi, M.D., Ph.D. (Department of Hematology and Oncology, Nagoya University Graduate School of Medicine); and Dr. Norio Ozaki, M.D., Ph.D. (Department of Psychiatry, Nagoya University Graduate School of Medicine). No compensation was provided.
      Dr. N. Iwata has received research support or speakers’ honoraria from or has served as a consultant to Janssen Pharmaceutical K.K., GlaxoSmithKline, Eli Lilly, Otsuka, Shionogi, Dainippon Sumitomo, Tanabe Mitsubishi, and Daiichi-Sankyo. Dr. Hashimoto has received research support or speakers’ fees from Pfizer Inc; Yoshitomiyakuhin Corporation; Dainippon Sumitomo Pharma Co., Ltd.; Novartis Pharma K.K.; GlaxoSmithKline K.K.; Hisamitsu Pharmaceutical Co., Inc.; Janssen Pharmaceutical K.K.; Nippon Zoki Pharmaceutical Co., Ltd.; Otsuka Pharmaceutical Co., Ltd.; Astellas Pharma Inc.; Eli Lilly Japan K.K.; and Abbot Japan Co., Ltd. Dr. Ikeda has received research support from Janssen Pharmaceutical K.K. The remaining authors report no biomedical financial interests or potential conflicts of interest.

      Appendix A. Supplementary Materials

      References

        • Picchioni M.M.
        • Murray R.M.
        Schizophrenia.
        BMJ. 2007; 335: 91-95
        • Dold M.
        • Leucht S.
        Pharmacotherapy of treatment-resistant schizophrenia: A clinical perspective.
        Evid Based Ment Health. 2014; 17: 33-37
        • Hasan A.
        • Falkai P.
        • Wobrock T.
        • Lieberman J.
        • Glenthoj B.
        • Gattaz W.F.
        • et al.
        World Federation of Societies of Biological Psychiatry (WFSBP) Guidelines for Biological Treatment of Schizophrenia, part 1: Update 2012 on the acute treatment of schizophrenia and the management of treatment resistance.
        World J Biol Psychiatry. 2012; 13: 318-378
        • Essali A.
        • Al-Haj Haasan N.
        • Li C.
        • Rathbone J.
        Clozapine versus typical neuroleptic medication for schizophrenia.
        Cochrane Database Syst Rev. 2009; (CD000059.)
        • Lewis S.W.
        • Barnes T.R.
        • Davies L.
        • Murray R.M.
        • Dunn G.
        • Hayhurst K.P.
        • et al.
        Randomized controlled trial of effect of prescription of clozapine versus other second-generation antipsychotic drugs in resistant schizophrenia.
        Schizophr Bull. 2006; 32: 715-723
        • Cohen D.
        • Bogers J.P.
        • van Dijk D.
        • Bakker B.
        • Schulte P.F.
        Beyond white blood cell monitoring: Screening in the initial phase of clozapine therapy.
        J Clin Psychiatry. 2012; 73: 1307-1312
        • Copolov D.L.
        • Bell W.R.
        • Benson W.J.
        • Keks N.A.
        • Strazzeri D.C.
        • Johnson G.F.
        Clozapine treatment in Australia: A review of haematological monitoring.
        Med J Aust. 1998; 168: 495-497
        • Munro J.
        • O’Sullivan D.
        • Andrews C.
        • Arana A.
        • Mortimer A.
        • Kerwin R.
        Active monitoring of 12,760 clozapine recipients in the UK and Ireland. Beyond pharmacovigilance.
        Br J Psychiatry. 1999; 175: 576-580
        • Alvir J.M.
        • Lieberman J.A.
        • Safferman A.Z.
        • Schwimmer J.L.
        • Schaaf J.A.
        Clozapine-induced agranulocytosis. Incidence and risk factors in the United States.
        N Engl J Med. 1993; 329: 162-167
        • Atkin K.
        • Kendall F.
        • Gould D.
        • Freeman H.
        • Liberman J.
        • O’Sullivan D.
        Neutropenia and agranulocytosis in patients receiving clozapine in the UK and Ireland.
        Br J Psychiatry. 1996; 169: 483-488
        • Inada K.
        • Ishigooka J.
        [Clozapine].
        Nihon Rinsho71. 2013; : 678-683
        • Whiskey E.
        • Taylor D.
        Restarting clozapine after neutropenia: Evaluating the possibilities and practicalities.
        CNS Drugs. 2007; 21: 25-35
        • Hummer M.
        • Kurz M.
        • Barnas C.
        • Saria A.
        • Fleischhacker W.W.
        Clozapine-induced transient white blood count disorders.
        J Clin Psychiatry. 1994; 55: 429-432
        • Phillips D.
        • Rezvani K.
        • Bain B.J.
        Exercise induced mobilisation of the marginated granulocyte pool in the investigation of ethnic neutropenia.
        J Clin Pathol. 2000; 53: 481-483
        • Nooijen P.M.
        • Carvalho F.
        • Flanagan R.J.
        Haematological toxicity of clozapine and some other drugs used in psychiatry.
        Hum Psychopharmacol. 2011; 26: 112-119
        • Dunk L.R.
        • Annan L.J.
        • Andrews C.D.
        Rechallenge with clozapine following leucopenia or neutropenia during previous therapy.
        Br J Psychiatry. 2006; 188: 255-263
        • Zhang J.P.
        • Malhotra A.K.
        Pharmacogenetics of antipsychotics: Recent progress and methodological issues.
        Expert Opin Drug Metab Toxicol. 2013; 9: 183-191
        • Athanasiou M.C.
        • Dettling M.
        • Cascorbi I.
        • Mosyagin I.
        • Salisbury B.A.
        • Pierz K.A.
        • et al.
        Candidate gene analysis identifies a polymorphism in HLA-DQB1 associated with clozapine-induced agranulocytosis.
        J Clin Psychiatry. 2011; 72: 458-463
        • Goldstein J.I.
        • Jarskog L.F.
        • Hilliard C.
        • Alfirevic A.
        • Duncan L.
        • Fourches D.
        • et al.
        Clozapine-induced agranulocytosis is associated with rare HLA-DQB1 and HLA-B alleles.
        Nat Commun. 2014; 5: 4757
        • Pirmohamed M.
        • Park K.
        Mechanism of clozapine-induced agranulocytosis: Current status of research and implications for drug development.
        CNS Drugs. 1997; 7: 139-158
        • Ikeda M.
        • Shimasaki A.
        • Takahashi A.
        • Kondo K.
        • Saito T.
        • Kawase K.
        • et al.
        Genome-wide environment interaction between depressive state and stressful life events.
        J Clin Psychiatry. 2015;
        • Price A.L.
        • Patterson N.J.
        • Plenge R.M.
        • Weinblatt M.E.
        • Shadick N.A.
        • Reich D.
        Principal components analysis corrects for stratification in genome-wide association studies.
        Nat Genet. 2006; 38: 904-909
        • Yamaguchi-Kabata Y.
        • Nakazono K.
        • Takahashi A.
        • Saito S.
        • Hosono N.
        • Kubo M.
        • et al.
        Japanese population structure, based on SNP genotypes from 7003 individuals compared to other ethnic groups: Effects on population-based association studies.
        Am J Hum Genet. 2008; 83: 445-456
        • Chang C.C.
        • Chow C.C.
        • Tellier L.C.
        • Vattikuti S.
        • Purcell S.M.
        • Lee J.J.
        Second-generation PLINK: Rising to the challenge of larger and richer datasets.
        Gigascience. 2015; 4: 7
        • Pruim R.J.
        • Welch R.P.
        • Sanna S.
        • Teslovich T.M.
        • Chines P.S.
        • Gliedt T.P.
        • et al.
        LocusZoom: Regional visualization of genome-wide association scan results.
        Bioinformatics. 2010; 26: 2336-2337
        • Butcher N.J.
        • Fung W.L.
        • Fitzpatrick L.
        • Guna A.
        • Andrade D.M.
        • Lang A.E.
        • et al.
        Response to clozapine in a clinically identifiable subtype of schizophrenia.
        Br J Psychiatry. 2015; 206: 484-491
        • Naber D.
        • Riedel M.
        • Klimke A.
        • Vorbach E.U.
        • Lambert M.
        • Kuhn K.U.
        • et al.
        Randomized double blind comparison of olanzapine vs. clozapine on subjective well-being and clinical outcome in patients with schizophrenia.
        Acta Psychiatr Scand. 2005; 111: 106-115

      Linked Article

      • Psychiatric Pharmacogenomics: How Close Are We?
        Biological PsychiatryVol. 80Issue 8
        • Preview
          A patient presents to your clinic with the results of a commercially available genetic panel and asks whether her medication regimen should be adjusted based on this information. How would you respond today? If you are an oncologist, you might be excited, and it is likely that you have invested significant time and energy learning how to interpret these results. However, if you are a psychiatrist, you might feel uncomfortable because of your relative unfamiliarity with this approach.
        • Full-Text
        • PDF