Validation study of randomly selected cases of PTSD diagnoses identified in a Swedish regional database compared with medical records: is the validity sufficient for epidemiological research?

Objectives In Sweden, the patients’ diagnoses are recorded in administrative registers. The research value of these registers is determined by their diagnostic validity, that is, if the diagnosis recorded meets the relevant diagnostic criteria. The aim of the study was to assess the validity of post-traumatic stress disorder (PTSD)-diagnoses as compared with case notes in medical records (MRs) and to test if there was a difference in validity by gender, migration status and those with and without psychotic symptoms. We hypothesised that the validity would be sufficient, using both Diagnostic and Statistical Manual of Mental Disorders (DSM)-IV and DSM-5 but higher according to DSM-IV than DSM-5, and that the validity would be the same for men and women, but different for Swedish-born and migrants, and for those with and without psychotic symptoms. Design and setting A validation of the register-diagnoses using MRs from treatment centres within the Region of Stockholm to examine whether patients with a register-diagnosis of PTSD fulfilled DSM criteria of PTSD according to the case notes in their MRs. Participants A random sample of 187 patients aged 18–64, who had been diagnosed with PTSD (F43.1 in the ICD-10) were drawn from the Region of Stockholm’s MR database 2013–2015. Primary outcome measure Validity of the PTSD diagnoses according to DSM-IV and DSM-5 as proportions of true positives with 95% CI. Results The hypothesised sufficient validity of the PTSD diagnoses was confirmed. Although the point-estimates for DSM-IV were higher than for DSM-5, the hypothesis that there would be significant differences in validity between DSM-IV and DSM-5 was not confirmed. There were no significant validity differences by gender, migration status and for those with and without psychotic symptoms. Conclusions This study has found that validity of the PTSD diagnoses in the register of the Region of Stockholm to be sufficient for epidemiological research.


GENERAL COMMENTS
This article reports the results of a small validation study of posttraumatic stress disorder diagnoses in a Swedish regional health database. While in general I find enormous value in validation studies of registry-based data, I believe the limitations of this study hinder its contribution to the literature. These are outlined below.
1) It is unclear to me why data that is coded according to ICD-10 would be validated against diagnostic criteria from another diagnostic classification system (in this case DSM). The symptoms of PTSD are not completely identical across ICD and DSM, so why would we expect ICD-based diagnoses to be valid according to DSM criteria? Further, the symptoms of PTSD changed between DSM-IV and DSM-V, so it is also unclear to me a) why we would expect diagnoses made using one set of symptoms to be valid according to the other and b) how diagnoses made during the DSM-IV time period could possibly be validated according to DSM-V because new symptoms were added to the DSM-V diagnosis that did not exist in the DSM-IV version.
2) Why were standard validity measures (e.g., positive predictive value or sensitivity and specificity) not calculated? 3) There is another larger validation study of PTSD (and other stress) diagnoses in the Danish national registries that the authors may wish to review: https://www.dovepress.com/validity-of-reaction-to-severe-stressand-adjustment-disorder-diagnose-peer-reviewed-fulltext-article-CLEP 4) Given the focus on validity of diagnoses across different subgroups there should be more information in the introduction about why this is important and why we might expect validity to vary in these groups. 5) I do not believe "feasible" is a term typically used to describe validity in the epidemiologic literature. 6) Was there a particular reason a sample size of 200 was chosen?

VERSION 1 -AUTHOR RESPONSE
Reviewer's Comments to Author: 1) It is unclear to me why data that is coded according to ICD-10 would be validated against diagnostic criteria from another diagnostic classification system (in this case DSM). The symptoms of PTSD are not completely identical across ICD and DSM, so why would we expect ICD-based diagnoses to be valid according to DSM criteria? Further, the symptoms of PTSD changed between DSM-IV and DSM-V, so it is also unclear to me a) why we would expect diagnoses made using one set of symptoms to be valid according to the other and b) how diagnoses made during the DSM-IV time period could possibly be validated according to DSM-V because new symptoms were added to the DSM-V diagnosis that did not exist in the DSM-IV version.
Thank you for giving us the opportunity to clarify this. Swedish psychiatrists use DSM and ICD interchangeably but code diagnoses in medical records using ICD 10 codes. Validity articles of the Swedish registers have traditionally used the DSM criteria (Ludvigsson et al, 2011) as neither the Swedish nor English versions of the ICD-10 contain operational diagnostic criteria, but, instead, they contain narrative descriptions of the disorders, justifying the validation against DSM criteria. We have written "We chose to assess the diagnoses in accordance with the DSM-system since it has become the global standard in psychiatric research and contains specific criteria for each diagnosis." The reason for validating against both DSMIV and DSM-5 is that the cases were obtain during the process of implementing DSM-5. 2) Why were standard validity measures (e.g., positive predictive value or sensitivity and specificity) not calculated?
Using positive predictive value is great advice and we have now changed the validity measure to positive predictive value (ppv) throughout. At page 4 under Statistical methods the wording is now: "We calculated the degree of coherence between raters, in percent and validity as positive predictive value (ppv) among the register diagnoses with 95% confidence interval (95% CI), ppv is defined as number of patients with a PTSD register diagnosis confirmed in the medical records divided by the total number of patients with a PTSD register diagnosis that we were able to validate against medical records." Table 2 and 3 are now displaying the results in ppvs. However, this did not alter the results or conclusions.  3) There is another larger validation study of PTSD (and other stress) diagnoses in the Danish national registries that the authors may wish to review: https://www.dovepress.com/validity-of-reaction-to-severe-stress-and-adjustment-disorder-diagnosepeer-reviewed-fulltext-article-CLEP Thank you for giving us the opportunity to quote this excellent paper. We are quoting it in the end of page 3: "A small validation study of PTSD was performed in Denmark 2015 testing the validity of PTSD 13 defined according to ICD-10 12 . The study used 18 cases of PTSD from the Danish Psychiatric Central Research Register and found a positive predictive value of 83%." 4) Given the focus on validity of diagnoses across different subgroups there should be more information in the introduction about why this is important and why we might expect validity to vary in these groups.
We are now highlighting this clearer wording it as follows: "Studies of refugees have, so far, often had a focus on PTSD, however, a study from 2016 showed that in particular male refugees also have an increased risk of non-affective psychosis 8 . This study has intensified an already ongoing debate regarding the validity of PTSD diagnoses among male and female migrants, especially with psychosis, as for mental health professionals, different cultural variation in presentation of psychiatric symptoms contributes to risk of being misdiagnosed 9 ." 5) I do not believe "feasible" is a term typically used to describe validity in the epidemiologic literature.
We have now altered the wording to "sufficient". 6) Was there a particular reason a sample size of 200 was chosen? Thank you for giving us the opportunity to clarify this. It was chosen in order to have the statistical power to do the sub-group calculations even if the validity was low and we now described this at page four with the wording "The reason for including 200 patients were to have the statistical power to do the sub-group calculations even if the validity was found to be low.".