The reliability of transcranial duplex scanning in parkinsonian patients: comparison of different observers and ultrasound systems

Purpose: To establish the inter-observer and intra-transducer reliability of "on-line" and "off-line" assessment of substantia nigra (SN) and raphe nuclei (RN) by transcranial duplex scanning (TCD) in a mixed study population.

Materials and methods: Out-patient neurology department of the University Hospital Maastricht. In total 24 subjects were investigated: 9 patients with idiopathic Parkinson's disease, 10 with parkinsonism from yet unclear origin, 1 with essential tremor and 4 healthy volunteers. Each patient was assessed four times by two independent experienced sonographers using two different ultrasound devices: SONOS 5500 and iU22; both Philips, Eindhoven, The Netherlands. The echointensity of the SN is evaluated qualitatively and quantitatively and the RN only qualitatively. 1. In the "on-line" assessment we determined: a) the inter-observer agreement of the four possible combinations. b) the intra-observer agreement of both sonographers using two different ultrasound systems. 2. In the "off-line" assessment a third sonographer re-examined the stored images. We determined the inter-observer agreement of the third sonographer with the "on-line" assessment of the other two sonographers. Cohen's k value was calculated for the agreement.

Results: 1a) The "on-line" inter-observer agreement of the four possible combinations of sonographer and transducer was: kappa 0.23 - 0.39 for the qualitative evaluation of the SN, kappa 0.31 - 0.56 for the quantitative evaluation of the SN and kappa 0.03 - 0.15 for the evaluation of the RN. 1b) The "on-line" intra-observer agreement was: kappa 0.53 - 0.67 for the qualitative evaluation of the SN, kappa 0.55 - 0.76 for the quantitative evaluation of the SN and kappa 0.45 - 0.47 for the evaluation of the RN. 2. The "off-line" inter-observer agreement was: kappa 0.32 - 0.67 for the qualitative evaluation of the SN, kappa 0.53 - 0.61 for the quantitative evaluation of the SN and kappa 0.08 - 0.33 for the evaluation of the RN.

Conclusion: For the SN we found mediocre accordance comparing both observers "on-line" with each other as well as comparing an "off-line" observer with both "on-line" observers. On the whole, the inter-observer and intra-observer agreement were moderate to substantial for the evaluation of the SN. "On-line" and "off-line" comparisons yielded comparable results. The agreement for the evaluation of the RN, on the contrary, was considerably lower. Our findings indicate that this TCD technique is not yet ready for the application in large population screenings.