Intra- and inter-observer agreement of brain MRI lesion volume measurements in multiple sclerosis: A comparison of techniques

M. Filippi, M. A. Horsfield, S. Bressi, V. Martinelli, C. Baratti, P. Reganati, A. Campi, D. H. Miller, G. Comi

Research output: Contribution to journalArticle

Abstract

The measurement of MRI lesion load in multiple sclerosis is increasingly being used to evaluate the natural history of the disease and to monitor the efficacy of treatments. If, as might occur in multicentre studies, lesion load is measured by several observers in different patients or by the same observer in serial scans, it would be necessary to utilize a technique which provides results with high inter- and intra-observer agreements. This study was performed to evaluate the intra-and inter-observer agreement of semi-automated lesion volume measurement using thresholding, and to compare them with those obtained using an arbitrary scoring system (ASS) and a quantitative manual tracing method (MTM). Brain MRIs were obtained for 20 clinically definite multiple sclerosis patients and were evaluated independently by three observers. The median intra- and inter-observer agreements were, respectively, 88.5% (range 69.0-96.8%) and 79.0% (range 73.3-98.3%) using the ASS, 95.0% (range 85.1-99.4%) and 93.4% (range 77.3-98.3%) for the MTM, 96.3% (range 94.2-98.9%) and 93.7% (range 83.8-98.3%) for the semi-automated technique. The intra- and inter-observer agreements for the semi-automated technique increased to 98.5% (range 96.3-99.8%) and 96.1% (range 90.5-98.6%) when a consensus in the choice of threshold for lesion segmentation was reached. The intra- and inter-observer agreements were significantly greater for the semi-automated method compared with both the arbitrary scoring and the MTMs. The intra-observer variability for the semi-automated technique was significantly lower (P <0.0001) than the inter-observer variability obtained using the same technique. These data indicate that it is possible to obtain high intra-and inter-observer agreements using a semi-automatic thresholding technique to quantify lesion volumes in multiple sclerosis. The technique may prove useful in multicentre studies, in which a single observer is still preferable.

Original languageEnglish
Pages (from-to)1593-1600
Number of pages8
JournalBrain
Volume118
Issue number6
Publication statusPublished - Dec 1995

Fingerprint

Multiple Sclerosis
Volume measurement
sclerosis
lesions (animal)
Magnetic resonance imaging
Observer
Brain
Observer Variation
brain
Multicenter Studies
methodology
Range of data
Scoring
Multiple sclerosis
Thresholding
Tracing
Arbitrary
Evaluate

Keywords

  • Brain MRI
  • Intra- and inter-observer agreement
  • Lesion volume measurements
  • Multiple sclerosis

ASJC Scopus subject areas

  • Neuroscience(all)
  • Statistics, Probability and Uncertainty
  • Applied Mathematics
  • Mathematics(all)
  • Statistics and Probability
  • Agricultural and Biological Sciences (miscellaneous)
  • Clinical Neurology

Cite this

Intra- and inter-observer agreement of brain MRI lesion volume measurements in multiple sclerosis : A comparison of techniques. / Filippi, M.; Horsfield, M. A.; Bressi, S.; Martinelli, V.; Baratti, C.; Reganati, P.; Campi, A.; Miller, D. H.; Comi, G.

In: Brain, Vol. 118, No. 6, 12.1995, p. 1593-1600.

Research output: Contribution to journalArticle

Filippi, M, Horsfield, MA, Bressi, S, Martinelli, V, Baratti, C, Reganati, P, Campi, A, Miller, DH & Comi, G 1995, 'Intra- and inter-observer agreement of brain MRI lesion volume measurements in multiple sclerosis: A comparison of techniques', Brain, vol. 118, no. 6, pp. 1593-1600.
Filippi, M. ; Horsfield, M. A. ; Bressi, S. ; Martinelli, V. ; Baratti, C. ; Reganati, P. ; Campi, A. ; Miller, D. H. ; Comi, G. / Intra- and inter-observer agreement of brain MRI lesion volume measurements in multiple sclerosis : A comparison of techniques. In: Brain. 1995 ; Vol. 118, No. 6. pp. 1593-1600.
@article{f881aae4244049bca7bbe875349fef54,
title = "Intra- and inter-observer agreement of brain MRI lesion volume measurements in multiple sclerosis: A comparison of techniques",
abstract = "The measurement of MRI lesion load in multiple sclerosis is increasingly being used to evaluate the natural history of the disease and to monitor the efficacy of treatments. If, as might occur in multicentre studies, lesion load is measured by several observers in different patients or by the same observer in serial scans, it would be necessary to utilize a technique which provides results with high inter- and intra-observer agreements. This study was performed to evaluate the intra-and inter-observer agreement of semi-automated lesion volume measurement using thresholding, and to compare them with those obtained using an arbitrary scoring system (ASS) and a quantitative manual tracing method (MTM). Brain MRIs were obtained for 20 clinically definite multiple sclerosis patients and were evaluated independently by three observers. The median intra- and inter-observer agreements were, respectively, 88.5{\%} (range 69.0-96.8{\%}) and 79.0{\%} (range 73.3-98.3{\%}) using the ASS, 95.0{\%} (range 85.1-99.4{\%}) and 93.4{\%} (range 77.3-98.3{\%}) for the MTM, 96.3{\%} (range 94.2-98.9{\%}) and 93.7{\%} (range 83.8-98.3{\%}) for the semi-automated technique. The intra- and inter-observer agreements for the semi-automated technique increased to 98.5{\%} (range 96.3-99.8{\%}) and 96.1{\%} (range 90.5-98.6{\%}) when a consensus in the choice of threshold for lesion segmentation was reached. The intra- and inter-observer agreements were significantly greater for the semi-automated method compared with both the arbitrary scoring and the MTMs. The intra-observer variability for the semi-automated technique was significantly lower (P <0.0001) than the inter-observer variability obtained using the same technique. These data indicate that it is possible to obtain high intra-and inter-observer agreements using a semi-automatic thresholding technique to quantify lesion volumes in multiple sclerosis. The technique may prove useful in multicentre studies, in which a single observer is still preferable.",
keywords = "Brain MRI, Intra- and inter-observer agreement, Lesion volume measurements, Multiple sclerosis",
author = "M. Filippi and Horsfield, {M. A.} and S. Bressi and V. Martinelli and C. Baratti and P. Reganati and A. Campi and Miller, {D. H.} and G. Comi",
year = "1995",
month = "12",
language = "English",
volume = "118",
pages = "1593--1600",
journal = "Brain",
issn = "0006-8950",
publisher = "Oxford University Press",
number = "6",

}

TY - JOUR

T1 - Intra- and inter-observer agreement of brain MRI lesion volume measurements in multiple sclerosis

T2 - A comparison of techniques

AU - Filippi, M.

AU - Horsfield, M. A.

AU - Bressi, S.

AU - Martinelli, V.

AU - Baratti, C.

AU - Reganati, P.

AU - Campi, A.

AU - Miller, D. H.

AU - Comi, G.

PY - 1995/12

Y1 - 1995/12

N2 - The measurement of MRI lesion load in multiple sclerosis is increasingly being used to evaluate the natural history of the disease and to monitor the efficacy of treatments. If, as might occur in multicentre studies, lesion load is measured by several observers in different patients or by the same observer in serial scans, it would be necessary to utilize a technique which provides results with high inter- and intra-observer agreements. This study was performed to evaluate the intra-and inter-observer agreement of semi-automated lesion volume measurement using thresholding, and to compare them with those obtained using an arbitrary scoring system (ASS) and a quantitative manual tracing method (MTM). Brain MRIs were obtained for 20 clinically definite multiple sclerosis patients and were evaluated independently by three observers. The median intra- and inter-observer agreements were, respectively, 88.5% (range 69.0-96.8%) and 79.0% (range 73.3-98.3%) using the ASS, 95.0% (range 85.1-99.4%) and 93.4% (range 77.3-98.3%) for the MTM, 96.3% (range 94.2-98.9%) and 93.7% (range 83.8-98.3%) for the semi-automated technique. The intra- and inter-observer agreements for the semi-automated technique increased to 98.5% (range 96.3-99.8%) and 96.1% (range 90.5-98.6%) when a consensus in the choice of threshold for lesion segmentation was reached. The intra- and inter-observer agreements were significantly greater for the semi-automated method compared with both the arbitrary scoring and the MTMs. The intra-observer variability for the semi-automated technique was significantly lower (P <0.0001) than the inter-observer variability obtained using the same technique. These data indicate that it is possible to obtain high intra-and inter-observer agreements using a semi-automatic thresholding technique to quantify lesion volumes in multiple sclerosis. The technique may prove useful in multicentre studies, in which a single observer is still preferable.

AB - The measurement of MRI lesion load in multiple sclerosis is increasingly being used to evaluate the natural history of the disease and to monitor the efficacy of treatments. If, as might occur in multicentre studies, lesion load is measured by several observers in different patients or by the same observer in serial scans, it would be necessary to utilize a technique which provides results with high inter- and intra-observer agreements. This study was performed to evaluate the intra-and inter-observer agreement of semi-automated lesion volume measurement using thresholding, and to compare them with those obtained using an arbitrary scoring system (ASS) and a quantitative manual tracing method (MTM). Brain MRIs were obtained for 20 clinically definite multiple sclerosis patients and were evaluated independently by three observers. The median intra- and inter-observer agreements were, respectively, 88.5% (range 69.0-96.8%) and 79.0% (range 73.3-98.3%) using the ASS, 95.0% (range 85.1-99.4%) and 93.4% (range 77.3-98.3%) for the MTM, 96.3% (range 94.2-98.9%) and 93.7% (range 83.8-98.3%) for the semi-automated technique. The intra- and inter-observer agreements for the semi-automated technique increased to 98.5% (range 96.3-99.8%) and 96.1% (range 90.5-98.6%) when a consensus in the choice of threshold for lesion segmentation was reached. The intra- and inter-observer agreements were significantly greater for the semi-automated method compared with both the arbitrary scoring and the MTMs. The intra-observer variability for the semi-automated technique was significantly lower (P <0.0001) than the inter-observer variability obtained using the same technique. These data indicate that it is possible to obtain high intra-and inter-observer agreements using a semi-automatic thresholding technique to quantify lesion volumes in multiple sclerosis. The technique may prove useful in multicentre studies, in which a single observer is still preferable.

KW - Brain MRI

KW - Intra- and inter-observer agreement

KW - Lesion volume measurements

KW - Multiple sclerosis

UR - http://www.scopus.com/inward/record.url?scp=0029584874&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0029584874&partnerID=8YFLogxK

M3 - Article

C2 - 8595488

AN - SCOPUS:0029584874

VL - 118

SP - 1593

EP - 1600

JO - Brain

JF - Brain

SN - 0006-8950

IS - 6

ER -