Comparing metrics to evaluate performance of regression methods for decoding of neural signals

Martin Spuler, Andrea Sarasola-Sanz, Niels Birbaumer, Wolfgang Rosenstiel, Ander Ramos-Murguialday

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The use of regression methods for decoding of neural signals has become popular, with its main applications in the field of Brain-Machine Interfaces (BMIs) for control of prosthetic devices or in the area of Brain-Computer Interfaces (BCIs) for cursor control. When new methods for decoding are being developed or the parameters for existing methods should be optimized to increase performance, a metric is needed that gives an accurate estimate of the prediction error. In this paper, we evaluate different performance metrics regarding their robustness for assessing prediction errors. Using simulated data, we show that different kinds of prediction error (noise, scaling error, bias) have different effects on the different metrics and evaluate which methods are best to assess the overall prediction error, as well as the individual types of error. Based on the obtained results we can conclude that the most commonly used metrics correlation coefficient (CC) and normalized root-mean-squared error (NRMSE) are well suited for evaluation of cross-validated results, but should not be used as sole criterion for cross-subject or cross-session evaluations.

Original languageEnglish
Title of host publicationProceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1083-1086
Number of pages4
Volume2015-November
ISBN (Print)9781424492718
DOIs
Publication statusPublished - Nov 4 2015
Event37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2015 - Milan, Italy
Duration: Aug 25 2015Aug 29 2015

Other

Other37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2015
CountryItaly
CityMilan
Period8/25/158/29/15

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Signal Processing
  • Biomedical Engineering
  • Health Informatics

Fingerprint Dive into the research topics of 'Comparing metrics to evaluate performance of regression methods for decoding of neural signals'. Together they form a unique fingerprint.

  • Cite this

    Spuler, M., Sarasola-Sanz, A., Birbaumer, N., Rosenstiel, W., & Ramos-Murguialday, A. (2015). Comparing metrics to evaluate performance of regression methods for decoding of neural signals. In Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS (Vol. 2015-November, pp. 1083-1086). [7318553] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/EMBC.2015.7318553