Abstract
Signals derived from the rat motor cortex can be used for controlling one-dimensional movements of a robot arm1. It remains unknown, however, whether real-time processing of cortical signals can be employed to reproduce, in a robotic device, the kind of complex arm movements used by primates to reach objects in space. Here we recorded the simultaneous activity of large populations of neurons, distributed in the premotor, primary motor and posterior parietal cortical areas, as non-human primates performed two distinct motor tasks. Accurate real-time predictions of one- and three-dimensional arm movement trajectories were obtained by applying both linear and nonlinear algorithms to cortical neuronal ensemble activity recorded from each animal. In addition, cortically derived signals were successfully used for real-time control of robotic devices, both locally and through the Internet. These results suggest that long-term control of complex prosthetic robot arm movements can be achieved by simple real-time transformations of neuronal population signals derived from multiple cortical areas in primates.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 51 print issues and online access
199,00 € per year
only 3,90 € per issue
Buy this article
- Purchase on SpringerLink
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
References
Chapin, J. K., Moxon, K. A., Markowitz, R. S. & Nicolelis, M. A. L. Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex. Nature Neurosci. 2, 664 –670 (1999).
Evarts, E. V. Pyramidal tract activity associated with a conditioned hand movement in the monkey. J. Neurophysiol. 29, 1011– 1027 (1966).
Mountcastle, V. B., Lynch, J. C., Georgopoulos, A., Sakata, H. & Acuna, C. Posterior parietal association cortex of the monkey: command functions for operation within extrapersonal space. J. Neurophysiol. 38, 871– 908 (1975).
Fetz, E. E. & Cheney, P. D. Muscle fields of primate corticomotoneuronal cells. J. Physiol. (Paris) 74, 239– 245 (1978).
Georgopoulos, A. P., Kalaska, J. F., Caminiti, R. & Massey, J. T. On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex. J. Neurosci. 2, 1527–1537 (1982).
Weinrich, M. & Wise, S. P. The premotor cortex of the monkey. J. Neurosci. 2, 1329–1345 (1982).
Wise, S. P., Boussaoud, D., Johnson, P. B. & Caminiti, R. Premotor and parietal cortex: corticocortical connectivity and combinatorial computations. Annu. Rev. Neurosci. 20, 25 –42 (1997).
Batista, A. P., Buneo, C. A., Snyder, L. H. & Andersen, R. A. Reach plans in eye-centered coordinates. Science 285 , 257–260 (1999).
Mitz, A. R., Godschalk, M. & Wise, S. P. Learning-dependent neuronal activity in the premotor cortex: activity during the acquisition of conditional motor associations. J. Neurosci. 11, 1855– 1872 (1991).
Humphrey, D. R., Schmidt, E. M. & Thompson, W. D. Predicting measures of motor performance from multiple cortical spike trains. Science 170, 758– 762 (1970).
Georgopoulos, A. P., Schwartz, A. B. & Kettner, R. E. Neuronal population coding of movement direction. Science 233, 1416–1419 (1986).
Schwartz, A. Direct cortical representation of drawing. Science 265, 540–542 (1994).
Fetz, E. E. & Cheney, P. D. Postspike facilitation of forelimb muscle activity by primate corticomotoneuronal cells. J. Neurophysiol. 44, 751–772 ( 1980).
Nicolelis, A. L. et al. Simultaneous representation of tactile information by distinct primate cortical areas rely on different encoding strategies. Nature Neurosci. 1, 621–630 (1998).
Laubach, M., Wessberg, J. & Nicolelis, M. A. L. Cortical ensemble activity increasingly predicts behaviour outcomes during learning of a motor task. Nature 405, 567–571 (2000).
Nicolelis, M. A. L., Ghazanfar, A. A., Faggin, B. M., Votaw, S. & Oliveira, L. M. Reconstructing the engram: simultaneous, multisite, many single neuron recordings. Neuron 18 , 529–537 (1997).
Stepniewska, I., Preuss, T. M. & Kaas, J. H. Architectonics, somatotopic organization, and ipsilateral cortical connections of the primary motor area (M1) of owl monkeys. J. Comp. Neurol. 330, 238–271 (1993).
Preuss, T. M., Stepniewska, I. & Kaas, J. H. Movement representation in the dorsal and ventral premotor areas of owl monkeys: a microstimulation study. J. Comp. Neurol. 371, 649–676 ( 1996).
Brillinger, D. R. Time Series. Data Analysis and Theory (Holden-Day, San Fransisco, 1981).
Bendat, J. S. & Piersol, A. G. Random Data. Analysis and Measurement Procedures (Wiley, New York, 1986).
Halliday, D. M. et al. The Fourier approach to the analysis of mixed time series/point process data. Theory and application to the study of physiological tremor, single motor unit discharges and electromyograms. Prog. Biophys. Mol. Biol. 64, 237–278 ( 1995).
Powell, M. J. D. Restart procedures for the conjugate gradient method. Math. Program. 12, 241–254 ( 1977).
Ghazanfar, A. A., Stambaugh, C. R. & Nicolelis, M. A. L. Encoding of tactile stimulus location by somatosensory thalamocortical ensembles. J. Neurosci. 20, 3761–3775 (2000).
Salisbury, J. K. & Srinivasan, M. A. Phantom-based haptic interaction with virtual objects. IEEE Comput. Graph. Appl. 17, 6–10 (1997 ).
Ferraina, S. et al. Combination of hand and gaze signals during reaching: Activity in parietal area 7m of the monkey. J. Neurophysiol. 77, 1034–1038 (1997).
Mussa-Ivaldi, F. A. Do neurons in the motor cortex encode movement direction? An alternative hypothesis. Neurosci. Lett. 91, 106– 111 (1988).
Scott, S. H., Sergio, L. E. & Kalaska, J. F. Reaching movements with similar hand paths but different arm orientations. II. Activity of individual cells in dorsal premotor cortex and parietal area 5. J. Neurophysiol. 78, 2413–2426 (1997).
Schmidt, E. M. Single neuron recording from motor cortex as a possible source of signals for control of external devices. Ann. Biomed. Eng. 8, 339–349 (1980).
Kennedy P. R. & Bakay, R. A. Restoration of neural output from a paralyzed patient by a direct brain connection. Neuroreport 9, 1707–1711 (1998).
Nicolelis, M. A. L. Hybrid brain-machine interfaces for translating thoughts into action. Nature (in the press).
Acknowledgements
This work was supported by grants from the National Institutes of Health and DARPA-ONR to M.A.L.N., J.K.C. and M.A.S., and the National Science Foundation to M.A.L.N. J.W. was supported by The Swedish Foundation for International Cooperation in Research and Higher Education, and the Swedish Medical Research Council. J.K., P.B. and M.L. were supported by NIH postdoctoral fellowships.
Author information
Authors and Affiliations
Corresponding author
Supplementary information
Rights and permissions
About this article
Cite this article
Wessberg, J., Stambaugh, C., Kralik, J. et al. Real-time prediction of hand trajectory by ensembles of cortical neurons in primates. Nature 408, 361–365 (2000). https://doi.org/10.1038/35042582
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1038/35042582
