Deep unsupervised learning on a desktop pc

A primer for cognitive scientists

Alberto Testolin, Ivilin Stoianov, Michele De Filippo De Grazia, Marco Zorzi

Research output: Contribution to journalArticle

17 Citations (Scopus)

Abstract

Deep belief networks hold great promise for the simulation of human cognition because they show how structured and abstract representations may emerge from probabilistic unsupervised learning. These networks build a hierarchy of progressively more complex distributed representations of the sensory data by fitting a hierarchical generative model. However, learning in deep networks typically requires big datasets and it can involve millions of connection weights, which implies that simulations on standard computers are unfeasible. Developing realistic, medium-to-large-scale learning models of cognition would therefore seem to require expertise in programing parallel-computing hardware, and this might explain why the use of this promising approach is still largely confined to the machine learning community. Here we show how simulations of deep unsupervised learning can be easily performed on a desktop PC by exploiting the processors of low cost graphic cards (graphic processor units) without any specific programing effort, thanks to the use of high-level programming routines (available in MATLAB or Python). We also show that even an entry-level graphic card can outperform a small high-performance computing cluster in terms of learning time and with no loss of learning quality. We therefore conclude that graphic card implementations pave the way for a widespread use of deep learning among cognitive scientists for modeling cognition and behavior.

Original languageEnglish
Article numberArticle 251
JournalFrontiers in Psychology
Volume4
Issue numberMAY
DOIs
Publication statusPublished - 2013

Fingerprint

Learning
Cognition
Computing Methodologies
Boidae
Weights and Measures
Costs and Cost Analysis

Keywords

  • Cognitive modeling
  • Computer cluster
  • Deep neural networks
  • GPUs
  • Hierarchical generative models
  • MPI
  • Parallel-computing architectures
  • Unsupervised learning

ASJC Scopus subject areas

  • Psychology(all)

Cite this

Testolin, A., Stoianov, I., De Filippo De Grazia, M., & Zorzi, M. (2013). Deep unsupervised learning on a desktop pc: A primer for cognitive scientists. Frontiers in Psychology, 4(MAY), [Article 251]. https://doi.org/10.3389/fpsyg.2013.00251

Deep unsupervised learning on a desktop pc : A primer for cognitive scientists. / Testolin, Alberto; Stoianov, Ivilin; De Filippo De Grazia, Michele; Zorzi, Marco.

In: Frontiers in Psychology, Vol. 4, No. MAY, Article 251, 2013.

Research output: Contribution to journalArticle

Testolin, A, Stoianov, I, De Filippo De Grazia, M & Zorzi, M 2013, 'Deep unsupervised learning on a desktop pc: A primer for cognitive scientists', Frontiers in Psychology, vol. 4, no. MAY, Article 251. https://doi.org/10.3389/fpsyg.2013.00251
Testolin, Alberto ; Stoianov, Ivilin ; De Filippo De Grazia, Michele ; Zorzi, Marco. / Deep unsupervised learning on a desktop pc : A primer for cognitive scientists. In: Frontiers in Psychology. 2013 ; Vol. 4, No. MAY.
@article{c7bd63f452ea4d2ba7932ff765adb53d,
title = "Deep unsupervised learning on a desktop pc: A primer for cognitive scientists",
abstract = "Deep belief networks hold great promise for the simulation of human cognition because they show how structured and abstract representations may emerge from probabilistic unsupervised learning. These networks build a hierarchy of progressively more complex distributed representations of the sensory data by fitting a hierarchical generative model. However, learning in deep networks typically requires big datasets and it can involve millions of connection weights, which implies that simulations on standard computers are unfeasible. Developing realistic, medium-to-large-scale learning models of cognition would therefore seem to require expertise in programing parallel-computing hardware, and this might explain why the use of this promising approach is still largely confined to the machine learning community. Here we show how simulations of deep unsupervised learning can be easily performed on a desktop PC by exploiting the processors of low cost graphic cards (graphic processor units) without any specific programing effort, thanks to the use of high-level programming routines (available in MATLAB or Python). We also show that even an entry-level graphic card can outperform a small high-performance computing cluster in terms of learning time and with no loss of learning quality. We therefore conclude that graphic card implementations pave the way for a widespread use of deep learning among cognitive scientists for modeling cognition and behavior.",
keywords = "Cognitive modeling, Computer cluster, Deep neural networks, GPUs, Hierarchical generative models, MPI, Parallel-computing architectures, Unsupervised learning",
author = "Alberto Testolin and Ivilin Stoianov and {De Filippo De Grazia}, Michele and Marco Zorzi",
year = "2013",
doi = "10.3389/fpsyg.2013.00251",
language = "English",
volume = "4",
journal = "Frontiers in Psychology",
issn = "1664-1078",
publisher = "Frontiers Media S.A.",
number = "MAY",

}

TY - JOUR

T1 - Deep unsupervised learning on a desktop pc

T2 - A primer for cognitive scientists

AU - Testolin, Alberto

AU - Stoianov, Ivilin

AU - De Filippo De Grazia, Michele

AU - Zorzi, Marco

PY - 2013

Y1 - 2013

N2 - Deep belief networks hold great promise for the simulation of human cognition because they show how structured and abstract representations may emerge from probabilistic unsupervised learning. These networks build a hierarchy of progressively more complex distributed representations of the sensory data by fitting a hierarchical generative model. However, learning in deep networks typically requires big datasets and it can involve millions of connection weights, which implies that simulations on standard computers are unfeasible. Developing realistic, medium-to-large-scale learning models of cognition would therefore seem to require expertise in programing parallel-computing hardware, and this might explain why the use of this promising approach is still largely confined to the machine learning community. Here we show how simulations of deep unsupervised learning can be easily performed on a desktop PC by exploiting the processors of low cost graphic cards (graphic processor units) without any specific programing effort, thanks to the use of high-level programming routines (available in MATLAB or Python). We also show that even an entry-level graphic card can outperform a small high-performance computing cluster in terms of learning time and with no loss of learning quality. We therefore conclude that graphic card implementations pave the way for a widespread use of deep learning among cognitive scientists for modeling cognition and behavior.

AB - Deep belief networks hold great promise for the simulation of human cognition because they show how structured and abstract representations may emerge from probabilistic unsupervised learning. These networks build a hierarchy of progressively more complex distributed representations of the sensory data by fitting a hierarchical generative model. However, learning in deep networks typically requires big datasets and it can involve millions of connection weights, which implies that simulations on standard computers are unfeasible. Developing realistic, medium-to-large-scale learning models of cognition would therefore seem to require expertise in programing parallel-computing hardware, and this might explain why the use of this promising approach is still largely confined to the machine learning community. Here we show how simulations of deep unsupervised learning can be easily performed on a desktop PC by exploiting the processors of low cost graphic cards (graphic processor units) without any specific programing effort, thanks to the use of high-level programming routines (available in MATLAB or Python). We also show that even an entry-level graphic card can outperform a small high-performance computing cluster in terms of learning time and with no loss of learning quality. We therefore conclude that graphic card implementations pave the way for a widespread use of deep learning among cognitive scientists for modeling cognition and behavior.

KW - Cognitive modeling

KW - Computer cluster

KW - Deep neural networks

KW - GPUs

KW - Hierarchical generative models

KW - MPI

KW - Parallel-computing architectures

KW - Unsupervised learning

UR - http://www.scopus.com/inward/record.url?scp=84878913949&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84878913949&partnerID=8YFLogxK

U2 - 10.3389/fpsyg.2013.00251

DO - 10.3389/fpsyg.2013.00251

M3 - Article

VL - 4

JO - Frontiers in Psychology

JF - Frontiers in Psychology

SN - 1664-1078

IS - MAY

M1 - Article 251

ER -