ORCA.IT: A New Web-Based Tool for Assessing Online Reading, Search and Comprehension Abilities in Students Reveals Effects of Gender, School Type and Reading Ability

Martina Caccia, Marisa Giorgetti, Alessio Toraldo, Massimo Molteni, Daniela Sarti, Mirta Vernice, Maria Luisa Lorusso

Research output: Contribution to journalArticle

Abstract

ORCA.IT, a new online test of online research and comprehension was developed for the Italian population. A group of 183 students attending various types of upper secondary schools in Northern Italy were tested with the new tool and underwent further cognitive and neuropsychological assessment. The different school types involved in the study are representative of the school population in the Italian system, but can also be easily compared with the educational systems of other countries. The new test turned out to have good psychometric properties after accurate item construction and final selection. In particular, Version 1 showed better characteristics than Version 2. Subsequently, comparison with one-way ANOVAs were performed to test whether differences exist between different school types, between groups with and without reading difficulties, and between males and females. Such differences are sometimes reported in the literature, but many remain controversial. Further, Pearson’s bivariate correlations were calculated to analyze associations between scores on the ORCA.IT and cognitive/neuropsychological variables. Finally, a stepwise regression analysis was performed on aggregated scores to identify the predictors of performance on each of the two versions. The test, especially in the most complete version (Version 1), appears to accurately and reliably capture students’ web searching abilities and online reading comprehension. The tool could highlight differences in online search and comprehension ability between students with and without reading difficulties, not penalizing overall performance but allowing very specific weaknesses to be pointed out. Further, it seems to be able to capture differences due to both educational pathways (different school types) and social attitudes (differences between males and females). Most interestingly, it shows to be clearly resting on specific cognitive and neuropsychological abilities, including language, memory, and attentional skills, which explain a large portion of the total variance. Offline text reading comprehension is a crucial predictor of online reading performance, while decoding ability is not. Prior knowledge also influences the results, as expected. The new tool turns out to be rather independent of previous Internet experience and to measure more cognitively grounded processes related to information gathering, processing, and communicating.

Original languageEnglish
Article number2433
JournalFrontiers in Psychology
Volume10
DOIs
Publication statusPublished - Nov 1 2019

Fingerprint

Aptitude
Reading
Students
Automatic Data Processing
Psychometrics
Internet
Italy
Population
Analysis of Variance
Language
Regression Analysis
Research

Keywords

  • assessment
  • education effects
  • gender effects
  • online reading comprehension
  • reading difficulties

ASJC Scopus subject areas

  • Psychology(all)

Cite this

@article{4275e11441c845f890e34326d040f820,
title = "ORCA.IT: A New Web-Based Tool for Assessing Online Reading, Search and Comprehension Abilities in Students Reveals Effects of Gender, School Type and Reading Ability",
abstract = "ORCA.IT, a new online test of online research and comprehension was developed for the Italian population. A group of 183 students attending various types of upper secondary schools in Northern Italy were tested with the new tool and underwent further cognitive and neuropsychological assessment. The different school types involved in the study are representative of the school population in the Italian system, but can also be easily compared with the educational systems of other countries. The new test turned out to have good psychometric properties after accurate item construction and final selection. In particular, Version 1 showed better characteristics than Version 2. Subsequently, comparison with one-way ANOVAs were performed to test whether differences exist between different school types, between groups with and without reading difficulties, and between males and females. Such differences are sometimes reported in the literature, but many remain controversial. Further, Pearson’s bivariate correlations were calculated to analyze associations between scores on the ORCA.IT and cognitive/neuropsychological variables. Finally, a stepwise regression analysis was performed on aggregated scores to identify the predictors of performance on each of the two versions. The test, especially in the most complete version (Version 1), appears to accurately and reliably capture students’ web searching abilities and online reading comprehension. The tool could highlight differences in online search and comprehension ability between students with and without reading difficulties, not penalizing overall performance but allowing very specific weaknesses to be pointed out. Further, it seems to be able to capture differences due to both educational pathways (different school types) and social attitudes (differences between males and females). Most interestingly, it shows to be clearly resting on specific cognitive and neuropsychological abilities, including language, memory, and attentional skills, which explain a large portion of the total variance. Offline text reading comprehension is a crucial predictor of online reading performance, while decoding ability is not. Prior knowledge also influences the results, as expected. The new tool turns out to be rather independent of previous Internet experience and to measure more cognitively grounded processes related to information gathering, processing, and communicating.",
keywords = "assessment, education effects, gender effects, online reading comprehension, reading difficulties",
author = "Martina Caccia and Marisa Giorgetti and Alessio Toraldo and Massimo Molteni and Daniela Sarti and Mirta Vernice and Lorusso, {Maria Luisa}",
year = "2019",
month = "11",
day = "1",
doi = "10.3389/fpsyg.2019.02433",
language = "English",
volume = "10",
journal = "Frontiers in Psychology",
issn = "1664-1078",
publisher = "Frontiers Media S.A.",

}

TY - JOUR

T1 - ORCA.IT

T2 - A New Web-Based Tool for Assessing Online Reading, Search and Comprehension Abilities in Students Reveals Effects of Gender, School Type and Reading Ability

AU - Caccia, Martina

AU - Giorgetti, Marisa

AU - Toraldo, Alessio

AU - Molteni, Massimo

AU - Sarti, Daniela

AU - Vernice, Mirta

AU - Lorusso, Maria Luisa

PY - 2019/11/1

Y1 - 2019/11/1

N2 - ORCA.IT, a new online test of online research and comprehension was developed for the Italian population. A group of 183 students attending various types of upper secondary schools in Northern Italy were tested with the new tool and underwent further cognitive and neuropsychological assessment. The different school types involved in the study are representative of the school population in the Italian system, but can also be easily compared with the educational systems of other countries. The new test turned out to have good psychometric properties after accurate item construction and final selection. In particular, Version 1 showed better characteristics than Version 2. Subsequently, comparison with one-way ANOVAs were performed to test whether differences exist between different school types, between groups with and without reading difficulties, and between males and females. Such differences are sometimes reported in the literature, but many remain controversial. Further, Pearson’s bivariate correlations were calculated to analyze associations between scores on the ORCA.IT and cognitive/neuropsychological variables. Finally, a stepwise regression analysis was performed on aggregated scores to identify the predictors of performance on each of the two versions. The test, especially in the most complete version (Version 1), appears to accurately and reliably capture students’ web searching abilities and online reading comprehension. The tool could highlight differences in online search and comprehension ability between students with and without reading difficulties, not penalizing overall performance but allowing very specific weaknesses to be pointed out. Further, it seems to be able to capture differences due to both educational pathways (different school types) and social attitudes (differences between males and females). Most interestingly, it shows to be clearly resting on specific cognitive and neuropsychological abilities, including language, memory, and attentional skills, which explain a large portion of the total variance. Offline text reading comprehension is a crucial predictor of online reading performance, while decoding ability is not. Prior knowledge also influences the results, as expected. The new tool turns out to be rather independent of previous Internet experience and to measure more cognitively grounded processes related to information gathering, processing, and communicating.

AB - ORCA.IT, a new online test of online research and comprehension was developed for the Italian population. A group of 183 students attending various types of upper secondary schools in Northern Italy were tested with the new tool and underwent further cognitive and neuropsychological assessment. The different school types involved in the study are representative of the school population in the Italian system, but can also be easily compared with the educational systems of other countries. The new test turned out to have good psychometric properties after accurate item construction and final selection. In particular, Version 1 showed better characteristics than Version 2. Subsequently, comparison with one-way ANOVAs were performed to test whether differences exist between different school types, between groups with and without reading difficulties, and between males and females. Such differences are sometimes reported in the literature, but many remain controversial. Further, Pearson’s bivariate correlations were calculated to analyze associations between scores on the ORCA.IT and cognitive/neuropsychological variables. Finally, a stepwise regression analysis was performed on aggregated scores to identify the predictors of performance on each of the two versions. The test, especially in the most complete version (Version 1), appears to accurately and reliably capture students’ web searching abilities and online reading comprehension. The tool could highlight differences in online search and comprehension ability between students with and without reading difficulties, not penalizing overall performance but allowing very specific weaknesses to be pointed out. Further, it seems to be able to capture differences due to both educational pathways (different school types) and social attitudes (differences between males and females). Most interestingly, it shows to be clearly resting on specific cognitive and neuropsychological abilities, including language, memory, and attentional skills, which explain a large portion of the total variance. Offline text reading comprehension is a crucial predictor of online reading performance, while decoding ability is not. Prior knowledge also influences the results, as expected. The new tool turns out to be rather independent of previous Internet experience and to measure more cognitively grounded processes related to information gathering, processing, and communicating.

KW - assessment

KW - education effects

KW - gender effects

KW - online reading comprehension

KW - reading difficulties

UR - http://www.scopus.com/inward/record.url?scp=85075234707&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85075234707&partnerID=8YFLogxK

U2 - 10.3389/fpsyg.2019.02433

DO - 10.3389/fpsyg.2019.02433

M3 - Article

AN - SCOPUS:85075234707

VL - 10

JO - Frontiers in Psychology

JF - Frontiers in Psychology

SN - 1664-1078

M1 - 2433

ER -