Recognizing Cognitive Load by a Hybrid Spatio-Temporal Causal Model from Multivariate Physiological Data

Publication Name

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)


Cognitive load recognition is challenging due to the inherent diversity and causality of multivariate physiological changes, with each of its instances having its own style of physiological events and their spatio-temporal causal dependencies. This leads us to define a hybrid model that employs Granger causality (GC) and Gramian angular difference fields (GADF) to discover diverse varieties of multivariate physiological events. In particular, our model introduces a GC network to explicitly characterize the unique temporal causal configurations of a particular cognitive state as a variable number of nodes and links. In addition, GADF maps are constructed to capture the inherit spatio-temporal dependency among multivariate signals in a 2D structural space. A capsule network is designed to merge these two heterogenous types of features together in a uniform way, and as a result, all local causal and spatio-temporal dependencies are globally consistent. Empirical evaluations on one benchmark dataset and two in-house datasets collected by ourselves in virtual reality learning environment suggest our model significantly outperforms the state-of-the-art approaches.

Open Access Status

This publication is not available as open access


13718 LNAI

First Page


Last Page


Funding Number


Funding Sponsor

National Natural Science Foundation of China



Link to publisher version (DOI)