MOV-AAD Dataset

A Large-Scale Multimodal Dataset for Auditory Attention During Moving Conversations

Download Dataset View on GitHub

Overview

Motivation

Auditory attention decoding (AAD) is often evaluated on static, simplified speech scenes that poorly match everyday listening conditions. MOV-AAD addresses this gap by providing a large-scale dataset designed to study auditory attention under moving, naturalistic conversations. The dataset combines 64-channel EEG with synchronized physiological recordings, enabling analysis of cross-modal neural and physiological markers of attention and listening effort in ecologically valid listening scenarios.

Overview of the MOV-AAD dataset and experimental paradigm
Figure 1: Overview of the MOV-AAD dataset and experimental paradigm.

Key Features

  • 64-channel EEG recordings from 50 participants
  • Synchronized multimodal physiological signals (eye tracking, respiration, GSR, PPG, SpO₂, temperature, motion)
  • Dynamic spatial movement of conversational speech sources (±90° azimuth)
  • Naturalistic conversational speech stimuli with behavioral attention tracking
  • Over 4,800 trials with approximately 64 minutes per participant

Applications

This dataset can be used for:

  • Robust auditory attention decoding in realistic spatial dynamics
  • Multimodal attention modeling integrating neural and autonomic measures
  • Listening effort assessment using peripheral physiological markers
  • Intersubject neural response analysis during naturalistic speech
  • Benchmarking selective auditory attention algorithms

Dataset Description

Overview Statistics

4,800
Total Trials
~64 min
Duration/Subject
2
Task Types
50
Participants

Data Composition

The dataset consists of two experimental conditions per participant:

Single Moving Speaker (40 trials): Participants listened to conversational narratives from a spatially moving source (-90° to +90°) with embedded repeated words for attention monitoring. Mean trial duration: ~40s.

Multiple Moving Speakers (56 trials): Two independent conversational streams presented simultaneously, both moving within the same azimuth range. Participants attended to one stream while ignoring the other. Mean trial duration: 44.2s (SD=2.0s).

Condition Trials/Subject Duration Percentage
Single Moving Speaker 40 ~40s/trial 42%
Multiple Moving Speakers 56 ~44s/trial 58%
Localization Task 54 sentences Short sentences -

Recording Modalities

All signals recorded at 1200 Hz with synchronized temporal alignment:

Modality Specification Description
EEG 64 channels, 1200 Hz High-density neural recordings (g.tec g.HIamp)
Eye Tracking Binocular, 1200 Hz Pupil dilation and gaze location (Tobii Pro Nano)
Respiration Flow & Effort, 1200 Hz Nasal airflow and thoracic belt
GSR 1200 Hz Galvanic skin response
PPG/Heart Rate 1200 Hz Photoplethysmography and derived HR
SpO₂ 1200 Hz Peripheral oxygen saturation
Temperature 1200 Hz Skin temperature
Accelerometer Triaxial, 1200 Hz Head motion tracking

Participant Information

  • Sample Size: 50 participants
  • Age Range: 18-38 years (Mean = 24, SD = 4.5)
  • Sex at Birth: 28 male, 22 female
  • Inclusion Criteria: Self-reported normal hearing, no history of neurological or attention disorders
  • Ethics: IRB approved, written informed consent obtained, participants compensated
  • Data Collection Period: October 2022 - June 2025

Data Structure & Download

Dataset Structure

The final dataset structure will be finalized and documented here upon release.

MOV-AAD/
├── participants/
│   ├── sub-001/
│   │   ├── eeg/
│   │   ├── physio/
│   │   ├── behavioral/
│   │   └── stimuli/
│   ├── sub-002/
│   └── ...sub-050/
├── stimuli/
├── preprocessing_scripts/
├── README.md
└── dataset_info.json

File Formats

  • EEG data: 64 channels @ 1200 Hz with 60 Hz notch filter
  • Physiological signals: 1200 Hz sampling rate
  • Audio recordings: Synchronized playback recordings for temporal alignment
  • Behavioral data: Event markers and timestamps
  • Spatial information: Azimuth trajectories

Download Instructions

📦 Via Zenodo

Download the complete dataset from Zenodo:

Download from Zenodo (Available Upon Publication)

DOI: [Will be added upon publication]

Dataset will be released upon paper acceptance

💻 Via Command Line

Download using wget or curl (available after publication):

# Using wget
wget [ZENODO_URL]

# Using curl
curl -O [ZENODO_URL]

# Extract
unzip MOV-AAD-dataset.zip

🐍 Via Python

Example code for loading the dataset:

# Example loading code will be provided
# upon dataset release with full documentation

Preprocessing Notes

  • EEG channels with abnormal variance are automatically detected and replaced via spherical interpolation
  • 3s pre-onset baseline segment included for each trial
  • Temporal alignment established via cross-correlation of recorded audio with original stimulus
  • Peripheral modalities provided in minimally processed form (1200 Hz with hardware notch filtering)
  • Preprocessing scripts will be provided for reproducibility

License

License information will be specified upon dataset release.

Baseline Results

Experimental Setup

We provide baseline results using standard decoding approaches to facilitate comparison with future work. All analyses use leave-one-trial-out cross-validation.

Models Evaluated

  • Envelope Reconstruction: Backward model (mTRF Toolbox v2.3) with 1-8 Hz band-pass filtered EEG, 0-400ms time lags
  • Trajectory Reconstruction: Delta band (0.05-2 Hz) and Alpha band (8-12 Hz) spatial trajectory decoding, 0-250ms lags
  • Auditory Attention Decoding (AAD): Linear Discriminant Analysis (LDA) using reconstruction correlations as features
  • Intersubject Correlation (ISC): Correlated component analysis across participants for stimulus-locked responses