Background
We recorded 13 EEG hyperscanning experiments in which groups of three people who know each other held free conversations. Each experiment contains 12 trials spanning three conversation types (free speech, consensus finding, and conflict). While they talked we recorded 96-channel EEG at 2.5 kHz and audio at 44.1 kHz.
The audio setup is the problem this thesis addresses. Each participant has a directional microphone placed 2 m away, with 120 degrees between neighbouring microphone axes, and a single omnidirectional microphone sits at the centre of the triangle. Directionality is not enough at this distance: every channel contains substantial speech bleed from the other two participants.
This leaks into the neuroscience. Speaker diarization currently relies on energy gating of the directional channels, so bleed produces false speaking intervals, which in turn corrupts the listening segments on which all our neural metrics are computed. Worse, the temporal response function (TRF), which measures how strongly a listener’s brain tracks speech, is currently estimated from the omnidirectional envelope. That envelope is a mixture of all three talkers, so we cannot presently ask the question we actually care about: does a listener’s cortex track the person they are listening to?
A clean per-speaker audio stream would fix all three of these at once.
* This project is being done in collaboration with the Neurotech team.
Objective
Implement, validate, and deploy a source separation algorithm that recovers three individual speech streams from our four-channel recordings.
Requirements
- Solid Python; PyTorch experience is a clear advantage.
- Background in digital signal processing (filtering, spectral analysis, and ideally array processing).
- Interest in working with real, imperfect experimental data.
- German language skills are helpful for judging separation quality by ear but are not required.
Resources
Access to a fully recorded multimodal dataset, a working analysis codebase, and GPU compute. Positive outcomes will be considered for publication.
Supervisors
Please send me an email (amirreza.asemanrafat@fau.de) with your resume and transcripts to apply for the topic. We will then get in contact with you if we are interested.
Please send me an email (amirreza.asemanrafat@fau.de) with your resume and transcripts to apply for the topic. We will then get in contact with you if we are interested.