Background
We recorded 13 EEG hyperscanning experiments in which groups of three acquainted people held 12 conversational trials each, spanning free speech, consensus finding, and conflict conditions. Alongside 96-channel EEG at 2.5 kHz and multimodal video, audio was captured via three directional microphones and one central omnidirectional microphone.
Our initial goal is Diarization (to detect who spoke when), and we then use this separated data for each participant to find connections between their EEG, their feedback on the conversation, and their voice signals. Voice quality, prosodic shifts, pitch contours, and vocal intensity carry rich information about dominance, emotional arousal, and conversational alignment. In a triad, these acoustic features interact dynamically: speakers adapt their pitch and volume to claim the floor, mirror each other during consensus, or diverge during conflict. We have the data and the synchronization; we need to find the vocal dimension of these interactions and tie them directly to our existing neural and behavioral metrics.
*This project is being done in collaboration with the Neurotech team.
Objective
Extract advanced acoustic features (such as pitch, formants, vocal energy contours, and speech rate metrics) from the audio recordings, refine speaker diarization using these acoustic properties, and relate these vocal dynamics to the EEG metrics and behavioural ratings already established in our pipeline.
Requirements
- Solid Python programming skills; familiarity with (interest in) audio processing libraries (e.g.,
librosa,torchaudio) is a clear advantage. - Background or strong interest in digital signal processing, acoustic feature extraction, and speech analysis.
- Comfort with statistical modeling or willingness to learn how to relate continuous time-series data to neural and behavioral outcomes.
Resources
A fully recorded and synchronized multimodal dataset, an established EEG analysis pipeline with metrics already computed, GPU compute, and a direct line of sight to publication.
Supervisors
Please send me an email (amirreza.asemanrafat@fau.de) with your resume and transcripts to apply for the topic. We will then get in contact with you if we are interested.
Please send me an email (amirreza.asemanrafat@fau.de) with your resume and transcripts to apply for the topic. We will then get in contact with you if we are interested.