ESSV Konferenz Elektronische Sprachsignalverarbeitung

Title: A Head-Mounted Camera System for the Measurement of Lip Protrusion and Opening during Speech Production

Authors: Fabian Klause, Simon Stone, Peter Birkholz


In this paper, we present a head-mounted lip observing camera system and the downstream processing steps and tools needed to obtain protrusion and opening of a subject’s lips during speech production. The helmet-like camera system consists of two consumer market cameras mounted directly to a subject’s head to keep their positions constant with respect to the skull, thus no post processing image stabilization or feature tracking is needed. The mounting system is built from aluminum pipe segments and 3D-printed hinges and joints. The cameras capture the profile and frontal view of the subject’s lips. In the two obtained image sequences, the lip contours are detected using the segmentation software Glottal- ImageExplorer. In these two 1-Bit lip contour images, several extrema are detected and converted to the desired parameters protrusion and opening. Exemplary results are presented for selected vowel-consonant-vowel sequences. All necessary files for the construction of the camera helmet and the extraction of the parameters are provided free of charge at birkholz-supplements. The video capturing and post-processing exclusively uses a chain of freeware tools to make the system available to the speech research community.

Year: 2017
In session: Poster
Pages: 145 to 151