MMX

The tutorial will be relevant to experienced researchers as well as students new to the field. To the former it will serve as an update and refresher with respect to current practices and methodologies, and to the latter it will provide a compact and easy-to-digest overview of this emerging new field of speech processing. Depending on the time allocated to this tutorial (which will preferably be a full-day but could be compressed to just a half day), the two key presenters will be joined by other invited experts in the various sub-fields to provide a broader coverage of the content and wider viewpoints for discussion.

Interaction Technology is currently veering towards the processing of spontaneous interactive speech and in doing so is necessarily incorporating information about the gestures and bodily movements alongside the sounds of speech for processing the communication of interpersonal as well as propositional information. This tutorial will present an overview of the state of the art in this area as well as introducing tools and techniques for the automatic processing of multimodal discourse-related data. It will review the presently-available multimodal corpora and show how they are being used in the analysis and modelling of prosodies and non-verbal behaviour. It will emphasise the importance of prosodic and paralinguistic information in the processing of human conversational interactions, above and beyond the propositional information contained in the speech. It will also show how the use of bodily gestures, eye-gaze, and non-speech vocalisations can combine to show a speaker's attentive states and discourse intentions in a way that can be processed alongside the 'text' of the dialogue to provide a rich source of interaction information.