Accessibility statement

Multimodality: Language & the Body - LAN00066H

« Back to module search

  • Department: Language and Linguistic Science
  • Module co-ordinator: Dr. Kobin Kendrick
  • Credit value: 20 credits
  • Credit level: H
  • Academic year of delivery: 2022-23

Module summary

Most of our language use and acquisition happens through face-to-face interaction, which gives us access not only to what we and others say, but also how we use our bodies in interacting with others. A growing area of research explores how language is connected to gesture, facial expression and eye gaze: how are multiple channels of information like lexical choice, syntactic structure, intonational structure, and other aspects of linguistic production connected to the use of the body as we talk? How do we interact with one another and with objects in a physical world? How does this shape our language, and how does our language reflect the demands of being human beings in bodies, and in a physical world? In this module, we explore language as a multimodal phenomenon, grounding our work in everyday interactions, and using the methodological resources of multimodal conversation analysis and gesture studies.

Related modules

Pre-requisite modules

Co-requisite modules

  • None

Prohibited combinations

  • None

Module will run

Occurrence Teaching period
A Autumn Term 2022-23

Module aims

Most of our language use and acquisition happens through face-to-face interaction, which gives us access not only to what we and others say, but also how we use our bodies in interacting with others. A growing area of research explores how language is connected to gesture, facial expression and eye gaze: how are multiple channels of information like lexical choice, syntactic structure, intonational structure, and other aspects of linguistic production connected to the use of the body as we talk? How do we interact with one another and with objects in a physical world? How does this shape our language, and how does our language reflect the demands of being human beings in bodies, and in a physical world? In this module, we explore language as a multimodal phenomenon, grounding our work in everyday interactions, and using the methodological resources of multimodal conversation analysis and gesture studies.

Module learning outcomes

At the end of this module you will be able to:

  • Recognise the different modalities that are relevant for language use in spoken face-to-face interaction, including: facial expression, manual gestures, phonetic production, eye gaze
  • Use appropriate software to study visible bodily actions, including gesture and speech
  • Be able to annotate and transcribe spoken and visible/gestural behaviours using standard transcription methods
  • Write about a problem in face-to-face interaction, drawing on technical skills, and showing an appreciation of some of the debates around the topic

You will know:

  • Various ways in which speakers use language and the body together as resources for meaning and action in face-to-face interaction
  • Major theories developed to account for specific modalities such as facial expressions, manual gesture, speech and eye gaze
  • Important empirical studies conducted on specific modalities and their coordination with language production

Module content

Practical stream

The practicals will cover the following topics:

  • Annotation of data in ELAN (software for the analysis and annotation of video data)
  • Making suitable data extracts
  • Making collections of comparable extracts for the purposes of analysis
  • Practicalities and house-keeping while working with digital data: making video clips, file formats, saving data, etc.
  • Transcription of data for presentation, using standard schemes for conversational data 

Content

The first half of the module will cover important topics in the analysis of face-to-face spoken interaction. The precise content will vary from year to year, but is likely to include:
1. Foundations of multimodality (including the relations between spoken and signed languages, and the place of gesture)
2. Gaze: its alignment to other events in interaction, its use in turn construction and the formation of social actions
3. Manual gesture: its alignment to other events in interaction, its internal structure, and its relation to language production and processing
4. Facial expression, including relations with prosodic features in the construction of turns at talk

The second half of the module will focus on a particular multimodal practice, such as pointing while producing a turn at talk, that will vary from year to year. 

Assessment

Task Length % of module mark
Essay/coursework
Essay 3000 words		 N/A 75
Essay/coursework
Essay 750 words		 N/A 25

Special assessment rules

None

Reassessment

Task Length % of module mark
Essay/coursework
Essay 3000 words		 N/A 75
Essay/coursework
Essay 750 words		 N/A 25

Module feedback

Oral feedback will be provided in class and individual meetings. Written feedback will be provided for each essay within twenty working days.

Indicative reading

Kendon, A. (2004). Gesture. Visible Action as Utterance. Cambridge: Cambridge University Press.

Mondada, L. (2016). Challenges of multimodality: Language and the body in social interaction. Journal of Sociolinguistics, 20(3), 336–366. http://doi.org/10.1111/josl.1_12177

Perniss, Pamela (2018). Why We Should Study Multimodal Language. Frontiers in Psychology 9:1109. http://doi.org/10.3389/fpsyg.2018.01109 


The information on this page is indicative of the module that is currently on offer. The University is constantly exploring ways to enhance and improve its degree programmes and therefore reserves the right to make variations to the content and method of delivery of modules, and to discontinue modules, if such action is reasonably considered to be necessary by the University. Where appropriate, the University will notify and consult with affected students in advance about any changes that are required in line with the University's policy on the Approval of Modifications to Existing Taught Programmes of Study.