Multimodal Metaphor and Expressive Movement
(2009-2013)
The project aimed at developing an interdisciplinary theory of expressive movements as well as a methodology that allows for their reconstruction. It brought together a linguistic take on reconstructing the dynamics of attention and affect through the use of multimodal metaphors in talk (Müller) with a film-analytic model of expressive movements and 'sensation-images' (Empfindungsbilder) (Kappelhoff).
As far as multimodally expressed metaphors (speech and gesture) in spoken discourse are concerned, we propose that they imply a dynamic activation and construction of cognitive and affective experience and that they constitute specific forms of mundane expressive movements. Regarding the orchestration of multimodal metaphors in film (orchestration of space, image, sound), we assume that they organize a specific form of cinematographic expressive movement, which play a core role in the recipients' dynamically structured orientation of affect.
The project carried out a comparative empirical study of metaphorically organized expressive movements in face-to-face communication, in news and sport reports, in German TV series, in contemporary German Cinema and in Classical Hollywood Cinema (Hitchcock, Film noir). The analysis of forms of multimodal metaphoric orchestration of expressive movements, as well as their distribution over the different types of audiovisually recorded data, served as point of anchorage for both: the development of a methodology and a theory of expressive movements – a theory which transcends the concept of affect as a symptomatic expression of inner states.
How two experiential realms are dynamically emerging and mapped onto each other in time (13 second extract from a German TV news report, taken from the Tagesschau, ARD, 20.10.2008, 8:00-08:15 pm, English translation)
Head of Project:
Hermann Kappelhoff (1)
Cornelia Müller (2)
Research assistants:
Sarah Greifenstein (1) (2009-2013)
Christina Schmitt (1) (2009-2013)
Dorothea Horst (2) (2012-2013)
Franziska Boll (2) (2012-2013)
Stefan Rook (2) (2009-2011)
Susanne Tag (2) (2009-2011)
Student assistents:
Thomas Scherer (1) (2009-2013)
Dorothea Horst (2) (2009-2011)
Franziska Boll (2) (2011-2012)
(1): Film studies, Freie Universität/Berlin
(2): Linguistics, European University Viadrina/Frankfurt (O.)
Publications
Horst, Dorothea/Boll, Franziska/Schmitt, Christina/Müller, Cornelia (2014): Gesture as interactive expressive movement: Inter-affectivity in face-to-face communication. In: Body – Language – Communication: An International Handbook on Multimodality in Human Interaction, Vol. 38.2. Cornelia Müller, Alan Cienki, Ellen Fricke, et al. (eds). Berlin, New York: De Gruyter Mouton, pp. 2112-2125.
Greifenstein, Sarah/Kappelhoff, Hermann (2014): The discovery of the acting body. In: Body – Language – Communication: An international handbook on multimodality in human interaction, Vol. 38.2. Cornelia Müller, Alan Cienki, Ellen Fricke, et al. (eds). Berlin, New York: De Gruyter Mouton, pp. 2070-2080.
Kappelhoff, Hermann/Greifenstein, Sarah (2014): Feeling Gloomy or Riding High: Timings of Melodrama and Comedy. In: Timing of Affect. Epistemologies, Aesthetics, Politics. Marie-Luise Angerer, Bernd Bösel and Michaela Ott Ott (eds). Zürich/Berlin: diaphanes, pp. 263-282.
Kappelhoff, Hermann/Müller, Cornelia (2011): Embodied meaning construction. Multimodal metaphor and expressive movement in speech, gesture, and feature film. In: Metaphor and the Social World 1.2, pp. 121-153.
Kappelhoff, Hermann/Bakels, Jan-Hendrik (2011): Das Zuschauergefühl. Möglichkeiten qualitativer Medienanalyse. In: Zeitschrift für Medienwissenschaft 5.2, pp. 78-95.
Müller, Cornelia/Tag, Susanne (2010): The dynamics of metaphor. Foregrounding and activation of metaphoricity in conversational interaction. In: Cognitive Semiotics 6, pp. 85-120.
Müller, Cornelia (2011): Reaction paper. Are ‘deliberate’ metaphors really deliberate. A question of human consciousness and action. In: Metaphor and the Social World (1).
Müller, Cornelia (2013): Gestures as a medium of expression: The linguistic potential of gestures. In: Body – Language – Communication: An international handbook on multimodality in human interaction (= HSK 38.1). Edited by Cornelia Müller, Alan Cienki, Ellen Fricke et al. Berlin, New York: De Gruyter Mouton.
Scherer, Thomas/Greifenstein, Sarah/Kappelhoff, Hermann (2014): Expressive movements in audiovisual media: Modulating affective experience In: Body – Language – Communication. An international handbook on multimodality in human interaction, Vol. 38.2. Cornelia Müller, Alan Cienki, Ellen Fricke, et al. (eds). Berlin, New York: De Gruyter Mouton, pp. 2081-2092.
Schmitt, Christina/Greifenstein, Sarah (2014): Cinematic communication and embodiment. In: Body – Language – Communication: An International Handbook on Multimodality in Human Interaction. Cornelia Müller, Alan Cienki, Ellen Fricke, et al. (eds). Berlin, New York: De Gruyter Mouton, pp. 2061-2070.
Schmitt, Christina/Greifenstein, Sarah/Kappelhoff, Hermann (2014): Expressive movement and metaphoric meaning making in audio-visual media. In: Body – Language – Communication: An international handbook on multimodality in human interaction, Vol. 38.2. Cornelia Müller, Alan Cienki, Ellen Fricke, et al. (eds). Berlin, New York: De Gruyter Mouton, pp. 2092-2112.
