Austin: “ Gestures ” 2002Do listener’s facial expressions influence speaker’s discourse ?

Claire Maury-Rouan
CNRS, UMR 6057, Université de Provence, (France)

1. Introduction

Although a lot has been written on verbal and on nonverbal behavior taken separately, as Lee and Beattie (1998) pointed out, “there have been relatively few attempts to describe the complex inter-relationships between verbal and non-verbal behavior” (1998: 51). Ch. Goodwin (1981), studying the relationship of gaze behavior to conversational organization and turn-taking negotiation by interactants, found that addressee’s gaze towards speaker was a necessary basis in hearership, and should be brought to the speaker before the speaker’s gaze arrived at the addressee. If this was not the case, dysfluencies might occur, often with the effect of bringing the addressee’s attention back to the speaker. Coherently with Kendon’s 1967 and S.D. Duncan (1977) findings, Beattie (1983) demonstrated that gaze movements were also related to planning periods: monitoring interactant’s behavior (looking at addressee) and planning speech (looking away) were cognitively incompatible activities: looking at addressee while planning speech resulted in dysfluencies and increasing difficulties. Rather than questioning the link to cognitive activities or to the turn-taking process, my study focuses on the relationship of speakers’ and listeners’ nonverbal behavior to the discourse strategies speakers develop, analyzed from an interactional point of view (Maury-Rouan 1998, Maury-Rouan 2001a, Maury-Rouan 2001b).
As J. Cosnier (2000) puts it: being a “happy speaker” implies being able to make one’s ideas explicit, understood, and moreover, to be approved, to share a point of view, to have an action performed, to convince others. Consistent with the findings of the above mentioned authors, Cosnier explains that much of this type of information reaches the speaker via the addressee’ s gaze direction and facial expressions (smiling, frowning, etc.) and suggests that such information might strongly influence a speaker’s discourse strategies. In line with Cosnier’s views, my study will focus on two main discourse strategies: modulation and hypocorrection, and the conditions in which they appear in a speaker’s discourse. I hypothesize that their presence cannot solely be accounted for by the needs of discourse content, or by listener’s audible reactions (mainly: mms and hms ). I would like to check whether nonverbal events: (1) speaker’s and addressee’s gaze behavior, and (2) synchronization of addressee’s facial expressions with content of speaker’s discourse might cause modulation or hypocorrection to appear.

2. Modulation, hypocorrection, vs. hesitation patterns.

Vion (1992) introduced the concept of modulation to describe strategies used by the speaker to soften his/her own words, avoiding assertive discourse as a possible threat to the recipient’s face.

E.g.: "this movie is excellent"
         Vs: (modulation) "I kind of liked it, though, in a way"

Performed through various linguistic forms – including modalizations , hedges
(as defined by Lakoff (1972) or polyphony, modulation generally speaking operates by establishing distance between speaker and utterance. Modulation belongs to the category of reflexive adaptation of one’s own speech but is specifically recipient-oriented as an interactive strategy.
Another strategy, hypocorrection (Maury-Rouan 1998) has similar interactive goals as it also works as a softener. Hypocorrection consists in an attempt to give one’s discourse a clumsy, colloquial, or even broken or dysfluent style, when introducing clever or innovating statements or ideas. By and large, hypocorrection allows the speaker, by toning down a potential flattering image of self, to avoid sounding pretentious or pedantic, thus reducing the risk of threat to recepient(s) face.

E.g. : The balance between East and West used to be very reassuring; we knew there was an East block”
V.s. (hypocorrection): The balance between East and West sort of ah:::: t’was like very ah::::(1.65) reassuring + ya’ know + we knew + well there were ++ there was a We+East block")

Hypocorrection may be carried out by slang, as well as by fuzzy use of connective particles (quoi, bon), resulting into a type of discourse that portrays oral style, to a degree that sometimes includes dysfluency patterns, as can be noticed in the example given above. Discriminating hypocorrection from genuine hesitation in such contexts can prove to be quite tricky.

3. Methods.

3.1 Multimodal corpus.
Analysis was based on 20 videotaped dyads. Subjects were expected to do their best to discuss a topic for 10 minutes. They sat facing each other, filmed by 3 cameras simultaneously. Two close-shot films were assembled into one synchronized split-screen, enabling us to analyze coordination of interactants’ behaviors.

3.2 Analyzing data.
A first transcription from the audio track only was analyzed, looking for clues of the presence of various discourse strategies on the basis of verbal and vocal parameters only.
In a second stage, related nonverbal behavior (gaze direction, facial expressions and head nods mainly) was taken into account, in order to check whether visual data matched previous verbal-based hypothesis. This divided approach of data was designed as an attempt to minimize the risk of circularity.

4. Findings.

4. 1. Addressee’s inadequate gaze behavior and facial expression does seem to induce destabilization of speaker.
The recepient’s uncooperative nonverbal behavior ([-gaze] , skeptical facial expression..) destabilizes the speaker, resulting into phases of stuttering: pour + pour + pourquoi (why +why + why). filled or empty pauses, as well as modulations : quoi (y’know) , tu vois ce que je veux dire(see what I mean).

4. 2. Head nods don’t seem to be sufficiently convincing as means of approval. Facial expression is more crucial evidence to the speaker as to the certainty of being approved.
Although clearly interactive, echoing nods are ordinarily beneath the level of conscious recall (McClave 1998: 367). Similarly, listeners head nods in my data seem to be induced almost automatically as a response to stressed syllables, or to support-seeking particles such as quoi (y’know) in speakers discourse: they are present even when the addressee does not agree at all with the speaker’s opinion. This appears in the addressee’s behavior, as he regularly nods at each crucial point, while producing repeated turn-taking efforts in order to object to the opinion being sustained.So head-nods do not compensate for lack of verbal support and an unexpressive face, as appears from repeated modulation efforts in the speaker’s discourse when such a situation occurs.

4. 3. When a speaker is experiencing actual speech planning difficulties, a certain type of discourse particles can actually operate as floor-holders, which sheds an unusual light over the functional value of such particles.
In some interactive situations, the speaker operates by another kind of semi-conscious strategy. While averted gaze apparently reveals genuine planning difficulties, speakers seem to be able to resort - more or less consciously - to what could be called discourse lures (Maury-Rouan 2001b).
The addressee is lured by connective particles such as donc, enfin, en fait literally: therefore, finally, in fact, into expecting a logical development of some sort (announced by apparently constructive connective particles) and thus remains patient and tolerant enough not to interrupt the hesitant utterance.

4. 4. Conversely, interactants may display hypocorrection.
Seemingly dysfluent patterns could be identified as cases of hypocorrection, because the speaker remained [+gaze] while displaying filled pauses, lexical repairs or stuttering patterns. Hypocorrection might be considered as a “strategy “operating as a softening act. 5.

Final Comments.

The analysis by and large confirmed initial hypothesis on the coordination of nonverbal regulation and discourse structure. However, it seems clear that the influence of gaze patterns and facial expression over the speaker’s strategy is at its height in the first 1-2 minutes of this type of interaction., when participants meet for the first time.
Another finding is that verbal features and nonverbal features are not always what they seem to be. A connective particle for instance may function as floor-holders; make-believe hesitations and colloquial language may work as affiliative strategies (softeners) etc.
It remains very difficult though to determine to what extent we are dealing with actual strategies, the degree of awareness of the behavior we described being unclear, though nonverbal clues (gestures and gaze behavior mainly) may prove to be helpful. But then also, maybe speakers are able to communicate without being clearly aware of every aspect of their actions, and a bit of intrinsic fuzziness may prove to be one of the conditions of speaker’s happiness.


Cosnier J., (2000):"La voix, les gestes, le corps", in:Tu parles ? Le Français dans tous ses états, Paris, Flammarion 325-340.
Duncan S.D. (1977): Face-to-face Interaction: Research, Methods and Theory. Hillsdale N.J. ; Lawrence Erlbaum Associates.
Goodwin, C. (1981): Conversational Organization . New York Academic Press.
Kendon, A. (1967): "Some functions of gaze direction in social interaction". Acta Psychologica, 26(1), 1-47.
Lakoff G. (1972) : “Hedges : A Study of Meaning Criteria and the Logic of Fuzzy Concepts” Papers for the Eighth Regional Meeting, Chicago Linguistic Society : 183-228.
Lee, V. & Beattie, G. (1998): " The Rhetorical Organization of Verbal and Nonverbal Behavior in Emotion Talk ". Semiotica 120-1/2, 39-92.
Maury-Rouan, C.(2000):"L’hypo-correction: entre sociolinguistique et analyse linguistique des interaction", in: Lengua, Discurso, Texto. Madrid, Visor Libros 1627-1638.
Maury-Rouan, C.(2001a):"Mimiques, regards et activités discursives" in: G.Barrier et N. Pignier (eds) Sémiotiques non verbales et modèles de spatialité, Limoges, Presses Universitaires de Limoges (sous presses).
Maury-Rouan, C.(2001b):"Le flou des marques discursives est-il un inconvénient ? vers la notion de ‘leurre discursif’ " Revue on-line Marges linguistiques (http://www. 195-208.
Mc Clave E.(1998) : Cognitive and Interactional Functions of Head Movements in Conversation, in: Santi S., Guaitella I. Cavé C. et Konopczynski G. (eds) Oralité et gestualité, L’Harmattan, Paris,365-370.
Seyfeddinipur M. et Kita S. (2001):"Gestures and dysfluencies in speech", in: Cavé C., Guaitella I., Santi S. Oralité et gestualité, Paris, L’Harmattan, 266-270.
Vion R. (1992) La Communication Verbale. Analyse des interactions. Paris, Hachette.