Despite the considerable body of research on gestures, some open questions remain concerning speech–gesture integration in language processing. Turning to reception, several studies have demonstrated that addressees understand messages better if they are accompanied by gestures (e.g., Kelly et al., 1999 Rogers, 1978), and conversely, that comprehension is negatively affected if information across modalities is contradictory (e.g., Cassell, McNeill, & McCullough, 1999).
However, given that gestures often express ‘imagistic’ information complementing or illustrating the verbally expressed meaning of an utterance as a whole, in its communicative context, the notion of ‘conceptual affiliates’ has been suggested instead (De Ruiter, 2000 see also McNeill, 2005, p. The semantic relationship between speech and gesture has been characterized in terms of ‘lexical affiliates’ (Schegloff, 1984) to denote the word or words whose meaning correspond to that expressed in gestures. Moreover, speakers gesticulate significantly less during speech disfluencies or pauses than during fluent speech (Graziano & Gullberg, 2018 McNeill, 1985 McNeill, 2005, pp. Speech and gestures co-occur and gesturing in the absence of speech is rare in normal conversation. The connection is reflected in the close temporal and semantic coordination between gestures and spoken utterances found in language production (Kendon, 2004 for an overview) whereby gestures and speech tend to express closely related meaning at the same time. Speech and gestures are seen as forming an integrated whole, where both parts are relevant. Proponents of contemporary gesture theories generally agree on the tight link between speech, language and gestures despite theoretical divides as to the precise nature of the link (De Ruiter, 1998 Hostetter & Alibali, 2008 Kelly, Özyürek, & Maris, 2009 Kita, Alibali & Chu, 2017 Krauss, Chen, & Gottman, 2000 McNeill, 2005).
Gestures, defined as (mostly manual) movements related to the expressive effort and recognized as being communicatively relevant (Kendon, 2004 McNeill, 1992), are prevalent in communication (face-to-face or in groups, e.g., Kendon, 2004 Özyürek, 2002), whether interlocutors are visible or not (Bavelas, Gerwing, Sutton, & Prevost, 2008). Gestures are an integral part of natural language use.