Language Development From an Ecological Perspective: Ecologically Valid Ways to Abstract Symbols

Language Development From an Ecological Perspective: Ecologically Valid Ways to Abstract Symbols Joanna Rączaszek-Leonardia, Iris Nomikoub, Katharina J. Rohlfingc, and Terrence W. Deacond a Faculty of Psychology, University of Warsaw; b Department of Psychology, c University of Portsmouth; Faculty of Arts and Humanities, Paderborn d University; Department of Anthropology, University of California at Berkeley Paper published as: Rączaszek-Leonardi, J., Nomikou, I., Rohlfing, K. J. & Deacon, T. W. (2018). Language Development From an Ecological Perspective: Ecologically Valid Ways to Abstract Symbols. Ecological Psychology, 30:1, 39-73, DOI: 10.1080/10407413.2017.1410387 ABSTRACT In the embodied, situated, enacted and distributed approaches to cognition, the coordinative role of language comes to the fore. Language, with its symbolic properties, arises from a multimodal stream of interactive events and gradually gains power to constrain them in a functional and adaptive way. In this article, we attempt to integrate three approaches to information in cognitive systems to provide a theoretical background to the process of development of language as such a coordinator. Ecological psychology provides an explanation for how any behaviors or events become informative through the process of “tuning” to affordances that control individual and collective behavior. The dynamical approach helps to operationalize this control as a functional reduction of degrees of freedom of individual and collective systems. Cognitive semiotics provides a typology of constraints showing their interrelations: it proposes conditions under which informational controls that function as indices and icons may become symbolic, providing a qualitatively different form of constraint, which can be partially ungrounded from the ongoing stream of multimodal events. The article illustrates the proposed processes with examples from actual parent-infant interaction and points to ways of verifying them in a more quantitative way. 2 The recent “ecological turn” in the cognitive sciences acknowledges the primacy of action, situated in a particular social and physical environment, as determining the emergence of cognitive skills (e.g., Gallagher, 2005; A. D.Wilson & Golonka, 2013; M. Wilson, 2002). Because embodiment, action- dependency, and distributed character pertain not only to motor coordination but also to such processes as thinking and language, their coordinative and interpersonal dimensions come to the fore. On this view, language, rather than being treated as a cognitive module or individual computational skill, is seen primarily as a system of constraints, which emerges in co-action in a particular physical and cultural environment and which has the power to control individual cognition and interindividual coordination (Rączaszek-Leonardi, 2009; Sinha, 2009; Steffensen & Fill, 2014). These changes in perspective are particularly important for the study of language development. First, they change the way we think about language development itself: not as a child “cracking the linguistic code,” that is, mastering an individual linguistic skill by acquiring (or parameterizing inborn) rules, but rather as a gradual tuning process adapting the child to the way language functions in social encounters, shaping everyday interactions from day one (Bruner, 1983; Rączaszek-Leonardi, Nomikou, & Rohlfing, 2013). Second, language development becomes a particularly useful window for developing an account of how language, as a system of symbols, may emerge from embodied multimodal interactions with others. This is because, especially in early development, it is acutely evident that early uses of language are fully grounded in streams of dynamical individual and interactive events. Yet, on the other hand, language has the undeniable capacity of removing us from the here and now, evoking abstract relationships as well as nonpresent or even nonexistent entities. Due to its compositional structure, it also has its own syntactic combinatorial properties, and, crucially, its elements 3 and structures seem to have a degree of “arbitrariness” and conventionality with respect to how linguistic forms relate to their semantic and communicative functions because the clues to these functions are usually missing from word sounds (Deacon, 2011; Peirce, 1931). Because language is initially acquired within the aforementioned embodied and grounded social communication context, the acquisition of such an “ungrounded” capacity to control communicative events over and above relating them to the immediately present context of action requires an explanation. In this respect, we see language acquisition as an inverse to Harnad’s symbol grounding problem (Harnad, 1990; Rączaszek-Leonardi & Deacon in preparation; Deacon (2012, in press)). Whereas Harnad, and many before him (e.g., Dreyfus, 1972; Searle, 1980), saw a problem in how abstract symbols get their meaning, that is, how they are grounded in the world, our problem is the opposite: how concrete physical events or objects, embedded causally in dynamical interactions, may ever become abstract and symbolic. Posed in this way, the problem is quite similar in its core, we claim, to the more general problem that ecological psychology has been facing: how to account for the nature and function of symbolic information from its exclusively dynamics- oriented perspective (e.g., Pattee, 1982; Reed, 1996; Rączaszek-Leonardi & Kelso, 2008). Formulated more specifically in the context of language development, posing the question in this way calls for a change in the starting assumptions. Assuming that linguistic forms present in the infant’s environment are intrinsically symbolic (i.e., are conventional, arbitrary, and have formal- systemic properties) in a sense creates the grounding problem, which requires explaining how the child learns about this symbolicity and grounds it semantically and pragmatically (Harnad, 1990; Varshavskaya, 2002). Assuming instead that linguistic utterances are initially immersed in dynamic interactional 4 events (Bruner, 1983; Lock, 1978; Zukow-Goldring, 1996), similar to any other action or gesture, and are “just” one type of such events, leads to the “ungrounding” problem, namely how, for a developing child, do these grounded forms ever gain symbolic properties? We propose to validate this approach to the symbol emergence problem in language development by illustrating the processes that are crucial for the “ungrounding” process using naturalistic examples from language development studies. The theoretical exposition is thus necessarily brief (a more detailed account can be found in Rączaszek-Leonardi & Deacon, in preparation), shortly introducing the three theoretical pillars on which we base our account of language development: ecological psychology supported by dynamical systems approach and cognitive semiotics. Next, from this integrated theoretical perspective, we describe the informational processes of interaction control and the ways language might be involved in them. This paves the way to a proposed process of ungrounding of some of the informational forms from the immediate context. We illustrate each element of this process with concrete examples from microanalyses of everyday infant-caretaker interactions, provided by longitudinal video corpora (Nomikou & Rohlfing, 2011; Szufnarowska & Rohlfing, 2014) and from the language development literature. Where possible, we point to the ways of verifying the proposed claims quantitatively and point to work in which this is already being done. Brief theoretical background We construct our account of language development on a foundation consisting of three theoretical approaches: ecological psychology, dynamical systems account of information in biological systems, and the semiotic approach to signification. We think that each of these frameworks contributes a crucial piece of the puzzle of language emergence in development and that their integration sheds light on how these frameworks might complement each other 5 in the effort to solve the ungrounding problem. Situating the explanation within ecological psychology has a twofold theoretical goal. First, ecological psychology recognizes the importance of the relation between an agent and the environment and spells out clearly how certain events and their parameters in the environment might become intentional and informative so that they provide functional constraints on behavior (Gibson, 1979/1986; Heft, 2001). Broadly speaking (for a more detailed account see Rączaszek-Leonardi, 2016), ecological psychology provides a framework that accounts for the ubiquitous embodiment of cognition by showing how certain forms (such as repetitive behaviors or sequences of events) become informative through their particular history in the organism- environment relations on multiple timescales. An important aspect of this framework is also its involvement of values as boundary conditions in the processes of perceiving and development of affordances (Hodges & Baron, 1992; Rączaszek-Leonardi & Nomikou, 2015). Second, the ecological psychology approach can be extended by providing an ecologically valid account of how some of those informational forms become symbolic. Our core assumption is that there is a parallel between a “symbol ungrounding problem” in language development and the difficulties that ecological psychology encounters attempting to account for the capacity of symbolic cognition, including language. We propose that integration with the other two approaches, dynamical systems account of information and cognitive semiotics, can bring us a step closer to dealing with this difficulty by leveraging principles of informational processes in ecological psychology to help explain symbolic functioning of humans. The contribution of the dynamical systems account is to provide a conceptualization of informational processes in living organisms as emergent 6 stabilization of selective constraints on dynamics1. The argument has been developed in many works (e.g., Deacon, 2011; Pattee, 1969, 1982; Pattee & Rączaszek-Leonardi, 2012; Polanyi, 1968; Rączaszek-Leonardi & Kelso, 2008) and for the space consideration we cannot present it at length. In short, it advocates the necessity of complementary descriptions to account for the functioning and adaptability of organisms both in terms of natural dynamics and informational forms. In contrast to most cognitive science approaches to symbols, this approach does not treat informational forms as amodal abstractions but rather as physical entities that act as constraints on organ- ism dynamics. Due to their historicity, and the irreversibility of the evolutionary processes that selected both their shapes and their constraining functions, the informational forms cannot be usefully subsumed within a dynamical description purely in terms of laws of physics (Pattee, 1969, 1982; Rączaszek- Leonardi & Kelso, 2008). Accepting such relations between dynamics and informational forms provides an operationalization of the meaning of the latter, which is compatible with ecological psychology, namely, as a process of functional constraining of a system, reducing its possible states and trajectories relevantly to given situational and boundary conditions. Because, due to their capability of capturing multiple interacting constraints in the study of behavioral dynamics, the dynamical systems methods were methods of choice (source of formalisms) for ecological psychology for a long time (Haken, 1990; Kelso, 1995), applied also in theories and research on development (Smith & Thelen, 2003; Smith et al., 2010; Thelen & Smith, 1994; van Geert, 1994), such operationalization should be compatible with this approach and facilitate the leverage of its explanatory power to apply to so- 1 The constraints are understood here as enabling constraints. The degrees of freedom of the system are bound so as the system is enabled to perform a function (like, e.g., in acquiring a skilled movement). 7 called higher cognitive processes. The dynamical systems approach helps to demystify the notion of “interpretation” in informational processes and cognitive semiotics. In this way, instead of being a hidden (and often underdefined) mental process, the interpretation of an informational constraint becomes directly measurable. Even if it can be seen as being partly a product of attentional and memory processes (e.g., Smith et al., 2010) it is also assessable in terms of the way it limits the possible states and trajectories of a system. Finally, cognitive semiotics introduces a finer-grained account of the informational constraining process. Semiotics distinguishes multiple relations that can hold between an informational form (in semiotics, a sign vehicle) and its “meaning” (Deacon, 1997, 2011; Peirce, 1931). An iconic relation depends on form similarity between sign vehicle and what it refers to; indexical reference depends on contiguity, or direct causal connection. Symbolic relations are more complex than the other two. They are usually described as devoid of any such (causal or shape-based) relationships and characterized by an arbitrary mapping (but see later). The application of such semiotic distinctions among the ways of signification provides an insight into the kinds of constraints that informational forms may provide as well as the kinds of historical processes they require to become effective. Most important, the semiotic hierarchy of the modes of signification proposed by Peirce (1931) and refined by Deacon (1997) for the domain of cognitive psychology and language evolution offers a framework for understanding the relations between immediately contingent or causal (indexical) or similarity-based (iconic) constraints and symbolic ones. Using this framework, we demonstrate how the emergence of symbolic signification rests on a rich infrastructure of indexical and iconic relations in which symbols participate. Language development as an ungrounding process Perhaps the most important difference distinguishing our approach (e.g., 8 Rączaszek-Leonardi & Deacon, in preparation) from most other theoretical approaches is that the concept of “symbol” is not treated in terms of a mapping relation. Rather, as we explain later, it is understood to be a relation that is dependent on a complex semiotic infrastructure created by prior communication. So the main problem is not how children ground abstract formal symbols (somehow delivered to them as such) but how their embodied, embedded, and situated communicative behaviors can ever become symbolic. This is what makes it an ungrounding process rather than a grounding process. Consequently, our analysis requires an account of how informational forms in general function within situational dynamics; how they become progressively decoupled from properties shared with these dynamical relations; and how, despite this decoupling, they maintain their controlling power over the pragmatic functions of communication. This is in contraposition with most approaches to language development. To our knowledge, not many frameworks tackled the problem in a similar way. Lock (1980), for example, presented some ideas of how within social interactions actions become coordinated with (and by) language. However, taken up by constructivist approaches (e.g., Tomasello, 2003), the learning context has been usually simplified, such as when, for example, gestures rather than situated behavior and events were interpreted as prelinguistic means helping children to enter the language system. In this article, we strive to take into account the complexity and history of interaction. Large fragments of the path to constructing our approach are shared with ecological psychology framework and its views on development in general and language development in particular. Our approach is informed by systemic theories of development, where physical, social, and environmental factors interact at each point (Bronfenbrenner, 1979; van Geert, 1994). Central to our approach is the notion of a continuity of development and the formation of a 9 developmental pathway. This means that we believe that any behavior, experience, or ability at any moment in time is a result of the cumulative effect of previous interactions within the system and at the same time shapes future development prospectively (Fogel, Garvey, Hsu, & West-Stroming, 2006; Hsu & Fogel, 2003). Furthermore, guided by ecological systems theory we consider the tuning processes in action as central developmental mechanisms resulting in gradual shaping of perception and behavior, which in this way can be informed by the physical and social attributes (Rączaszek-Leonardi et al., 2013) and values inherent in the system (Nomikou & Rączaszek-Leonardi, 2015). Language development is thus placed within a theory of development that emphasizes adaptation and perception–action cycles in the world and “uses differentiation model rather than construction models … or the maturation models” (Dent, 1990, p. 690) to account for developmental change. Congruently with earlier approaches to language development within ecological psychology, functional and pragmatic aspects of language come to the fore (e.g., Dent, 1990; Reed, 1995). The focus is thus on “how the use of language can be an event or part of an event” (Dent, 1990, p. 690) in a changing environment of a developing child (Read, 1995). However, although we do feel that considerable efforts have been made to bring the researchers closer to a “theory of the environment in which language-learning children find themselves” (Dent, 1990, p. 194), perhaps because of the general individualistic Zeitgeist of the time, even those functionalist approaches emphasized more the possibility that language is an affordance “through which” invariants of the world can be detected. In our approach, we strive to see language as part of events but not just any part, or even a part “through which” one might see the world but a part with controlling (constraining) role, accumulated through a history of interaction (ontogeny) and in the process of cultural selection. We stress that the elaboration of the theory of acquiring language as a system of social coordination crucially includes an explanation how an active agent 10 becomes increasingly in control over (mainly social) environment. We cannot do justice to a spectrum of the relevant approaches here to make a comprehensive comparison. Let us only mention two other works that are undoubtedly an inspiration to us in the present task. One is Bruner’s (1983) account of language development, in which he underscores the importance of interactive routines at each step of this process. The other is Elizabeth Bates’s work published in The Emergence of Symbols (1979), in which she also employs a semiotic perspective to analyze the process of symbol acquisition. Our work can be treated as an amendment and extension to her account (for details see Rączaszek-Leonardi & Deacon, in preparation) undertaken in an ecological vein, that is, focusing on understanding how language is a part of acting with others that accumulates a particular controlling role and not on internal mental processes. Because within this framework, informational forms function as constraints on complex multimodal dynamics, the first step, which is the topic of the section “Shaping Early Interaction Dynamics,” is to identify the relevant dynamics and explain how they may become controlled. This requires studying human interactivity at very early stages and acknowledging a variety of multimodal informational controls at work that make early caregiver-infant interaction already relevant, purposeful, intersubjective, intentional, and conventional. The integration of the three approaches proposed earlier allows for the analysis of this process as a progressive emergence, within the specific niche, of social affordances, which have the power to enable meaningful behaviors of interactants. Next, in the section “The Ways Language Means” we acknowledge that from the earliest moments of interactions within the social world, language serves as a constraint on social behavior (Nomikou & Rohlfing, 2011). Thus, speech is always fully grounded in the multimodal streams of events, and— 11 through a history of interaction—regulates the trajectories of social encounters similar to the way that any other gesture and action might do. Thus, it is important to note, as many researchers in language development indeed have (Bates, 1979; Bruner, 1983), that language has an important, multifaceted informational control role before becoming symbolic. Language can play this role because from early on, infants are sensitive toward human speech. This can help them, for example, to tune to the rhythms of interaction (Trevarthen, 1974) and to partition actions, which may in turn help in categorization of objects and events (see a summary in Rohlfing & Tani, 2011). Semiotic analysis is particularly helpful here and enables us to illustrate how language functions in a rich network of semiotic relations other than symbolic and point out that such grounding is crucial for subsequently developing its symbolicity. Finally, in the section “The Emergence of Symbols,” we propose (after Deacon, 1997, 2011) that for the ungrounding of such functionally grounded language (a set of controls on dynamics) the crucial process is grounding in another system. Thus utterances, which always remain grounded in interactions, are at the same time connected with other linguistic utterances. Both types of grounding are realized via semiotic relations (iconic, indexical) that reflect the causal structure of events, allowing for making predictions and for control. However, grounding utterances in the system of other utterances allows for a qualitatively different type of control, not only by individual linguistic elements but also by the relations among them. It is this systemic property, and not the conventionality or arbitrariness, which appears in much “earlier” types of semiotic control, that allows for ungrounding of symbols from the ongoing stream of events and brings a novel (formal) type of causality to control those dynamics (Bates, 1979; Deacon, 2011). Showing that this indeed might be the way this process happens in language development requires showing two things: first, that an infant’s 12 environment is structured in such a way as to prioritize the emergence of the linguistic layer (e.g., utterances increasingly often surrounded by other utterances, both in individual speech and in dialogue), which may facilitate tuning to relations among linguistic utterances. Second, that those relations are effective constraints that enable novel (e.g., combinatorial) forms of control. We now turn to elaborating each of the three processes and illustrating each of them with examples from real interactions. Shaping early interaction dynamics Adhering to the main tenets of the ecological psychology approach makes a researcher turn toward the environment of the developing human and be especially attentive to the emergence of relations between a cognitive system and this environment (Gibson, 1979/1986; Mace, 1977). Due to the formation of such relations in evolution, environment is already inherently informative for an agent, as the agent’s body and senses are tuned to perceive it in terms of affordances, that is, tuned for particular adaptive action–perception cycles (Gibson, 1979/ 1986). In development, these relations can be fine-tuned, this time in a culturally specific way. This angle is useful to understand the already rich and meaningful dynamics of infant-caregiver interactions and their progressive shaping by tuning to particular events and behaviors. To this end, in previous work we employed ecological psychology principles of tuning to affordances in development to show how any action or gesture might become meaningful in parent-infant interaction (Rączaszek-Leonardi, 2016; Rączaszek- Leonardi et al., 2013). What is crucial for this to occur is a particular structuring of the infant’s environment. In the case of social-cognitive skills (i.e., virtually the entirety of skills at the very early stages of the infant’s life), acquiring them through such tuning depends on social recreation of important events for the developing agent. We termed such an environment of contingent relations, recreated for infants by their conspecifics, “social physics” (Rączaszek- 13 Leonardi & Deacon, in preparation) to underscore its inevitability and systematicity, which helps a child discover its “laws” and tune the senses to these, as directly picked up social affordances (for a description of the rich structure of the social environment, intertwining social and community practices, artifacts and places, see Heft, this issue). Thus, we underscore the importance of a socially structured environment, consisting of reenacted social routines, or “formats” (Bruner, 1983; Rohlfing, Wrede, Vollmer, & Oudeyer, 2016), in the process of which particular actions become informative (controlling). Such actions (gazes, movements, gestures, vocalizations) gradually take on the role of constraints enabling particular individual and interactive behavior. Note that this account is, in a sense, “movement first” (Rączaszek-Leonardi et al., 2013). Infants’ actions are embedded (sometimes even by a kind of coercion) within culturally shaped recreated sequences of events, for example, when a mother looms over the baby and smiles to elicit a smile at a definite moment of interaction. Due to repetition, which preserves the crucial characteristics of routines, the infant can tune to particular actions as affordances for his or her own actions and use his or her own actions as affordances that elicit the actions of others (Nomikou, Leonardi, Radkowska, Rączaszek-Leonardi, & Rohlfing, 2017). In other words, he or she learns how the “social physics” works and how to control events within it. This is congruent with the ecological psychology account of how actions and objects may become intentional: rather than by being embedded in belief systems, they are embedded in particular “projects” (Merleau-Ponty, 1963), which have their own goal-directed (Heft, 1989; Rohlfing et al., 2016), value-realizing (Hodges & Baron, 1992; Rączaszek-Leonardi & Nomikou, 2015), and collective (Rączaszek-Leonardi & Cowley, 2012; Rączaszek- Leonardi et al., 2013; Richardson, Marsh, & Schmidt, 2010) structure. The development of such social control obviously encompasses a much 14 broader set of social signals than just linguistic ones. Following, we illustrate this process with an example of a familiar nonlinguistic behavior: gazing. From early on, the gaze of an infant functions as an informational constraint on the mother, significantly changing her behavior in a dyadic encounter. Such regularities lead to the gaze gradually becoming a sign to be used communicatively not only by the mother but also by the child. Example 1: Gaze as interaction coordinator This interaction is between a mother and her 3-month-old son. The sequence begins with a verbalization of the action of the infant by the mother, namely, that he has put a toy in his mouth (line 2) and he does that habitually (line 4). 1 I: <vocalization> 2 M: immer alles in den Mund, ne?a) always everything in the mouth huh? 3 (0.9 s) 4 M: so (.) mach immer alles rein da so (.) put always everything in there 5 (1.2 s) Until this point, the infant has been gazing up toward the ceiling (image a). In line 6 the infant turns his head to the side and the mother immediately follows 15 the infant’s gaze and looks in the same direction (image b). The mother reacts to the gaze shift of the infant and repeatedly asks the infant what he is looking at (lines 6–10). 6 M: b)ja was ist denn da? (0.4 s) was ist denn da? Yes what is there? (0.4 s) what is there? 7 (0.3 s) 8 M: Mäuschen (1 s) hm? little mouse (1 s) huh? 9 (0.4 s) 10 M: was ist denn da? (..) ist das ein Würfel? What is there?(..) is that a cube? After the pause in line 10, she asks the infant if what he is looking at is a stuffed toy in the form of a cube and repeats her attribution of his interest in line 11, while at the same time acting upon the object, grasping it, shaking it (image c), and repositioning it at a higher location (image d). 16  11 M: ist dasc) ein Würfel?d) is that a cube? What makes the stabilization of gaze as a signal for constraining mutual attention and thus interaction possible is a systematic enactment of only a subset of possible events around the child. This provides a culturally structured environment within which an infant learns how gazes usually work in social interactions (Rączaszek-Leonardi et al., 2013). It provides a child with a high degree of predictability of the events and a capacity for moving “properly” in the social world. Due to the way that gaze and other actions control social interactions, already in the case of nonverbal behaviors the child may predict the uptake of his or her signaling as well as learn to respond acceptably to the 17 caregiver behavior. Each of the three approaches we integrate in this article has a role in helping to understand what is going on in creating such information-controlled social dynamics. The framework of ecological psychology ensures that due attention is paid to the structuring of the niche, but in addition to the traditional approach we acknowledge the predominantly social character of this world (Rączaszek-Leonardi et al., 2013; Reed, 1995). Two tenets of ecological psychology need to be underscored as particularly helpful here: the dependence of cognition on action and the value-realizing aspect of cognition. First, as shown earlier, the formation of affordances in development takes place in an “action first” manner (e.g., Thelen, 1985). The changing action repertoire enables novel perceptions and informational specifications of behaviors to be provided by the environment (Reed, 1995), endowing the offerings of the world with particular intentionality (Heft, 1989). In our case, it is crucial that practically any broadening of the early repertoire of an infant occurs within interactions, thus it develops “co-action first,” giving a primacy to the “we” (Rączaszek-Leonardi & Cowley, 2012), that is, to collective action over individual action. This changes the unit of adaptability and broadens the repertoire of what might be attainable (a dyad might be able to do more than an individual). Second, tuning one’s actions to such constraints in dialogical co-action within a specific cultural context (Bruner, 1983; Rączaszek-Leonardi et al., 2013) makes them much richer and value saturated than a simple association in the infant’s head: “I gaze—mother follows.” Gaze becomes an affordance, makes sense, only in a larger schema of events, such as realizing common goals and preserving important societal and cultural values. This is another advantage that stems from employing the framework of ecological psychology: reintroducing values into scientific explanations of behavior (Gibson & Crooks, 18 1938; Hodges, 2014; Hodges & Baron, 1992; Hodges & Rączaszek-Leonardi, in preparation). Values provide boundary conditions on the timing, sequential order, and regularity of social affordances, therefore fine-tuning the coordination of attention, mutual respect, and agency of the participants (Nomikou et al., in press; Rączaszek-Leonardi & Nomikou, 2015). Connectedly, shaping the social affordances is aided by the fact that the world of the infant is saturated by “dynamically changing emotional contours” (Leavens et al., 2014, p. 1), which scaffold the awareness of other people’s affective states. In the aforementioned example, the contingent responsiveness to infant gaze, the continuous use of rising intonation at the end of each utterance of the mother, but also the lack of tension on the infant’s face and body, all indicate the emotional attractiveness of the current action for both participants and constitute affordances (constraints) for its trajectory (Jensen & Pedersen, 2016). The dynamical systems approach to the informational role of social affordances serves to guide operationalization of the constraining function of the situation and particular behaviors. The way in which and extent to which the parent’s and child’s behavior as well as the dyad’s as a whole are constrained can be measured using dynamical systems tools (Kelso, 1995; Thelen & Smith, 1994). This includes measuring the degree of constraint (dimensionality reduction; e.g., Riley, Richardson, Shockley, & Ramenzoni, 2011; Shockley, Santana, & Fowler, 2003; Yu & Smith, 2012), the strength of coupling (Nomikou, Leonardi, Rohlfing, & Rączaszek-Leonardi, 2016; Warlaumont, Richards, Gilkerson, & Oller, 2014), the specific properties of coupling (Leonardi, Nomikou, Rohlfing, & Rączaszek-Leonardi, 2016), and identification of the nonobvious parameters and timescales on which behaviors might be matched (Abney, Warlaumont, Oller, Wallot, & Kello, 2016). For example, in Nomikou et al. (2016), we have shown quantitatively, using cross- recurrence analyses, how the gaze of an infant is an important behavior for the 19 mother (i.e., how it constrains her behavior in the sense that some of her behaviors after the child’s change in gaze direction become more probable than others) and how mother’s gaze becomes increasingly important for a child (i.e., constrains the child’s gaze). Finally, a semiotic analysis of the informational processes is useful for distinguishing among the variety of the types of constraining relations that a given behavior provides. In our example, the gaze of a child serves as an index for the mother that the child has an interest in interacting with or obtaining some object. In response the mother strives to create a “social physics” around this gaze, helping it to become a communicative index useful in future contexts. Thus the child comes to anticipate this effect and thereby uses his or her gaze as a controlling index. In addition, because the directionality of the child’s gaze is spatially correlated with the directionality of the gaze and actions of the mother, it provides iconic information, via this parallel form, about the focus of attention. Later development of pointing gestures further tunes this communicative function (e.g., Gomez, 2007). Semiotic analysis also makes evident another important feature of such information-controlled dynamics. It demonstrates that these early iconic and indexical behaviors are already conventional and therefore partially arbitrary with respect to which aspects of behavior are taken to be significant. Conventionality and arbitrariness are thus not the exclusive property of symbols. What becomes an affordance and which behaviors it will specify depends on selective reenactments of certain sequences of events. These sequences may use natural propensities but are also differently shaped by a given culture. A caregiver will not take any response as a valid contribution to an interaction. In an example described by Heller & Rohlfing (2017), a 9- month-old girl wanted to point with her nose to a picture as a reaction to the mother’s question “Where is the spoon?” The mother did not accept it as a 20 conventional means and kept asking. Finally, she answered herself by pointing, and the girl imitated it soon after. We give similar examples below in interactions with much younger children (see, e.g., Example 2). Cultural conventionality is evident long before symbolic language emerges. It is the enacted cultural “social physics” and not material physics that shapes the niche of a developing child and is a source of constraints on behavior. By the same token, because the enactments are selected for their interaction control value, they may bind selected features of behavior with specific types of effects, which leads the way to abstractness and generalization, similarly as in Gibson’s ecological psychology framework (Gibson, 1986) there is abstraction of invariants from a structured environment. In other words, semiotic analysis helps us realize how interactive dynamics is meaningfully constrained by a variety of informational relations. The world of an infant is full of directly perceived intentional, conventional, and even abstract constraints (that we might describe as semiotic affordances) even before language emerges as a symbolic system. The ways language means: A variety of semiotic engagements of linguistic forms The individual and interactive dynamics are thus from a very early stage under control by actions that become affordances. Such actions obviously also include linguistic utterances. Language is an important part of the multimodal streams of behavior from day one or even earlier (Nomikou & Rohlfing, 2011). From birth, it is intertwined in interactional value-realizing and goal-oriented actions. For this reason, it is important NOT to treat language used in those interactions as something distinct from other actions. Without giving utterances any privileged role or any putative properties of symbols (such as being arbitrary, conventional, abstract, or formal), we thus see them as any other behavior functioning in action–perception control loops. Just as any other actions and 21 gestures, they will be built upon natural sensitivities and tuned to as important events in “social physics” that help infants predict and, with time, control patterns of social interaction. In other words, using semiotic terminology, utterances in early interactions can be shown to function as indices and icons, acquiring the power of controlling behavior in the way that other social signals do. And they are already conventional—selectively honed by a history of interactions—but not yet symbolic. This again points to the importance of routines (Heller & Rohlfing, 2017 Rączaszek-Leonardi et al., 2013; Rohlfing et al., 2016) and play (Bruner, 1983; Nomikou et al., 2017) in establishing an early role for the nonsymbolic use of words, enabling the child to become an active user of these signs in interaction control. Routines and “formats” (Bruner, 1983), which constitute the social niche, thus include also Wittgenstein’s “language games” (Wittgenstein, 1953/1958), providing structures where behaviors (including utterances), objects, and events can become informational, that is, gain the power to functionally constrain interactional behavior. But at this stage these utterances are still fully grounded iconic and indexical signs, being embedded in the stream of physical, multimodal co-actions with the infant, regulated by goals (Rohlfing et al., 2016) and values (Nomikou & Rączaszek-Leonardi, 2015) long before a value or a goal can be grasped by an infant. In the following section we present two examples of situations in which language serves as such control, where the particular enactments tune a child to the constraining effects that certain utterances (and/or their properties, such as prosody or stress) tend to have in interaction. They thus function as indices and icons in social physics, helping to predict and control the events. Example 2: Where does ‘Hello’ go in the interaction? Here we give an example of the mother using a specific utterance “Hello” in 22 specific moments of interaction, contingent on infant’s behavior. VP01_1T (00:20– 00:28) 3-month-old boy The infant is still in a drowsy state and the mother begins to gently stroke his cheeks and blow air on his face to wake him up. She is leaning over the infant with her elbows resting on the changing table (image above line 2). In line 1 the mother’s attempt is successful as in the left image above line 2 the infant opens his eyes very slightly. The mother has her face positioned right in front of the infant and as soon as she sees a slight opening of the eyelids she produces a prolonged greeting while smiling (right image above line 2). 1 ((mother blows into infant’s face)) 2 M: ja:: hallo ((breathes in and blows)) yes hello 3 M: ja hallo wer ist denn da? yes hello who is there? The sequence continues around half a minute later (see the following transcript) in which we have the next opening of the infant eyes. In line 1 we can see that although the infant has opened his eyes (left image above line 1) the mother has not noticed it because she is opening his romper suit (central image above line 23 1). VP01_1T (00:53 – 00:59) 1 M: ah ist das schön ah isn’t that nice 2 M: ja hallo: yes hello 3 ((inbreathe)) ja hallo: yes hello Upon concluding her utterance she looks up (right image above line 1) and 24 notices that the infant has opened his eyes. In reaction to this in line 2, she initiates a new greeting. This greeting is contrasted to her previous utterance in multiple ways. She modifies her head position making a swaying movement in synchrony with her utterance; she raises her eyebrows and produces a big smile (the image above line 2). As the infant maintains his attention to her face she repeats the greeting (line 3), this time marking the transition to the next utterance with an exaggerated inbreathe and movement of the head. In the sequence presented so far the mother responds to the opening of the infant’s eyes with a rich range of behaviors. From the perspective of the infant, opening the eyes and engaging in eye contact is met with a set of very specific cues, that is, a consistent linguistic form “hello” accompanied by coordinated facial expressions and head movements. At the same time, the combination of these verbal and affective cues construct an emotional setting that might motivate the infant to participate in the interaction and “share” the interactive experience with the caregiver. Various properties of speech may play various semiotic functions, not only the form of a word (at the beginning the form is probably least likely to be differentiated properly) but also the timing of vocalization and the intonation patterns might initially be especially potent constraints (Gratier & Trevarthen, 2008; Trevarthen, 1974). In connection to later development, Bruner described such a semiotic process of attentional control when he said, “The first phase of managing joint attention, very much under the control of the mother, thus appears to result in the child discovering signals in the mother’s speech that indicate that the mother is attending to “something to look at” (Bruner, 1983, p. 73, emphasis added). Thus children may develop sensitivity to “undifferentiated deictics” (in a sense quite abstract, as in “there is something somewhere in the environment to attend to”), often indicated by a rising intonation pattern (carved perhaps from the natural sensitivity to fast-rising stress). Such 25 behaviors of caregivers become “alerting signals” about the possibility of attentional shift. Adding to this, the rich experiences gained by such affective exchanges might motivate infants to follow another person’s focus of attention (Leavens et al., 2014). The importance of the prosodic contour is evident in our example. Continuing the same interaction, 20 s later, the sequence proceeds, with the mother initiating a peek-a-boo game in which she uses the infant’s socks to cover and uncover his eyes. After a few iterations of the game a new set of greetings are produced by the mother, requiring a richer participation from the infant. VP01_1T (01:15 – 01:26) Figure 1 Interaction sequence between mother and infant. The mother’s turns are verbal utterances and the infant’s turns are smiles. The arrows indicate the sequencing of the turn exchanges. The numbers refer to the intonation curves presented in Figure 2. The sequence begins with the mother’s question “Where is the baby?” (Figure 1, first text box on the left) upon which the infant gazes neutrally at the mother 26 (first image on the left). She responds with “There he is.” The infant responds to this with a smile (indicated with the number 5) upon which the mother pauses the peek-a-boo game and responds with a greeting. The subsequent production of smiles produces consecutive greetings. The sequence also shows a climax as the next smile produced by the infant in the sequence is more intense than the previous. Mother and infant reciprocate each other’s greeting/smile four times in this sequence. It is only when the infant returns to a more neutral position (last image on the right) that the sequence is completed and the mother returns to the peek-a-boo game. In this sequence, we can see the active shaping of a smile as the correct response to a greeting. The infant receives a very specific reaction to a very specific behavior that he produces. In comparison to the first part of the sequence in which “eye contact” was the necessary requirement for the mother’s reaction, here we see that the infant response is further specified as a smile. Thus, it is a different signal by the infant that receives the rewarding of the multimodal package of the greeting. As shown by the arrows indicating the exchange of turns, the mother produces one greeting after each smile (and no greeting if a child does not smile). At the same time, the mutual gaze at each other’s face and the exchange of smiles is a kind of acknowledgment of the jointness of the interaction (Carpenter & Liebal, 2012) and in this case also an awareness of the interactive roles of the participants. Apart from the recurring “hello” and the nonverbal modalities coordinated with this greeting such as looming forward and approaching the infant’s face, smiling and raising eyebrows, loud inbreathes synchronized with head movements, there is still an even richer package of resources framing this activity, such as a distinct intonation pattern repeated with every greeting. As illustrated in Figure 2, all greetings produced by the mother in this larger sequence follow a more or less bell-shaped intonation curve, with a high contrasting rise at the beginning and a lowering at the end. This produces a 27 strong cue, indicating that all these instances (which follow the infant’s smile) potentially share some qualities and are of the same kind. Figure 2. Illustration of the intonation curves corresponding to the greetings of the mother from the elaborated example. The red circles mark the exact intonation curves, which show a pattern of a sudden rise and then fall. In this data corpus such openings were often followed by an enactment of an entire greeting routine such as “How are you?” or “Did you sleep well last night?” The consistent prosodic shape of the “Hello” further helps differentiate this particular utterance and place it in the stream of multimodal events. The form of prosodic contour may later serve as a distinguishing property of utterances placed at this particular moment of interaction and, for example, make it possible to substitute “Hello” with other utterances, such as “Good morning” having a similar pragmatic function. 28  It is often the case that the indexicality and iconicity of an utterance come together when not only the contingency in time helps predict the occurrence of a certain event but also the property of an utterance (accompanying certain event) allows one to predict its physical parameters. Below we show an example of shaping the prosody of the utterance in a way that it fits the action of the mother (some properties of the utterances are thus icons for some properties of action). Example 3: Iconic cues to action in language VP11_1T (05:48 – 05:59) 3-month-old boy In this example the mother is dressing the child. She has just finished putting his legs in the tights and in line 1 rolls the infant on his side to pull up his tights over his diaper. In doing so she synchronizes the onset and offset of her utterance with the duration of the rolling movement. More specifically, while grasping the infant’s arm (left image above line 1) she produces the “one” (see line 1) and then initiates the rolling movement by repeating the word “roll” three times throughout the movement (central image above line 1) until she stops (right image above line 1). By doing so she recreates with her utterance the physical property of the rolling movement, that is, a sound that cycles. This is made even more perceivable through her intonation, which is somewhat U-shaped (moves down and then up). 29  1 M: ein mal [rolle] rolle [rolle:: one time roll roll roll 2 I: [(voc.)] [(voc.) 3 M: ja [ah ist] die doofe Flasche im Weg yes ah it’s the stupid bottle in the way 4 I: [(voc.)] 5 (3.11) 6 M: [u::nd wieder zü::ruck and again back 7 I: [(voc.) A similar pattern is recreated in line 6 in which she rolls the infant back on his back. Here she again times her utterance precisely to follow the movement of the infant’s body. She places her hands on the infant’s back when saying “and” (left image above line 6) and stretches the word “back” so that it coincides with the duration of the rolling movement. Note that here she again uses the same intonation patterns as in the previous rolling movement, tying the two together (see Figure 3). Thus, in a similar “ecological” manner as any other action or gesture, various properties of speech are selected to play a constraining role in interactions. Tuning to them is similar to tuning to any other social affordance. In a similar way as in the examples of, for example, gaze, we can use the 30 dynamical systems analyses to assess the extent that utterances in early interactions constrain the degrees of freedom of the interactive system. Semiotic analysis shows why speaking of early language use should not be confused with using language as a system of symbols. Figure 3 Intonation pattern of mother’s utterance accompanying the rolling movement of the infant’s body. Utterances are first treated as indices (used to predict timing, occurrence of events) or icons (used to predict certain properties of events). They elicit emotions, establish rhythms, partition events, and manage attention long before they become symbolic and—it is important to note—serve these afterward as well! Such connection to dynamic interaction, we claim, provides the ground for subsequent forms of symbolic reference and control that emerge with later language development. Such grounding does not vanish, thus ensuring that linguistic means of control (which will, in time, become symbolic) remain nonmiraculously, causally connected to the ongoing stream of events. In this treatment of utterances of language—first as indices and icons and therefore NOT as arbitrarily mapping to things in the world—we can be 31 more precise in our analysis of the referential properties of language. It is important to note that “names” for “objects” at these early stages are rather icons and indices for aspects of coordinated behavior (sometimes, indeed, with respect to objects). An uttered noun for an adult might be a “name” of an object, but most likely for an infant in early interactions it is an attentional or action coordinator. An utterance “Oh, look!” is not a name of the thing or even a name for action but rather an index predicting the dyad’s joint gaze structure. In short, in early interactions speech is very often performative in the sense that it affords a sequence of multimodal actions. It is within these pragmatic frames, and not “semantic” or “image schemas” frames (which are traditionally linked to describing or mapping the world), that utterances gain their controlling properties. Speaking of later language development, Bruner (1983) noted, Paradoxically, the learning of speech acts may be easier and less mysterious than the learning either of syntax or semantics. For the child’s syntactic errors are rarely followed by corrective feedback, and semantic feedback is often lax. But speech acts, on the contrary, get not only immediate feedback but also correction. (pp. 37—38) The accentuated performative role and the immersion in multimodal co- action makes the ungrounding problem seemingly harder. How can language ever detach from this carefully orchestrated, multimodal dyadic enactment to become symbolic? However, paradoxically, understanding the functional groundedness in the various modalities and co-actions is a key support for the ungrounding process, which at the same time preserves these causal influences on social dynamics. The emergence of symbols: Not just conventional icons or indices but elements of a system These distinctions are at the core of the ungrounding problem. To reiterate: this 32 is a problem of (a) how grounded iconic and indexical informational forms can give rise to the degree of abstractness, arbitrariness, and formal properties of a symbolic system and at the same time (b) how they remain informational with respect to individual and interactive dynamics, that is, causally intertwined in linguistically mediated co-action. As we pointed out in the previous section, any actions and gestures can become informative controls on co-action, with some degree of conventionality and arbitrariness even though indexical and iconic. These are thus not exclusive properties of symbolic reference as exemplified by language. Conventionality is already a feature of all the actions that are socially shaped into cultural routines. According to Deacon (2011), it is a common mistake to confuse the conventionality of symbolic reference with the conventionality of the form of the sign vehicles themselves. One can easily have conventional icons, as exemplified by the international variety of icons for toilets for women and men, and conventional indices as, for example, in “social physics” of culturally distinct greeting gestures. Thus, the property of conventionality is neither a distinguishing nor sufficient property for informational forms to be symbolic. Although the form of sign vehicles able to provide symbolic reference is necessarily conventional, they may also retain a degree of iconicity (Dingemanse, Blasi, Lupyan, Christiansen, & Monaghan, 2015; Kohler, 1929, 1947; Perniss, Thompson, & Vigliocco, 2010) or indexicality, but these features are not determinants of their referential function. Symbols must be doubly conventional: conventional sign vehicles with conventionally determined reference. So, the ungrounding process involves decoupling sign vehicle properties from the properties of what they refer to. It is in this process of abandoning intrinsic grounding that “by using language first for limited ends the child comes finally to recognize its more powerful, productive uses” (Bruner, 1983, p. 7). 33  Following the semiotic account provided by Deacon (1997), we propose that the ungrounding from the immediate stream of multimodal co-action is possible because of a shift from direct iconic and indexical relations of utterances to other multimodal events to using iconic and indexical relations between sign vehicles to disambiguate reference. Whereas presymbolic grounding relies on being causally and predictively involved as controls on multimodal interactions, symbolic grounding is mediated by systemic icons and indexical relationships among linguistic forms themselves. This indirect grounding is not just “learning abstract associations” between utterances and referents. It is a function of iconicity and indexical relations between these forms and how this higher order relational structure retains a grounded iconic or indexical relation to social physics. Linguistic utterances, unlike most of other controlling actions or gestures, are thus embedded in parallel, both in ongoing multimodal interactivity in which linguistic forms are indices and icons controlling the interaction and (also as icons and indices) within complex linguistic structures. The latter loosens the grip of the first grounding, giving linguistic utterances partial freedom from the social physics they modulate. In a similar vein, Zukow-Goldring (1996, p. 207), spoke about “additional perceptual structure” that is provided by the parents in order to highlight new elements that are introduced in routinized actions. It is important to note that infants can rely on “social physics” and their knowledge about what is happening. Parents use these structures, then, to relate language to these events. Rader and Zukow-Goldring (2010) successfully showed that providing language and action simultaneously is an effective way to convey the meaning of a new word to 9-to-14-month-old infants. Although this experiment clearly shows that parents can limit choices and socialize children’s attention (Zukow- Goldring, 1990), it does not address the process of ungrounding the language. In her work, Zukow-Goldring (1996) provided, however, some ideas about this problem. More specifically, she was of the opinion that routines provide a solid 34 basis for language understanding because the child knows what will happen. Thus, the knowledge about actions and their order is at the bottom of symbolic understanding: “Knowing what is going on appears to emerge a step of two ahead of knowing how to relate language to those events” (p. 208). In fact, Zukow-Goldring (1996) proposed that language might be particularly helpful in breakdowns in communication bringing up invariant aspects between familiar and unfamiliar settings. In order to show how this contributes to the ungrounding of symbolic forms in language development, two kinds of evidence are required: first, we must demonstrate how the infant’s enacted environment facilitates tuning to the relations among linguistic utterances. This involves making modality-specific (linguistic) contingencies in the co-actions with infants more salient with age, giving them a priority over multimodal contingencies. Second, we must demonstrate how the iconic and indexical relations among utterances, and not undifferentiated expressions, become effective constraints on co-action and provide the possibility for novel modes of social control. Figure 4. A cross-recurrence profile of vocalizations, showing the emergence of turn- taking structure at 6 and 8 months. Note that the probability (shown as percentage 35 recurrence, that is, matching behavior of mother and infant) that the simultaneous vocalization of infant and mother will co-occur (i.e., match at lag 0) decreases with the infant’s age. Emergent linguistic layer The emergent linguistic “modality” (not a module), which might facilitate this second kind of grounding of linguistic utterances in other utterances is visible in the distribution of possible reactions to infants’ vocalizations. Using an earlier established dialogicity of actions (now-you, now-me) parents establish the same pattern in a vocal domain. This process can be observed from the earliest months, first in the emergent turn-taking structure of actions (e.g., Kaye & Wells, 1980) and then in the vocal modality (Leonardi et al., 2016). In the latter case, we observe a decrease in the probability of overlaps between the ages of 3 and 6 months (Figure 4), which stands in contrast to, for example, progressively coupled structure of gaze in the same age range (increase in the probability of mutual gaze, where instead of the valley, present in Figure 4, we observe a peak at the 0 lag; Nomikou et al., 2016). Another crucial process in co-creation of the realm of language is the infant-age-dependent propensity of the caretakers to single out some vocalizations rather than others and to respond with language to more language- like vocalizations (Radkowska, Nomikou, Leonardi, Rohlfing, & Rączaszek- Leonardi, 2017; Warlaumont et al., 2014). Again, note that a system created in this way is a dialogical system: a child discovers that some elements go together, for example, as adjacent pairs and not only in a stream of mother’s talk. Such proto-conversations begin early and may have diverse forms, from the first months when mothers take the vocalizations of infants as contributions to dialogue and embed them either in elaborate conversations or playful 36 imitations. Following we have examples of both. Example 4: Responses to early infant vocalizations Example VP10_6T (05:05 – 05:35) 8-month-old boy This example begins with the mother having just finished changing the diaper and she is leaning in toward the infant, who is holding and moving a tube of cream in his hands (see image above line 1). The mother begins by asking a question (line 1) and after a long pause in line 2 proceeds with an elaboration (lines 3 and 4). 01 M: ist die tube interessant is the tube interesting? 02 (1.74) 03 M: ja(h)a (.) von vielen verschiedenen seiten kann man die= =angucken ne? ye(h)es (.) from different sides you can watch it, huh? 04 (1.13) 05 M: und die sieht von jeder seite anders aus and it looks different from each side 37 In line 06 the infant extends his arm holding the tube toward the mother while shifting his gaze toward the mother (left image above line 6). The mother at this point is looking at the infant (central image above line 6). Having engaged in eye contact with her he vocalizes (line 6) and then directly switches his gaze back to the tube (right image above line 6). The mother in line 7 responds to this initiative by imitating the infant vocalization while shifting her gaze to the tube, too (images above line 7). They are now jointly looking at this object. 06 I: ((voc.) 07 M: ((imitates)) ((inbreathe)) ((imitates)) The sequence continues with mother and infant looking at the tube and engaging in a conversation-like turn-taking sequence that ends in line 14 with a statement from the mother about the action of the infant’s hands (line 14). 08 I: ((voc)) 38  09 M: ((imitates)) 10 I: ((voc)) 11 M: eine schöne tube a pretty tube 12 (0.96) 13 I: ((voc)) 14 M: und was die hände alles so machen können and the hands all the stuff they can do Until line 14 the interaction gives the impression of being about the tube. The infant shows awareness of the positioning of his verbal utterances in terms of the interaction structure with the mother but also in terms of the management of attention with her (raising the tube, engaging in eye contact with her, then vocalizing and returning his gaze to the object). Also, the mother is treating the infant’s utterances as meaningful contributions in a dialogue, creating pauses for those contributions at specific moments. In line 16 a new episode begins that has a different structure, where not so much the object of mutual attention but rather exchanging these playful vocalizations is the main focus. The infant has now lowered his arm holding the tube and after a pause looks up at the mother and vocalizes while sustaining eye contact with her. What follows is an iteration of three sets of turns (lines 16– 21). 39 15 (0.81) 16 I: ((voc)) 17 M: ((imitation)) 18 (0.96) 19 I: ((voc)) 20 M: ((imitation)) 21 (1.36) 22 I: ((voc)) 23 M: ((imitation)) 24 I: ((infant smiles)) 40  25 M: ((mother smiles)) The pairs of turns in this sequence are very precisely matched as can be seen in Figure 5. They are separated by clear pauses, meaning that the infant waits after the vocalization of the mother to initiate the next pair. The mother produces a vocalization that resembles the pitch level of the infant and the phonological properties. Interestingly she is adding an element to the sequence as the intonation patterns of mother and infant are complementary, with the infant vocalization having a slight rising intonation, while the imitation of the mother has a falling intonation. These patterns give the sequence a playful character, which is evidenced by the fact that after the three sets of turns the infant stops vocalizing and produces a large smile (image above line 22), which is then followed by a mother smile (image above line 23) and the sequence ends there. 41 Figure 5 Intonation pattern of the mother-infant vocal play described earlier. Marked in red are the infant’s utterances and in green the mother’s utterances. It becomes visible that the vocalizations are matched in pitch level and that they show complementary intonation patterns (rising-falling) As in the case of establishing particular actions (e.g., gaze) as informative in interactive situations, also in this process, specific games, routine formats, and pragmatic frames are important. For example, some of the games, with older children, have a dialogical structure, which forces the interactants to remain within a vocal modality, with components of utterances provided by both participants. Rohlfing et al. (2016) recognized that, for example, labeling routines are also such formats, preparing the child to perceive a label and to remember it. In this way the linguistic modality is accentuated by an increase in the probability of responding to language-like vocalizations with language. The linguistic modality emerges, progressively differentiating from the multimodal interactions of infants and caregivers2. The discovery of linguistic statistical 2 This direction of function specialization in development has been also argued as a 42 regularities in this system might be facilitated by this emergence. Relations matter This emerging layer, in which utterances enter in relations with other utterances is crucial for loosening the strong constraining role that specific, structurally undifferentiated elements of language have in multimodal interactions. It is within this layer that relationships among utterances will be noticed, which is crucial for this loosening. As a first analysis of the process it is useful to examine a highly simplified example. In cognitive-semiotic terms, this process has been investigated by Deacon (1997) and illustrated by his account of how a very limited form of symbolic communication emerged in chimpanzees trained to use an artificial symbol system. In this well-known case (Savage-Rumbaugh, Rumbaugh, Smith & Lawson, 1980), it is easier to observe the transition from a grounded indexical understanding of individual sign vehicles (lexigrams) to understanding them as a system of symbolic relations both because of the simplicity of the system and because the process is slowed down, as the chimpanzees struggled to make the transition. In other work we’ve described the theoretical framework for this passage in more detail (Rączaszek- Leonardi& Deacon, in preparation). In this study six lexigrams (buttons marked with randomly correlated patterns corresponding to two foods, two drinks, and two means of delivery) needed to be pressed in two-lexigram combinations (ignoring trial initiation actions) in order to control delivery of two foods and two drinks using their corresponding delivery devices. Given the combinatorial structure of the task, viable alternative to native “modularity,” most notably by Annette Karmiloff-Smith, 1992). 43 in which a lexigram for a food or drink had to be combined in a specific order with the appropriate delivery lexigram, only four of 30 two-lexigram combinations were meaningful (and vastly many more if combinations of more than two lexigrams could be selected). So even discovering the correct combinations by trial and error would not be a trivial challenge, but the problem was made more difficult because the chimps tended to understand the problem indexically, that is, fixating on a successful trial as a holistic one-to- one mapping between a combination of lexigrams and a food/drink reward and not noticing that working combinations involved an agreement rule between lexigrams. To overcome this tendency the experimenters ultimately had to train them first to make erroneous indexical associations and then systematically extinguish these associations. When, by a systematic process of elimination, the chimpanzees finally limited their choices to only the four “correct” combinations, they had effectively learned to avoid a huge number of incorrect combinations. The mnemonic demands of keeping track of all excluded possibilities is therefore quite significant. So discovering that the same result can also be obtained by merely attending to the four relations of agreement and exclusion of two-lexigram combinations vastly eases the mnemonic load. It also foregrounds the correlated physical relations distinguishing the delivery of food versus drink. Four stages of this transition are depicted in Figure 6. 44 Figure 6 Stages for transition from a grounded indexical understanding of individual sign vehicles (lexigrams) to understanding them as a system of symbolic relations. The transition from indexical to symbolic use of lexigrams in this chimpanzee study can serve as a guide to understand the transition that occurs in child language development. In language development, patterns stabilizing which utterances go together are provided by the adult’s utterances and by enactments of early dialogue. This grounding of utterances in other utterances provides a possibility for their partial ungrounding from simple causal and iconic relations to ongoing events. This is because the interactions now can be influenced not only by single (or undifferentiated) linguistic forms but also by the relations among them. Below we provide examples of interactive situations and events that may facilitate this change. One of the earliest situations that may lead to noting utterance-utterance relations and, in turn, their involvement in interaction is when the stream of talk 45 from a mother is tightly coupled to the co-action with a child. Again play routines and games are important here. They provide structures in which the two types of grounding match (in a sort of iconic relation): the structure of the events will iconically reflect the structure of language used in the games. As Bruner (1983, p. 40–41) noted, “Lexical and phrasal substitutes” appear for “familiar gestural and vocal means for effecting various communicative functions.” This iconicity is reminiscent of the pragmatic matrix that is postulated for early utterances (Tomasello, 2003). Although the child does not seem to be linguistically productive using first utterances, the repetition of some words within a context is a clear reflection of pragmatic understanding. In the following section we present an example of such a “parallel” language-action game with a 3-month-old infant. Example 5: The coordination of language and co-action structures VP06_1T (08:00 – 08:12) 3-month-old girl In the aforementioned example the mother is accompanying her speech with a conventionalized movement of her hands. She is “walking up” the infant’s body while talking about the mouse climbing up, touching the infant’s forehead while “knocking” on the door and finishing by touching the infant’s nose while saying “Mrs. Noseman.” We can see here a longer verbal utterance 46 accompanied with a set combination of movements performed on the infant’s body. Another way of structuring the niche of the infant that may lead to noticing how relations work to constrain interactions comes from the pragmatic involvement of speech in increasingly complex situations. As the child’s abilities grow (he or she can sit, shift attention swiftly, engage with multiple objects and persons) so do his or her needs in dealing with the world. Bruner (1983) claimed that the pragmatic formats of invitational requests are especially conducive for children to use their first multiword utterances (“when one was secure enough to invite, one had the courage as well to try out new forms”; Bruner, 1983, p. 115). Also, as mentioned earlier, the pragmatic aspects of language receive much more feedback and correction than “purely” semantic or syntactic ones. This may also aid the process of detaching from the simple indexicality and iconicity of word use. As some researchers claim (Brown, as quoted by Bruner, 1983, p. 35), “at the two-word stage of language acquisition more than three quarters of the child’s utterance embody only [a] half-dozen semantic relations that are, at base, case or caselike relations—Agent-Action, Action-Object, Possession etc.” Rather than (or in addition to) reflecting the conceptual organization of knowledge (e.g., Fillmore, 1976; Nelson, 1974) these may reflect the structures of co-actions that children are most often involved in at this age and that provide pragmatic frames for discovering how relations constrain. Here the ecological perspective on a changing niche together with maturational processes can again be helpful (see Reed, 1995). As children become more able “doers” they need to involve adults into more complex co-actions: thus the social context is a source of structure as well. Coordinative needs, with time, become too complex for one-word controls. Another process that aids in discovering that relations matter, which might be more difficult to demonstrate on an observation-by-observation basis, 47 rather requiring large child-directed-speech corpora, is noting the involvement of specific forms that are shared by a variety of linguistic contexts. For example, as we note even in our corpora, mothers often use a child’s name (which becomes a salient linguistic form rather early) in conjunction with a commenting or directive speech such as in “What are we doing today, Iris?” “What a mess have we made, Iris!” Bortfeld, Morgan, Golinkoff, and Rathbun (2005) pointed to a crucial role of this familiar element of speech for learning language’s systemic character. In our Example 2, this is also evident in the case of “Hello!” which is followed by various elaborations (“Who is there?” “Where is the baby?” Or, in other contexts, “How are you?” “Did you sleep well?”). This is one of the ways to signal the compositionality of language parallel with involvement in various social situations. This systemic aspect has been underscored by usage-based approaches to grammar and to language acquisition (e.g., Tomasello, 2003). However, this approach to language development can also benefit from a deeper semiotic analysis, revealing signification hierarchies. Deacon (in press) shows that some rudiments of linguistic structuring (such as recursiveness or predicate structure) can also be explained by their involvement in semiotic relations. Thus, for example, pointing and related deictic means can often substitute for noun phrases, demonstrating the role of indexicality in providing grounding for the otherwise ungrounded symbolic features of the utterance. This may facilitate the transition from pointing gestures to two-word structures and the integration of grounded pragmatic communications into the combinatorial logic of early grammar and syntax. This sort of progressive entwining of pragmatic semiotic interactions into linguistic structures may modify their grounding in co-action, but this grounding is never lost. In this way a formal causality that is stabilized in selected linguistic structures may seep into the basic indexical and iconic 48 grounding, resulting in linguistic (syntactic and other) structures modifying even the basic perceptions and interactions with the world (Lucy, 1997). The way it is achieved in specific cases requires extensive further research. We are still at the beginning of this endeavor and the goal of this article has been to show where to start and to provide basic conceptual and methodological tools for organizing such research in a way that does justice both to the functional grounding of symbols and to their structural ungroundedness. Conclusions An ecologically valid approach to understanding how symbolic systems emerge and work requires changing the direction of how we posit the symbol- grounding problem (see also Rączaszek-Leonardi & Deacon, in preparation). All signs used for communication start out as any other physical event and become recruited for communication because of their particular history of exerting control within organism-organism and organism-environment relations. We have endeavored to show that physical actions (including utterances) that serve iconic and indexical roles in regulating co-action maintain their grounding in these activities, i.e., they become parts of “social physics.” Thus, it is not their grounding that is a problem but rather explaining how such grounded physical forms ever get to be symbols. We proposed that an important step in this process hinges on the systemicity of signs that become symbols: besides being grounded in co-actions, they are also grounded in relations with other signs. We have tried to trace this process on the timescale of language development. “Ungrounding” of symbolic forms takes place in a multithread complex process that at the same time maintains grounding of the system in which they are embedded and in which iconic and indexical grounding is progressively augmented or replaced by symbol-symbol relations. This involves aspects of conventionalization, abstraction, generalization, and systematicity. Only some 49 of those aspects have been elaborated in this article. We hope that we made clear that some of the properties that characterize symbolic systems appear much earlier in the semiotic hierarchy of informational forms than on the level of a fully developed linguistic system, providing a necessary background. In this article, we have indicated how conventionality might emerge already at the level of simple actions; how the exerted control can be sharpened by a history of coordinated interactions; and, finally, how the final step in the process of ungrounding symbols, namely, establishing their systemic properties, might arise. We used the foundation constructed out of three approaches to meaning in cognitive systems, which, as we hope to have shown, can complement and enrich each other and provide a toolbox of processes for a holistic account of the ungrounding process. The ecological psychology framework makes it clear how the process of language development is tied to the developmental niche. This niche changes over time both with the physical abilities of the infant (Reed, 1995, 1996; Walle & Campos, 2013) and with the parents adapting the games and pragmatic frames and values to be realized to the developing capabilities of the infants. In this way, infants are maintained as active and responsible partners of co- action. Because ecological psychology does not take symbols for granted, all behaviors (including linguistic ones) are understood as properly grounded in ongoing perception–action cycles and in slower loops of learning, development, and evolution, which sensitize (tune) perceptual systems to the controlling role of affordances for action. The dynamical systems approach, which for years provided ecological psychology with its methodological and analytical toolbox, is also useful in this endeavor by allowing the effects of constraints to be measurable in terms of the reduction of the degrees of freedom of a system under semiotic control. 50  Finally, semiotics provides a conceptual toolbox for analyzing the hierarchic typology of constraints and the historical processes they have to undergo to become means for regulating the social physics that the child finds himself or herself in. More important, the semiotic perspective forms a bridge between mere social physics and language by distinguishing different ways that sign forms can be grounded in the world of pragmatic co-action. This emphasizes that the apparent conundrum posed in the cognitive sciences is just the tip of a semiotic iceberg that involves a rich infrastructure of dynamical iconic and indexical relationships. As Peirce wrote, “Symbols grow. They come into being by development out of other signs” (Peirce 1931, Vol. II, #302). In the study of language development, it is therefore crucial to understand its embedding in (a) the ongoing dynamical multimodal context of significant co-actions and (b) the context of other linguistic utterances, which ultimately emerges as a quite distinct layer of dialogical interactions. It is important to note that, even though the latter unleashes novel forms of control mediated by the structures of language, language never becomes ungrounded from the first types of context thus retaining the hold on dynamical interaction. Although historically the language development problem has often been posed with a directionality similar to that which we have advocated here (see, e.g., the titles of classic works in the domain, such as The Emergence of Symbols in Development (Bates, 1979), Piaget’s Play, Dreams, and Imitation in Childhood (1945/1962) or Werner and Kaplan’s Symbol formation (1963)), even within these domains the properties of symbols were rather taken for granted as things to be learned by a child and not to be explained by the process of freeing informational forms from the immediate dynamical context. It is visible even in the work of Elizabeth Bates, who, similar to our account, employed both the dynamical systems-like types of explanations and semiotics to construct her theory of symbols emergence. A closer look at these analyses of, for example, learning a “name” of an object, reveals that the name is treated as an association 51 or mapping to an object where this arbitrary association is learned by a child (e.g., Bates, 1979). The immersion of the utterance in complex social physics— first as an index and/or icon— with the intonation, rhythm, and stress being equally important as the form, is largely taken for granted and ignored (for exceptions see, e.g., Zukow-Goldring & Rader, 2001). Our examples from microanalyses of early phases of language development in interaction illustrate the ways language is grounded in interactive situations. This highlights the role of culturally stabilized reenacted routines (Bruner, 1983) in which the instantiation of repetitive indexical and iconic relations enables generalization over diverse situations. By performing microanalyses of particular moves involved in communicative interactions, we have provided examples of both the iconic and indexical semiotic processes that are at work in the development of a symbolic system. With these examples, we aimed at pointing to the potential of our approach to contribute to developmental theories delineating how the interaction between the caregiver and the infant enables taking the infant’s skills to the next level of his or her development. The present approach evidences the continuity of language with other intentional communication by underscoring the richness of the functional organization of co-action that underlies the capacity to use language. Even the systemicity, which we deemed crucial for the process of detachment of linguistic structures from the iconic and indexical relations to ongoing events, is present in nonlinguistic modalities, for example, in systemic organization of functional or pragmatic frames, which are the basis of pretend play (e.g., Szokolsky, 2006). The discontinuity in efficiency of control comes thus more from superimposing various properties than from the emergence of a totally novel one, the involvement of speech, which provides energetically cheap and easily replicable structures (Rączaszek-Leonardi, 2009), being probably one of 52 the most important. Obviously, many challenges remain on the way to fleshing out the process of symbolic emergence. In our article, we have only traced this process at its very early stages. We left off the account before, among other pertinent processes, the subsequent massive statistical learning, which undoubtedly takes place once the rudiments of the system are in place, aids discovery of novel constructions and relations (Tomasello, 2003), and symbolically transforms the function of the more basic semiotic relations. However, in this article we offered only a first look at some basic principles that govern the process that gradually transfers the capacity of functional control to language structures. We hope it can serve as a guide for future similar theoretical approaches to these later-to-develop and more complex processes and for the kinds of empirical work and the data that can validate them. Funding This work was supported by the NCN-DFG collaborative Beethoven project Early Semantic Development (EASE) UMO-2014/15/G/HS1/04536. References Abney, D. H., Warlaumont, A. S., Oller, D. K., Wallot, S., & Kello, C. T. (2016). Multiple coordination patterns in infant and adult vocalizations. Infancy, 22, 514–539. doi:10.1111/infa.12165 Bates, E. (1979). Intentions, conventions and symbols. In E. Bates, L. Benigni, I. Bretherton, L. Camaioni, & V. Volterra (Eds.), The emergence of symbols: Cognition and communication in infancy (pp. 33–68). New York, NY: Academic Press. Bates, E., Benigni, L., Bretherton, I., Camaioni, L., & Volterra, V. (1979). The emergence of symbols: Cognition and communication in infancy. New York, 53  NY: Academic Press. Bortfeld, H., Morgan, J. L., Golinkoff, R. M., & Rathbun, K. (2005). Mommy and me: Familiar names help launch babies into speech-stream segmentation. Psychological Science, 16(4), 298–304. doi:10.1111/j.0956- 7976.2005.01531.x Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature and design. Cambridge, MA: Harvard University Press. Bruner, J. S. (1983). Child’s talk: Learning to use language. Oxford, UK: Oxford University Press. Carpenter, M., & Liebal, K. (2012). Joint attention, communication, and knowing together in infancy. In A. Seemann (Ed.), Joint attention: New developments in psychology, philosophy of mind, and social neuroscience (pp. 159–181). Cambridge, MA: MIT Press. Deacon, T. W. (1997). The symbolic species: The co-evolution of language and the brain. New York, NY: W.W. Norton. Deacon, T. W. (2011). The symbol concept. In M. Tallerman & K. Gibson (Eds.), The Oxford handbook of language evolution (pp. 393–405). Oxford: Oxford University Press. Deacon, T. (2012). Beyond The Symbolic Species. In T. Schilhab, F. Stjernfeldt, and T. Deacon (eds.) The Symbolic Species Evolved, Springer, pp. 9–38. Deacon, T. (in press). Beneath symbols: Convention as a semiotic phenomenon Evolution & Contextual Behavioral Science: A Reunification. Steven C. Hayes & David Sloan Wilson (eds.), New Harbinger Publications. Dent, C. H. (1990). An Ecological Approach to Language Development: An Alternative Functionalism. Developmental Psychobiology, 23(7), 679–703. 54 Dingemanse, M., Blasi, D. E., Lupyan, G., Christiansen, M. H., & Monaghan, P. (2015). Arbitrariness, iconicity and systematicity in language. Trends in Cognitive Sciences, 19, 603–615. doi:10.1016/j. tics.2015.07.013 Dreyfus, H. (1972). What computers can’t do. New York, NY: Harper & Row. Fillmore, Ch. J. (1976). Frame semantics and the nature of language. In: Harnad S.R., Steklis H.D., Lancaster J. (eds.) Origins and Evolution of Language and Speech. New York: The New York Academy of Sciences, (Annals of the New York Academy of Sciences, Vol. 280). Fogel, A., Garvey, A., Hsu, H.-C., & West-Stroming, D. (2006). Change processes in relationships: A relational-historical research approach. New York, NY: Cambridge University Press. Gallagher, S. (2005). How the body shapes the mind. Oxford, UK: Oxford University Press. Gibson, J. J. (1986). The ecological approach to visual perception. Hillsdale, NJ: Erlbaum. (Original work published 1979) Gibson, J. J., & Crooks, L. E. (1938). A theoretical field analysis of automobile- driving. American Journal of Psychology, 51, 453–471. doi:10.2307/1416145 Gómez, J. C. (2007). Pointing behaviors in apes and human infants: A balanced interpretation. Child Development, 78, 729–734. doi:10.1111/j.1467- 8624.2007.01027.x Gratier, M., & Trevarthen, C. (2008). Musical narrative and motives for culture in mother-infant vocal interaction. Journal of Consciousness Studies, 15, 122–158. Haken, H. (1990). Synergetics as a tool for the conceptualization and 55  mathematization of cognition and behavior: How far can we go? In H. Haken & M. Stadler (Eds.), Synergetics of cognition (pp. 2– 31). Berlin, Germany: Springer. Harnad, S. (1990). The symbol grounding problem. Physica D, 42, 335–346. doi:10.1016/0167-2789 (90)90087-6 Heft, H. (1989). Affordances and the body: An intentional analysis of Gibson’s ecological approach to visual perception. Journal for the Theory of Social Behaviour, 19(1), 1–30. doi:10.1111/j.1468- 5914.1989.tb00133.x Heft, H. (2001). Ecological psychology in context: James Gibson, Roger Barker, and the legacy of William James’s radical empiricism. Mahwah, NJ: Erlbaum. Heller, V., & Rohlfing, K. J. (2017). Reference as an interactive achievement: Sequential and longitudinal analyses of labeling interactions in shared book reading and free play. Frontiers in psychology, 8, 139. Hodges, B. H. (2014). Rethinking conformity and imitation: divergence, convergence, and social understanding. Frontiers in Psychology: Cognitive Science, 5, 726. doi: 10.3389/fpsyg.2014.00726 Hodges, B. H., & Baron, R. M. (1992). Values as constraints on affordances: Perceiving and acting properly. Journal for the Theory of Social Behaviour, 22, 263–294. doi:10.1111/j.1468-5914.1992. tb00220.x Hodges, B. H., & Rączaszek-Leonardi, J. (submitted). Values as Constraints on Action, Perception, and Cognition: Theory and Method. Hsu, H. C., & Fogel, A. (2003). Stability and transitions in mother-infant face- to-face communication during the first 6 months: A microhistorical approach. Developmental Psychology, 39(6), 1061. doi:10.1037/0012- 1649.39.6.1061 56 Jensen, T. W., & Pedersen, S. B. (2016). Affect and affordances: The role of action and emotion in social interaction. Cognitive Semiotics, 9(1), 79–103. doi:10.1515/cogsem-2016-0003 Karmiloff-Smith, A. (1992). Beyond modularity: A developmental perspective on cognitive science. Cambridge, MA: MIT Press/Bradford Books. Kaye, K., & Wells, A. J. (1980). Mothers’ jiggling and the burst-pause pattern in neonatal feeding. Infant Behavior and Development, 3, 29–46. doi:10.1016/S0163-6383(80)80005-1 Kelso, J. A. S. (1995). Dynamic patterns: The self-organization of brain and behavior. Cambridge, MA: MIT Press. Köhler, W. (1929). Gestalt psychology. New York, NY: Liveright. Köhler, W. (1947). Gestalt psychology (2nd ed.). New York, NY: Liveright. Leavens, D. A., Sansone, J., Burfield, A., Lightfoot, S., O’Hara, S., & Todd, B. K. (2014). Putting the “joy” in joint attention: Affective-gestural synchrony by parents who point for their babies. Frontiers in Psychology, 5, 879. doi:10.3389/fpsyg.2014.00879 Leonardi, G., Nomikou, I., Rohlfing, K. J., & Rączaszek-Leonardi, J. (2016). Vocal interactions at the dawn of communication: The emergence of mutuality and complementarity in mother-infant interaction: In Proceedings of the IEEE ICDL-EpiRob, Cergy-Pontoise, 288–293. Lock, A. E. (1978). Action, Gesture and Symbol: The Emergence of Language. London: Academic Press. Lock, A. (1980). The guided reinvention of language. London, UK: Academic Press. 57 Lucy, A. J. (1997). Linguistic relativity. Annual Review of Anthropology, 26, 291–312. doi:10.1146/ annurev.anthro.26.1.291 Mace, W. M. (1977). James J. Gibson’s strategy for perceiving: Ask not what’s inside your head, but what your head’s inside of. In R. E. Shaw & J. Bransford (Eds.), Perceiving, acting, and knowing. Hillsdale, NJ: Erlbaum. Merleau-Ponty, M. (1963). The phenomenology of perception (C. Smith, Trans.). London, UK: Routledge and Kegan Paul. Nelson, K. (1974). Concept, word, and sentence: Interrelations in acquisition and development. Psychological review, 81, 267–285. Nomikou, I., Leonardi, G., Radkowska, A., Rączaszek-Leonardi, J., & Rohlfing, K. (2017). Taking up an active role: Emerging participation in early mother-infant interaction during peek-a-boo routines. Frontiers in Psychology, 8, 1656. doi:10.3389/fpsyg.2017.01656 Nomikou, I., Leonardi, G., Rohlfing, K., & Rączaszek-Leonardi, J. (2016). Constructing interaction: The development of gaze dynamics. Infant and Child Development, 25(3), 277–295. doi:10.1002/ icd.1975 Nomikou, I., Leonardi, G., Radkowska, A., Rączaszek-Leonardi, J., & Rohlfing, K. (2017). Taking up an active role: Emerging participation in early mother-infant interaction during peek-a-boo routines, Frontiers in Psychology, Cognitive Science. doi:10.3389/fpsyg.2017.01656 Nomikou, I., & Rohlfing, K. J. (2011). Language does something: Body action and language in maternal input to three-month-olds. IEEE Transactions on Autonomous Mental Development, 3(2),113–128. doi:10.1109/TAMD.2011.2140113 Pattee, H. H. (1969). How does a molecule become a message? Developmental 58  Biology Supplement, 3, 1–16. Pattee, H. H. (1982). Cell psychology: An evolutionary approach to the symbol-matter problem. Cognition and Brain Theory, 5(4), 325–341. Pattee, H. H., & Rączaszek-Leonardi, J. (2012). Laws, language and life. Dordrecht, The Netherlands: Springer. Peirce, C. S. (1931). In C. Hartshorn & P. Weiss (Eds.), Collected papers of Charles Sander Peirce: Vol. II. Elements of logic. Cambridge, MA: Harvard University Press. Perniss, P., Thompson, R. L., & Vigliocco, G. (2010). Iconicity as a general property of language; Evidence from spoken and signed languages. Frontiers in Psychology, 1, 1–17. doi:10.3389/fpsyg. 2010.00227 Piaget, J. (1962). Play, dreams, and imitation in childhood. New York, NY: W. W. Norton. (Original work published 1945) Polanyi, M. (1968). Life’s irreducible structure. Science, 160, 1308–1312. doi:10.1126/science. 160.3834.1308 Rączaszek-Leonardi, J. (2009). Symbols as constraints: The structuring role of dynamics and self-organization in natural language. Pragmatics and Cognition, 17, 653–676. doi:10.1075/pc.17.3.09ras Rączaszek-Leonardi, J. (2016). How does a word become a message? An illustration on a developmental time-scale. New Ideas in Psychology, 42, 46– 55. Rączaszek-Leonardi, J., & Cowley, S. J. (2012). The evolution of language as controlled collectivity. Interaction Studies, 13(1), 1–16. doi:10.1075/is.13.1.01rac 59 Rączaszek-Leonardi, J., & Deacon, T. W. (in preparation). The symbol ungrounding problem. Rączaszek-Leonardi, J., & Kelso, J. A. S. (2008). Reconciling symbolic and dynamic aspects of language: Toward a dynamic psycholinguistics. New Ideas in Psychology, 26, 193–207. doi:10.1016/j. newideapsych.2007.07.003 Rączaszek-Leonardi, J., & Nomikou, I. (2015). Beyond mechanistic interaction: Value-based constraints on meaning in language. Rączaszek-Leonardi˛ J., Nomikou, I., & Rohlfing, K. J. (2013). Young children’s dialogical actions: The beginnings of purposeful intersubjectivity. IEEE Transactions in Autonomous Mental Development, 5, 210–221. doi:10.1109/TAMD.2013.2273258 Rader, N. de Villiers, & Zukow-Goldring, P. (2010). How the hands control attention during early word learning. Gesture, 10, 202–221. Radkowska, A., Nomikou, I., Leonardi, G., Rohlfing, K., & Rączaszek- Leonardi, J. (2017). Scaffolding vocal development: maternal responsiveness to infant speechlike vocalizations at three, six and eight months. Poster at 14th International Congress for the Study of Child Language. Reed, E. S. (1995). The ecological approach to language development: A radical solution to Chomsky’s and Quine’s problems. Language & Communication, 15, 1–29. doi:10.1016/0271-5309(94)E0010-9 Reed, E. S. (1996). Encountering the world: Toward an ecological psychology. New York, NY: Oxford University Press. Richardson, M. J., Marsh, K. L., & Schmidt, R. C. (2010). Challenging egocentric notions of perceiving, acting and knowing. In B. Mesquita, L. F. 60  Barrett, & E. R. Smith (Eds.), The mind in context (pp. 307–333). New York, NY: Guilford Press. Riley, M. A., Richardson, M. J., Shockley, K., & Ramenzoni, V. C. (2011). Interpersonal synergies. Frontiers in Psychology, 2, 38. doi:10.3389/fpsyg.2011.00038 Rohlfing, K. J., & Tani, J. (2011). Grounding language in action. IEEE Transactions on Autonomous Mental Development, 3, 109–112. doi:10.1109/TAMD.2011.2140890 Rohlfing, K. J., Wrede, B., Vollmer, A. L., & Oudeyer, P. Y. (2016). An alternative to mapping a word onto a concept in language acquisition: Pragmatic frames. Frontiers in Psychology, 7, 470. doi:10.3389/fpsyg.2016.00470 Savage-Rumbaugh, E. S., Rumbaugh, D. M., Smith, S. T., & Lawson, J. (1980). Reference: The linguistic essential. Science, 210 (4472), 922-925. Searle, J. R. (1980). Minds, brains and programs. Behavioral and Brain Sciences, 3, 417–424. doi:10.1017/S0140525X00005756 Shockley, K., Santana, M. V., & Fowler, C. A. (2003). Mutual interpersonal postural constraints are involved in cooperative conversation. Journal of Experimental Psychology: Human Perception and Performance, 29(2), 326– 332. Sinha, C. (2009). Language as biocultural niche and social institution. In V. Evans & S. Pourcel (Eds.), New directions in cognitive linguistics (pp 289– 310). Amsterdam, The Netherlands: John Benjamins. Smith, L. B., & Thelen, E. (2003). Development as a dynamic system. Trends in cognitive sciences, 7(8), 343–348. doi:10.1016/j.langsci.2013.08.003 61 Smith, L. B., Colunga, E., & Yoshida, H. (2010). Knowledge as Process: Contextually Cued Attention and Early Word Learning. Cognitive Science, 34(7), 1287–1314. http://doi.org/10.1111/j.1551- 6709.2010.01130.x Steffensen, S. V., & Fill, A. (2014). Ecolinguistics: The state of the art and future horizons. Language Sciences, 41A, 6–25. doi:10.1016/j.langsci.2013.08.003 Szokolsky, A. (2006). Object use in pretend play: Symbolic or functional? In A. Costall & O. Dreier (Eds.), Doing things with things: The design and use of everyday objects (pp. 67–86). New York, NY: Routledge. Szufnarowska, J., & Rohlfing, K. J. (2014). Enfolding interaction with two- month-olds. In: Proceedings of the 16th European Conference on Developmental Psychology, Lausanne, Switzerland. Bologna: Monduzzi Editore, 213–218. Szufnarowska, J., & Rohlfing, K. J. (2014). Enfolding interaction with two- month-olds. In Proceedings of the 16th European Conference on Developmental Psychology. Lausanne, Switzerland, Bologna: Monduzzi Editore, 213–218. Thelen, E. (1985). Developmental origins of motor coordination: leg movements in human infants. Developmental Psychobiology, 18, 1–22. Thelen, E., & Smith, L. B. (1994). A dynamic systems approach to the development of cognition and action. Cambridge, MA: Bradford Books/MIT Press. Tomasello, M. (2003). Constructing a language: A usage-based account of language acquisition. Cambridge, MA: Harvard University Press. Trevarthen, C. (1974). Conversations with a two-month-old. New Scientist, 896, 62  230–235. van Geert, P. (1994). The developing body and mind. Dynamic systems of development: Change between complexity and chaos. Hertfordshire, UK: Harvester Wheatsheaf. Varshavskaya, P. (2002). Behavior-based early language development on a humanoid robot. Proceedings of the Second International Workshop on Epigenetic Robotics: Modeling Cognitive Development in Robotic Systems, 94, 149–158. Walle, E. A., & Campos, J. J. (2013). Infant language development is related to the acquisition of walking. Developmental Psychology, 50, 336–348. doi:10.1037/a0033238 Warlaumont, A. S., Richards, J. A., Gilkerson, J., & Oller, D. K. (2014). A social feedback loop for speech development and its reduction in autism. Psychological Science, 25(7), 1314–1324. doi:10.1177/0956797614531023 Werner, H., & Kaplan, B. (1963). Symbol formation: An organismic developmental approach to language and the expression of thought. New York, NY: Wiley. Wilson, A. D., & Golonka, S. (2013). Embodied cognition is not what you think it is. Frontiers in Psychology, 4, 58. doi:10.3389/fpsyg.2013.00058 Wilson, M. (2002). Six views of embodied cognition. Psychonomic Bulletin and Review, 9, 625–636. doi:10.3758/BF03196322 Wittgenstein, L. (1958). Philosophical investigations. Oxford, UK: Blackwell. (Original work published 1953) Yu, C., & Smith, L. B. (2012). Modeling cross-situational word-referent learning: Prior questions. Psychological Review, 119, 21–39. 63  doi:10.1037/a0026182 Zukow-Goldring, P., & Rader, N. (2001). Perceiving referring actions. Developmental Science, 4, 28–30. Zukow-Goldring, P. (1996). Sensitive caregiving fosters the comprehension of speech: When gestures speak louder than words. Early Development and Parenting, 5, 195–211. doi:10.1002/(SICI)1099-0917(199612)5:4%3c195::AID EDP133%3e3.0.CO;2-H Zukow-Goldring, P. (1990). Socio-perceptual basis for the emergence of language: An alternative to innatist approach. Developmental Psychobiology, 23, 705–726. doi:10.1002/dev.420230711 64