Psychology

Abstract: In describing motion events verbs of manner provide information about the speed of agents or objects in those events. We used eye tracking to investigate how inferences about this verb-associated speed of motion would influence the time course of attention to a visual scene that matched an event described in language. Eye movements were recorded as participants heard spoken sentences with verbs that implied a fast ("dash") or slow ("dawdle") movement of an agent towards a goal. These sentences were heard whilst participants concurrently looked at scenes depicting the agent and a path which led to the goal object. Our results indicate a mapping of events onto the visual scene consistent with participants mentally simulating the movement of the agent along the path towards the goal: when the verb implies a slow manner of motion, participants look more often and longer along the path to the goal; when the verb implies a fast manner of motion, participants tend to look earlier at the goal and less on the path. These results reveal that event comprehension in the presence of a visual world involves establishing and dynamically updating the locations of entities in response to linguistic descriptions of events.

Abstract: Linear mixed-effects models (LMEMs) have become increasingly prominent in psycholinguistics and related areas. However, many researchers do not seem to appreciate how random effects structures affect the generalizability of an analysis. Here, we argue that researchers using LMEMs for confirmatory hypothesis testing should minimally adhere to the standards that have been in place for many decades. Through theoretical arguments and Monte Carlo simulation, we show that LMEMs generalize best when they include the maximal random effects structure justified by the design. The generalization performance of LMEMs including data-driven random effects structures strongly depends upon modeling criteria and sample size, yielding reasonable results on moderately-sized samples when conservative criteria are used, but with little or no power advantage over maximal models. Finally, random-intercepts-only LMEMs used on within-subjects and/or within-items data from populations where subjects and/or items vary in their sensitivity to experimental manipulations always generalize worse than separate F1 and F2 tests, and in many cases, even worse than F1 alone. Maximal LMEMs should be the 'gold standard' for confirmatory hypothesis testing in psycholinguistics and beyond.

Abstract: We analyzed how syntactic flexibility influences sentence production in two different languages, English and Russian. In Study 1, speakers were instructed to produce as many structurally different descriptions of transitive-event pictures as possible. Consistent with the syntactically more flexible Russian grammar, Russian participants produced more descriptions and used a greater variety of structures than their English counterparts. In Study 2, a different sample of participants provided single-sentence descriptions of the same picture materials while their eye-movements were recorded. In this task, English and Russian participants almost exclusively produced canonical SVO-active-voice structures. However, Russian participants took longer to plan their sentences, as reflected in longer sentence onset latencies and eye-voice spans for the sentence-initial Subject noun. This cross-linguistic difference in processing load diminished toward the end of the sentence. Stepwise GLM analyses showed that the greater sentence-initial processing load registered in Study 2 corresponded to the greater amount of syntactic competition from available alternatives (Study 1), suggesting that syntactic flexibility is costly regardless of the language in use.

Abstract: TEST is a novel taxonomy of knowledge representations based on three distinct hierarchically organized representational features: Tropism, Embodiment, and Situatedness. Tropic representational features reflect constraints of the physical world on the agent's ability to form, reactivate, and enrich embodied (i.e., resulting from the agent's bodily constraints) conceptual representations embedded in situated contexts. The proposed hierarchy entails that representations can, in principle, have tropic features without necessarily having situated and/or embodied features. On the other hand, representations that are situated and/or embodied are likely to be simultaneously tropic. Hence while we propose tropism as the most general term, the hierarchical relationship between embodiment and situatedness is more on a par, such that the dominance of one component over the other relies on the distinction between offline storage vs. online generation as well as on representation-specific properties.

Abstract: Three experiments investigated how perceptual, structural, and lexical cues affect structural choices during English transitive sentence production. Participants described transitive events under combinations of visual cueing of attention (toward either agent or patient) and structural priming with and without semantic match between the notional verb in the prime and the target event. Speakers had a stronger preference for active-voice sentences (1) when their attention was directed to the agent, (2) upon reading an active-voice prime, and (3) when the verb in the prime did not match the target event. The verb-match effect was the by-product of an interaction between visual cueing and verb match: the effect of agent-cueing was diminished when the prime verb matched the target event. Persistence of visual cueing effects in the presence of both structural and lexical cues suggests a direct mapping from referent-directed visual attention onto the assignment of grammatical roles in a spoken sentence.

Abstract: In human communication, direct speech (e.g., Mary said, "I'm hungry") is perceived as more vivid than indirect speech (e.g., Mary said that she was hungry). This vividness distinction has previously been found to underlie silent reading of quotations: Using functional magnetic resonance imaging (fMRI), we found that direct speech elicited higher brain activity in the temporal voice areas (TVA) of the auditory cortex than indirect speech, consistent with an 'inner voice' experience in reading direct speech. Here we show that listening to monotonously spoken direct versus indirect speech quotations also engenders differential TVA activity. This suggests that individuals engage in top-down simulations or imagery of enriched supra-segmental acoustic representations while listening to monotonous direct speech. The findings shed new light on the acoustic nature of the 'inner voice' in understanding direct speech.

Abstract: In human communication, direct speech (e.g. Mary said: "I'm hungry") is perceived to be more vivid than indirect speech (e.g. Mary said [that] she was hungry). However, the processing consequences of this distinction are largely unclear. In two experiments, participants were asked to either orally (Experiment 1) or silently (Experiment 2, eye-tracking) read written stories that contained either a direct speech or an indirect speech quotation. The context preceding those quotations described a situation that implied either a fast-speaking or a slow-speaking quoted protagonist. It was found that this context manipulation affected reading rates (in both oral and silent reading) for direct speech quotations, but not for indirect speech quotations. This suggests that readers are more likely to engage in perceptual simulations of the reported speech act when reading direct speech as opposed to meaning-equivalent indirect speech quotations, as part of a more vivid representation of the former.

Abstract: We investigated how conceptually informative (referent preview) and conceptually uninformative (pointer to referent's location) visual cues affect structural choice during English transitive sentence production. Cueing the Agent or the Patient prior to presenting the target event reliably predicted the likelihood of selecting this referent as the sentential Subject, triggering, correspondingly, the choice between active and passive voice. Importantly, there was no difference in the magnitude of the general Cueing effect between the informative and uninformative cueing conditions, suggesting that attentionally driven structural selection relies on a direct automatic mapping mechanism from attentional focus to the Subject's position in a sentence. This mechanism is, therefore, independent of accessing semantic, and possibly lexical, information about the cued referent provided by referent preview.

Abstract: In human communication, direct speech (e.g. Mary said: "I am hungry") is perceived to be more vivid than indirect speech (e.g. Mary said [that] she was hungry). However, for silent reading, the representational consequences of this distinction are still unclear. While many of us share the intuition of an "inner voice" particularly during silent reading of direct-speech statements in text, there has been little direct empirical confirmation of this experience so far. Combining functional magnetic resonance imaging (fMRI) with eye-tracking in human volunteers, we show that silent reading of direct versus indirect speech engenders differential brain activation in voice-selective areas of the auditory cortex. This suggests that readers are indeed more likely to engage in perceptual simulations (or spontaneous imagery) of the reported speaker's voice when reading direct speech as opposed to meaning-equivalent indirect speech statements, as part of a more vivid representation of the former. Our results may be interpreted in line with embodied cognition and form a starting point for more sophisticated interdisciplinary research on the nature of auditory mental simulation during reading.

Abstract: In some theories of sentence comprehension, linguistically relevant lexical knowledge, such as selectional restrictions, is privileged in terms of the time-course of its access and influence. We examined whether event knowledge computed by combining multiple concepts can rapidly influence language understanding even in the absence of selectional restriction violations. Specifically, we investigated whether instruments can combine with actions to influence comprehension of ensuing patients of (as in Rayner, Warren, Juhuasz, & Liversedge, 2004; Warren & McConnell, 2007). Instrument-verb-patient triplets were created in a norming study designed to tap directly into event knowledge. In self-paced reading (Experiment 1), participants were faster to read patient nouns, such as hair, when they were typical of the instrument-action pair (Donna used the shampoo to wash vs. the hose to wash). Experiment 2 showed that these results were not due to direct instrument-patient relations. Experiment 3 replicated Experiment 1 using eyetracking, with effects of event typicality observed in first fixation and gaze durations on the patient noun. This research demonstrates that conceptual event-based expectations are computed and used rapidly and dynamically during on-line language comprehension. We discuss relationships among plausibility and predictability, as well as their implications. We conclude that selectional restrictions may be best considered as event-based conceptual knowledge rather than lexical-grammatical knowledge.

Abstract: In the two experiments reported here, we uncovered evidence for shared structural representations between arithmetic and language. Specifically, we primed subjects using mathematical equations either with or without parenthetical groupings, such as 80 - (9 + 1) * 5 or 80 - 9 + 1 * 5, and then presented a target sentence fragment, such as "The tourist guide mentioned the bells of the church that . . . ," which subjects had to complete. When the mathematical equations were solved correctly, their structure influenced the noun phrase - for example, either "the bells of the church" or "the church," respectively - that subjects chose to attach their sentence completion to. These experiments provide the first demonstration of cross-domain structural priming from mathematics to language. They highlight the importance of global structural representations at a very high level of abstraction and have potentially far-reaching implications regarding the domain generality of structural representations.