The influence of intermodal conflict on perception of verbal stimuli

This paper deals with the problem of multimodal perception of verbal signs. Human perception is multimodal. The study of multimodal texts and multimodal perception of words is a step towards the study of multimodal perception in general. The author conducted a series of experiments aimed at the study of the competition of the audial and visual channels while homogeneous verbal information is being presented in the condition of modality conflict. The author pursued the following research objectives: to study characteristics of bimodal perception of verbal stimuli in conflict modalities; to identify the impact of conflict "background" information on the perception of information extracted from the field of voluntary attention; to select a leading modality in bimodal (audial-visual) perception. The research hypothesis is that when the voluntary attention is focused on verbal information оf one of the modalities (visual), information acquired automatically via the second modality (audial) will affect the perception of the former original modality as well as the solution of the tasks related to this original modality. The experiment method was selected as the leading research method. The experimental series consisted of two independent experiments. The main experiment presupposed parallel bimodal presentation of words that contain identical or conflicting information in animateness category, and consisted of preparatory (pretest) and primary stages. The second experiment aimed at identifying the dominant modality in the multimodal perception of verbal stimuli. Visual modality proves to be a leading modality for the perception of verbal information. In other words, modality has a leading role in the modality - type of the information opposition in case of multimodal perception. The experimental data show that the existing functional asymmetry of intermodal verbal perception (visual modality playing a leading role) does not depend on the type of category (no interaction between factors). Therefore, this result should be replicable in terms of other semantic factors. However, the difference in speed during the perception of audio-visual pairs with conflict and identical information (mismatch pairs are perceived significantly faster) shows significant effects of type of information on perception: the background information (which sounds outside voluntary attention of respondents) accelerates or slows down the perception of information from the field of voluntary attention. Thus, it is possible to speak about some cognitive processes, accompanying the perception of information which are not determined by a leading modality.

Keywords

Authors

References