On multimodal perception of the text (a psycholinguistic experiment)

The paper focuses on the main results of research in the field of multimodal perception of the texts. This research sphere is one of the most urgent in modern cognitive science. The term multimodality is associated with the psychological term modality as belonging to a particular sensory system. It is associated with the actualization of several perceptual channels, and therefore attention is focused on the study of the respondent's perception or these texts. The author offers her solution to the "competing modalities" in the bi-modal perception of the verbal information. The author takes into account the "hypothesis of modal expediency", according to which the contribution of different sensory modalities in a multimodal perception is linked to the type of the perceived stimulus. Psycholinguistic experiment was selected as the leading research method. It consists of two stages. The experiment is based on the hypothesis that the perception of the text with simultaneous updating of two modalities they will find a functional asymmetry, and the character of this asymmetry will be correlated with the type of the implied information. The correlation factors are: the type of modality (audial vs visual) and the type of the stimulus-text, varying according to the style features (scientific and poetic information). Texts have different cognitive essence. This texts were shown simultaneously in different modalities: visual and audial. At each stage of the experiment texts modalities interchanged. I further analysed the amount of displayed information on each text of each modality in comparison with the source texts. All phases of the experiment showed the presence of functional asymmetry of perception channels. The visual modality has become the dominant modality for each type of text. This was shown by analysing the amount of displayed information. In the analysis of representations I revealed two types of reproduction of the information. They were parallel insertion (~ 11%) - reaction reproducing the poetic and scientific material in turn and the playback sequence (~ 89%) in which first comes the information in one modality, then - in the other. The main feature of the representation types of information was revealed in the analysis of responses to the playback sequence. The reaction sequence scientific -poetic text has been the most frequent (~ 76% of all reactions of this type). This sequence is not determined by the fact of the leading modality, therefore, depends on some cognitive processes.

Keywords

Authors

References