Dialect Corpus Thematic Markup: The Experience of Tomsk Dialectologists

The article discusses an urgent problem in the field of corpus linguistics: the implementation of text markup by topic. It presents the experience of implementation of such a markup in the Tomsk dialect corpus. The research team has achieved impressive results: at the moment, the Tomsk dialect corpus contains 1,600 texts, divided into more than 20,000 thematic fragments, and is the most representative resource of this kind in Russia. The authors of the article interpret the practical experience of Tomsk researchers in a broad context against the background of decisions made by the developers of the Russian National Corpus and the emerging Russian dialect corpora. The authors identify factors that give rise to difficulties in the implementation of thematic markup of dialect texts by subject areas. The factors include: the oral nature of dialect communication (thus, a close connection of the text with the situation of its generation, overlapping or intersection of themes); in some cases, a weak degree of coherence of texts due to the peculiarities of fixing the material, difficulties in understanding the texts of local culture “from outside”, lack of a unified methodology for thematic markup. An analysis of the available developments in the field of creating regional corpora makes it possible to identify general techniques used in practice. The techniques include: manual thematic markup; hierarchy of the thematic list, which generally includes two levels of generalization; markup of the topic of separate text fragments, not the text as a whole; use of “soft” markup with the ability to assign several thematic labels to the same fragment and partial overlap of text fragments. The developers of the Tomsk dialect corpus propose specific methodological steps to implement thematic markup. The markup includes 3 stages: the person doing the markup (1) intuitively breaks the text into fragments united by a common content and determines these fragments’ boundaries preparatively; (2) determines the keywords of the fragment (based on the lists of keywords from the instructions) and, in some cases, the semantic dominant of the text; (3) identifies the final boundaries of the fragment and the choice of a topic from the available list. The list of topics for the markup in the Tomsk dialect corpus currently includes 77 items; it is not exhaustive and is gradually updated in the course of work. The potential content of the texts on each topic and the thematic belonging of the “controversial” fragments are determined as a result of group discussions. The user of the corpus can learn the details of these discussions by referring to the instructions posted on the website. The authors also briefly describe the technical side of thematic markup, provide samples of marked-up text fragments. The presented experience can be applied to create other corpus resources and used in the field of theoretical studies of dialect speech from the standpoint of discourse analysis.

Keywords

Authors

References