The evaluation of the proximity of text categories for solving electronic documents classification tasks

The article deals with the problem of classification of texts by the example of their assignment to a particular age group of recipients. In practice, texts for one age category of recipients can also be considered as addressed to another age when they conditionally correspond to the levels of their communicative development. In this case, we can discuss the magnitude of the differences between text categories. In previous research authors have formulated an approach to mathematical modeling of the problem of classification. Suppose given a text T and a set of categories K = {K!,K ₂,^,K _B}. Need to find a subset of K , i.e., a category, which may be associated with the text: T ~ Kj , Kj = {Kj : T ~ K }, where i = jij2,---j _m and 1 < i < n. So, the category Ki can be presented as K ={qK,wK}, j = й, where q j is classification feature, w j is a weight coefficient of classification feature, L is a total count of classification features. If the feature descriptions of the two texts are identical, we call these texts belonging to the same taxonomic rank, and then we have Т. = T- This is an equivalence relation because it satisfies the conditions of reflexive, symmetric, and transitive. Consequently, many texts can be divided into disjoint equivalence classes and one can construct factor set by the equivalence relation. Texts having an equivalence class are carriers of one of the feature descriptions, which allows us to consider they are equivalent. Converting the original set of texts in the factor set is the process of forming the contents of the classes of texts addressed to a specific age audience. In fact, as mentioned above, we are talking about a certain quantifying the differences between the categories of texts with no matching feature descriptions and belonging to different classes of equivalence: р(К,Kj ) = 0 о К = К1. Measure of proximity of categories may be defined as the Mahalanobis distance because the features of objects are statistically dependent and their relevance is determined by the weight coefficient s. Then, the distance between the categories of K _i and Kj represented as vectors characterizing their classification features к = (q ,q*,...,q ), j = (q^ q^^ qK ^ are defined as р(( , Kj ) = V ( К - Kj ) A cat QaU К - Kj ) , where A _cat is a matrix of weight coefficients, C _cat is a matrix of covariance, which is the matrix built by pairwise covariance of the elements in vectors К and Kj. Pairwise covariance of features values for vectors K _i and Kj is: cov ((, qi )=1 z(q t - q )(qKt - qi)), _к 1 к к 1 к where q _i = - Z (q _in ) , qj = Z (qj _n ) are average values in the text samples, n = 1,.. .L. nt=1 nt=1 The proposed approach to the evaluation of the proximity of categories texts is implemented due to the development of the prototype of software system for automatic classification of texts in Russian based on age categories of recipients.

Keywords

Authors

References