Evaluation methods of quantity influence and task complexity of b-group on the result accuracy of unified state examination

Today the test being used in the unified state examination includes tasks of B and C groups. The tasks of A group involving the choice of a correct answer version were excluded from the test few years ago because of low level of complexity. Tasks of B and C groups have numerical answers which are almost impossible to guess. The difference between these groups is that the level of complexity of B group is lower than that of C group. Therefore, the check of B group results is carried out by means of the final numerical answers electronically (via the computer). To solve the task of C group the final numerical answer is not enough, it requires detailed text description of the solution. These tasks are checked by experts trained for this purpose. The article describes the three main methods of estimation of qualification levels of students passing the Unified State Examination. There are two direct methods: a method of scaling which is used in the exam nowadays and a modified scaling method proposed by the author in his previous works. These methods allow us to convert a primary score of students to a scale of percentage logits which characterize a qualification level of the participant. The third method investigated in this work was proved by the author in previous works, the method of initial points. This method is an indirect method that makes it possible to calculate the evaluation of participants’ qualification levels in two steps. The first step consists in obtaining assessments of latent parameters of qualification levels measured in logits. The second step consists in converting logits to percentage logits. After describing the methods of estimates obtaining of qualification levels of testing participants the article highlights the main stages of simulation of testing process. The simulation uses the well-known theory of G. Rasch, a mathematical model of testing. It discusses some elements of a computer program performing a simulation of testing. The result of the work of this program is diagrams and tables that show the character of dependence of accuracy of the three methods on the number and complexity of tasks of B group. On the basis of the material obtained the author makes specific conclusions and gives recommendations aimed at improving in accuracy of exam estimations. 1) to estimate the results of the unified state exam, it is recommended to use the method of initial points whose accuracy is 1.5 - 2 times higher than that of the scaling method which is used in the exam nowadays. 2) When developing the examination test in which the scaling method is used, it is recommended to use 12 tasks of B group, the difficulty of which is determined by 2 logits; if the method of initial points is used, it is recommended to use the maximum acceptible number of tasks with a complexity level ranging (0,4; 0,8). 3) If 15 tasks of B group are used, it is recommended to set the following levels of difficulty for these tasks: for scaling method - (-1,5: -0.6), for the method of initial points (-1,8; 0), for the modified scaling method - (-!.5; -0,3).

Keywords

Authors

References