ANALISYS OF PATTERNS IN MATHEMATICAL TESTS

When analyzing the results of testing, along with determining the level of knowledge of students, the structure of the available knowledge is in interest. To determine the degree of systematic or fragmentary knowledge of examinees, Gutmann’s patterns are used in the work. When assessing test items in a two-point scale, the result of the test for each student can be written in the form of an ordered sequence of zeros and ones, that is called the pattern of the examinee. Each pattern is associated with a number, called the pattern regularity. This number characterizes the degree of ordering of student’s knowledge and the deviation of the knowledge structure of the individual examinee from the average of a sample. An ordered structure of knowledge assumes that the examinee correctly solves simple items up to a certain level of complexity, then they can not solve more complicated items. If the student has correctly solved several difficult items, but could not find the right answer in simple items, then the structure of his knowledge is fragmentary. The distribution of pattern regularity is studied by means of a statistical analysis on the basis of test results of the students of National Research Moscow State University of Civil Engineering in two topics of mathematical analysis. To test and process its results, the authors developed test items banks and a special computer program that makes it possible to generate rapidly the unique individual test variants. All items of both tests were offered in an open form, i.e. without a list of possible answers. Thus, during testing students are deprived of the opportunity to write off or guess the answer. A new concept of ideal patterns corresponding to the existing set of examinee patterns is introduced. It is assumed that all test items of the ideal test are independent, and the probability of finding the correct answer for each item of the ideal test coincides with the probability of determining the correct answer found from the sample of the students being tested. Each possible ideal pattern is compared with the probability of its appearance, equal to the product of the probabilities of choosing the right and wrong answers in this pattern. It is shown that ideal and test patterns are statistically close. The introduction of ideal patterns enables to approximate a random sample of a limited number of patterns obtained from the test results with a complete set of all possible test patterns with given probabilities of their appearance. This opens up wide opportunities for studying the distribution of patterns by methods of mathematical statistics. The hypothesis about beta-distribution of regularity of test and ideal patterns with close values of distribution parameters was considered and proved. The hypothesis was tested using the Pearson’s criterion. Based on the analysis of distributions, a comparison is made between the fragmentation levels of students’ knowledge in two topics of mathematics. It is shown that the structure of knowledge of examinees in the topic “Differential calculus” is more fragmented than in the topic “Introduction to Analysis”.

Keywords

Authors

References