Scale Construction Study based on Item Response Theory: Evaluation of Proficiency in Basic Mathematical Contents

Abstract

This article presents a study on scale construction, based on the Item Response Theory (IRT), to measure the proficiency in basic mathematical contents, which are key to the follow-up of Calculus and similar subjects, for those entering courses in the Exact Sciences area. The one-dimensional logistic model with three parameters was adopted, which establishes zero as the mean and a standard deviation of 1, for individuals’ proficiencies. The estimated proficiencies were transformed in another scale, opting for values adopted by Brazilian evaluation systems: 250 and 50. The measurement instrument consisted of a test with 36 items, with five alternatives each, only one of them correct, that were elaborated based on a reference matrix, divided into three themes, “Space and Form”, “Quantities and Measures”, and “Numbers and Operations, Algebra and Functions”. Each subject is composed of competencies, which describe the skills to be measured. To build the scale, proficiency levels were specified, representing points selected by the researchers to be pedagogically interpreted. Once the anchor levels are established, anchor items were defined based on some criteria, such as the number of correct answers, the percentage of correct answers and the difference between their values, for consecutive levels. Based on these criteria, three methods of items’ positioning were compared, showing the difficulties of interpretation in points of the scale. Such difficulties made it possible to propose another method, segmenting the scale into ranges of proficiency, based on hierarchical groupings of levels, which allowed the scale to be interpreted in all its breadth.