Development of qualified items for nursing education assessment: The progress testing experience

Abstract

Aim: To analyze the psychometric characteristics of items in the nursing inter-institutional progress testing for the years 2019, 2021, 2022 and 2023. Background: Progress testing is a validated method for evaluating professional undergraduate education, aimed at identifying knowledge gain in a continuous and progressive manner, with potential benefits for nursing education. However, for its results to be useful, the evaluation items used in the test must have good psychometric performance. Design: A cross-sectional study. Methods: A sample of 377 items (multiple-choice questions) was applied to 4678 students in four years of progress testing. The difficulty and discrimination indexes were analyzed using descriptive statistics, ANOVA and simple linear regression. Results: The average difficulty index of the test items ranged between 0.39 and 0.46. The areas of child and adolescent health, women's health and adult health had the most difficult items, while the areas of management, mental health and public health had the least difficult items. Discrimination index ranged from 0.35 to 0.43. There was a difference between discrimination index between the years of application (p < 0.001), with a significant increase in the discrimination index (p < 0.001) in the trend analysis. Students in the final years showed lower levels of difficulty and discrimination when compared with students in the initial years, demonstrating that the test is easier and there is less dispersion of performance among students in the final years. Conclusions: The items are not difficult and have good discrimination. A gradual annual increase in the discrimination index of the items was observed. This study provides useful information for the psychometric analysis and quality assurance of knowledge assessment items, both for the implementation of similar PT experiences and in the use of multiple-choice questions for other knowledge assessment purposes.