Relationships between Bloom’s taxonomy, judges’ estimation of item difficulty and psychometric properties of items from a progress test: A prospective observational study

Página do item simplificado
dc.contributor.authorHamamoto Filho, Pedro Tadao [UNESP]
dc.contributor.authorSilva, Eduardo
dc.contributor.authorRibeiro, Zilda Maria Tosta
dc.contributor.authorHafner, Maria de Lourdes Marmorato Botta
dc.contributor.authorCecilio-Fernandes, Dario
dc.contributor.authorBicudo, Angélica Maria
dc.contributor.institutionUniversidade Estadual Paulista (Unesp)
dc.contributor.institutionEdudata Informática
dc.contributor.institutionFaculdade de Medicina de Marília (FAMEMA)
dc.contributor.institutionUniversidade Estadual de Campinas (UNICAMP)
dc.date.accessioned2020-12-12T02:03:24Z
dc.date.available2020-12-12T02:03:24Z
dc.date.issued2020-01-01
dc.description.abstractBACKGROUND: Progress tests are longitudinal assessments of students’ knowledge based on successive tests. Calibration of the test difficulty is challenging, especially because of the tendency of item-writers to overestimate students’ performance. The relationships between the levels of Bloom’s taxonomy, the ability of test judges to predict the difficulty of test items and the real psychometric properties of test items have been insufficiently studied. OBJECTIVE: To investigate the psychometric properties of items according to their classification in Bloom’s taxonomy and judges’ estimates, through an adaptation of the Angoff method. DESIGN AND SETTING: Prospective observational study using secondary data from students’ performance in a progress test applied to ten medical schools, mainly in the state of São Paulo, Brazil. METHODS: We compared the expected and real difficulty of items used in a progress test. The items were classified according to Bloom’s taxonomy. Psychometric properties were assessed based on their taxonomy and fields of knowledge. RESULTS: There was a 54% match between the panel of experts’ expectations and the real difficulty of items. Items that were expected to be easy had mean difficulty that was significantly lower than that of items that were expected to be medium (P < 0.05) or difficult (P < 0.01). Items with high-level taxonomy had higher discrimination indices than low-level items (P = 0.026). We did not find any significant differences between the fields in terms of difficulty and discrimination. CONCLUSIONS: Our study demonstrated that items with high-level taxonomy performed better in discrimination indices and that a panel of experts may develop coherent reasoning regarding the difficulty of items.en
dc.description.affiliationDepartment of Neurology Psychology and Psychiatry Universidade Estadual Paulista (UNESP)
dc.description.affiliationEdudata Informática
dc.description.affiliationFaculdade de Medicina de Marília (FAMEMA)
dc.description.affiliationDepartment of Medical Psychology and Psychiatry Universidade Estadual de Campinas (UNICAMP)
dc.description.affiliationDepartment of Pediatrics Universidade Estadual de Campinas (UNICAMP)
dc.description.affiliationUnespDepartment of Neurology Psychology and Psychiatry Universidade Estadual Paulista (UNESP)
dc.format.extent33-39
dc.identifierhttp://dx.doi.org/10.1590/1516-3180.2019.0459.r1.19112019
dc.identifier.citationSao Paulo Medical Journal, v. 138, n. 1, p. 33-39, 2020.
dc.identifier.doi10.1590/1516-3180.2019.0459.r1.19112019
dc.identifier.fileS1516-31802020000100033.pdf
dc.identifier.issn1516-3180
dc.identifier.scieloS1516-31802020000100033
dc.identifier.scopus2-s2.0-85083718509
dc.identifier.urihttp://hdl.handle.net/11449/200314
dc.language.isoeng
dc.relation.ispartofSao Paulo Medical Journal
dc.rights.accessRightsAcesso aberto
dc.sourceScopus
dc.subjectEducational measurements
dc.subjectMedical education
dc.subjectPsychometrics
dc.titleRelationships between Bloom’s taxonomy, judges’ estimation of item difficulty and psychometric properties of items from a progress test: A prospective observational studyen
dc.typeArtigo
unesp.author.orcid0000-0001-6436-9307[1]
unesp.author.orcid0000-0003-4489-1610[2]
unesp.author.orcid0000-0001-7368-3154[3]
unesp.author.orcid0000-0002-5471-4434[4]
unesp.author.orcid0000-0002-8746-1680[5]
unesp.author.orcid0000-0003-3043-5147[6]

Arquivos

Pacote Original
Agora exibindo 1 - 1 de 1
Imagem de Miniatura
Nome:
S1516-31802020000100033.pdf
Tamanho:
275.72 KB
Formato:
Adobe Portable Document Format
Baixar

Coleções

Artigos - Neurologia, Psicologia e Psiquiatria - FMB