Wang, C., Moss, J. D., Wiles, B. C., Li, Q., Lu, Y., Cho, H. J., & Levesque-Bristol, C.
Manuscript submitted for publication.
The Calculus Concept Inventory (CCI) is a popular test of conceptual understanding of differential calculus. However, the CCI is lengthy, and studies documented inconsistent findings regarding the reliability and validity of the instrument. Given that item response theory (IRT) has advantages over classical test theory when analyzing concept inventory data, we used IRT to validate a short version of the CCI (S-CCI) with a large sample size (N = 1,188). We first examined a set of measurement parameters including item difficulty, discrimination, and correct-guessing chances to evaluate the psychometric characteristics of the S-CCI. We found that the S-CCI is sufficiently capable of assessing students’ conceptual understanding of calculus, especially for students with high-level abilities. Second, we compared the effectiveness of using IRT-estimated abilities and normalized gain as measures of learning gains by examining the associations between both measures of gains and students’ learning outcomes. Our findings showed that the IRT-estimated gain is a stronger predictor of academic outcomes than the normalized gain. Third, we compared the IRT-estimated gains between interactively-engaged approach and lecture-based approach. The result indicated that students in the interactively-engaged sections achieved greater gains than the students in the lecture-based sections.
Keywords: item response theory, Calculus Concept Inventory, psychometric evaluation, normalized gain, active learning