Construct validity of scores/measures from a developmental assessment in mathematics using classical and many-facet Rasch measurement

Area of Science:

• Educational Measurement
• Psychometrics
• Mathematics Education

Background:

• Developmental assessments are crucial for tracking student progress.
• Validating mathematics assessments ensures accurate measurement of learning.
• Rasch measurement offers robust methods for analyzing assessment data.

Purpose of the Study:

• To validate a mathematics assessment using classical and Rasch measurement.
• To examine the psychometric properties of the assessment's structure and reliability.
• To identify areas for improvement in task construction based on measurement models.

Main Methods:

• Classical test theory and three-faceted Rasch measurement were employed.
• Analysis of field test data from 280 elementary school students.
• Confirmatory Factor Analysis (CFA) and reliability analyses (Cronbach's alpha, inter-rater reliability).

Main Results:

• Descriptive statistics indicated improved performance with age.
• A two-factor model (CFI=.94) fit the data better than a one-factor model (CFI=.87).
• Convergent validity coefficients ranged from .28 to .47; reliability coefficients ranged from .87 to .89.

Conclusions:

• The assessment demonstrated acceptable reliability and validity.
• The Rasch model analysis revealed an ordered scale structure but high task difficulty.
• Gaps in the Rasch item map suggest the need for additional task development.