FAA Office of Aerospace Medicine
Civil Aerospace Medical Institute
PUBLICATIONS
AVIATION MEDICINE REPORTS

Report No: DOT/FAA/AM-01/4

Title and Subtitle: Latent Trait Theory Analysis of Changes in Item Response Anchors

Report Date: February 2001

Authors: Farmer, W.L., Thompson, R.C., Heil, S.K.R., and Heil, M.C.

Abstract: The purpose of this study was to evaluate the effect that modifications in item response anchors have on responses to survey items. Twenty-nine items were administered in 1993 and 1995 as part of more extensive attitude surveys to two random samples of Federal Aviation Administration employees. Changes in the response scales (5-point Likert) between the two survey administrations ranged from no change at all to extensive re-anchoring of the response categories. Item responses were modeled via two-parameter graded response models based on item response theory. Changes in the way the item responses functioned between both years were assessed using the differential item functioning (DIF) method recommended by Muraki (1997).

Twenty-four of the 29 items displayed significant levels of DIF, indicating that the response categories did not measure the constructs of interest in a similar fashion across the two administrations. Items whose response anchors had been changed substantially exhibited significant DIF more frequently than those where the change in anchors was less drastic. These results suggest that researchers and practitioners take a conservative approach when considering the revision of measuring scales for a particular set of items.

Key Words: Item Anchors, Latent Trait Theory, Item Response Theory, Differential Item Functioning, Measurement Equivalence

No. of Pages: 18