차별문항기능에 대한 IRT에 기초한 문항적합도 지수의 민감도 분석

Abstract

This study examined five Rasch-model-based item fit indices: unweighted and weighted standardized indices (denoted UWz and Wz), standardized likelihood index (denoted Lz), Extended Caution Indices (denoted ECI2z and ECI4z), in terms of their distributional properties and power of detecting item bias. The procedures utilized in this study involve simulated data. The procedures to conduct simulation of item bias were as follows; First, two groups (reference vs. Focal) of 500 persons were generated using SIMTEST computer program, each with a mean ability of zero and a standard deviation of one logit. The number of items was set at 60. The item difficulties for the reference and focal group had uniform distribution with a standard deviation of one logit. Second, in order to simulate item bias, responses for the focal group were generated with item difficulties 0.5 and 1.0 logits more difficult than the item difficulties used with the reference group. The number of items that contain the bias was varied (10,15, and 20 out of 50). The two groups were then combined for the analysis. Third, to compute the probabilites of observed scores, the ability and difficulty parameters were estimated by the BIGSTEPS computer program for each examinee and item. And then, the IRT parameters were used to calculate unweighted and weighted standardized fit index using BIGSTEPS computer program, standardized likelihood index, ECI2z, and ECI4z using IFIT computer program for the item-fit indices. The results indicated that althrough these five standardized item-fit indices did not depart significantly from a normal distribution, it appeared that the Type I error rates were not resonable. For the power of five standardized item-fit indices to detect item bias, the result showed that all indices did perform poorly across various conditions. Therefore, it is possible to conclude that all indices that were used in this study are inadequate measures of fit for detecting item bias. With respect to the UWz or Wz index, these findings are not unexpected because Smith(1991a) already reported that total fit index was ineffective index for detecting item bias. In this context, it is interesting to note another fit index called "between fit index". The between fit index is only available in the IPARM computer program (Smith,1991b). The most distinctive feature of between fit index,claimed by Smith(1994a), is that because this fit index is based on relevant criterion subgroups, this makes it possible to divide total groups into subgroups, such as sex or race, to detect subgroups differences usually defined as bias. Smith(1994a) also demonstrated that between fit index is an useful tool to detect systemic aberrant response patterns. In conclusion, another fit measures for item bias analysis (e.g., between fit index) should be apply or ,if necessary, developed under the Rasch model.