How good are current models for evaluating the cultural loading and linguistic complexity of individual tests in individually administered intelligence batteries?
To date, the most visible work has been based on the Flanagan, Ortiz et al. groups cross-battery work and their presentation of the Culture-Language Interpretive Matrix (C-LIM) in which individual tests in IQ batteries are categorized in terms of their perceived linguistic demand and cultural loading.
I've always believed that the C-LIM made logical and theoretical sense, but was in sore need of some empirical research evidence support.
Previously I presented an attempt by myself and Jeff Evans to quantify the linguistic demands of individually administered tests. That research effort was conducted in the spirit of stimulating others to attempt more sophisticated methods for validating the quantification of the two dimensions of the C-LIM. That post and unpublished research report is available here. Until my attendance at NASP 2009 in Boston this past week, I was unaware of any other empirical attempts to investigate the validity of this model (for example, I've routinely searched the ProQuest digital dissertation abstract service for any CHC, XBA, etc. related dissertations, and to date have not found one that has investigated this matrix-- check out those found at WMF Dissertation Abstracts Project). [Note - if such disserations exist and I've missed them, please contact me to rectify my lack of knowledge and I'll make the appropriate post]
At NASP, John Kranzler and students presented a poster (click here for single page PPT image of poster) (albeit based on a small n---thus caution is urged in generalizing from the results) that investigated the C-LIM classifications of certain Woodcock-Johnson III (WJ III) tests [conflict of interest disclosure - I'm a coauther of the WJ III). The purpose of Kranzler et al's study "was to empirically examine the relationship between English-language proficiency and linguistic demands on the cognitive test performance of a sample of culturally and linguistically diverse children in the English for Speakers of Other Languages (ESOL) program". The authors concluded that:
Results of this study for an ESOL sample do not follow the predicted pattern based on the C-LIM for the WJ-III. It is important to note that this is a small sample and replication is needed. Nonetheless, these results do not support the use of the C-LIM. These results are consistent with the general conclusion that, on IQ tests, there is very little evidence to suggest that any of the cognitive factor, scale, or subtest profile differences can be used to improve decisions about individuals.
Of course, an alternative explanation is that the C-LIM model may be correct/valid but the Flanagan, Ortiz et al. classifications of some of the WJ III tests are not accurate.
We need more research on this matrix and related interpretations. It is nice to see that some folks are attempting to do this. Kudos to Kranzler and his students for adding a small piece of empircal data to this literature.
Technorati Tags: psychology, educational psychology, school psychology, neuropsychology, special education, learning disabilities, LD, cross-battery, XBA, CHC, Cattell-Horn-Carroll, WJ III, WJ III NU, cultural oading, linguistic complexity