Showing posts with label CLIM. Show all posts
Showing posts with label CLIM. Show all posts

Tuesday, November 22, 2016

Research Bytes: A Systematic Examination of the Linguistic Demand of Cognitive Test Directions Administered to School-Age Populations

A Systematic Examination of the Linguistic Demand of Cognitive Test Directions Administered to School-Age Populations

  1. Damien C. Cormier1
  2. Okan Bulut1
  3. Deepak Singh1
  4. Kathleen E. Kennedy1
  5. Kun Wang1
  6. Alethea Heudes1
  7. Adam J. Lekwa2
  1. 1University of Alberta, Edmonton, Canada
  2. 2Rutgers University, New Burnswick, NJ, USA
  1. Damien C. Cormier, Department of Educational Psychology, University of Alberta, 6-107E Education North, Edmonton, Alberta, Canada T6G 2G5. Email: dcormier@ualberta.ca

Abstract

The selection and interpretation of individually administered norm-referenced cognitive tests that are administered to culturally and linguistically diverse (CLD) students continue to be an important consideration within the psychoeducational assessment process. Understanding test directions during the assessment of cognitive abilities is important, considering the high-stakes nature of these assessments. Therefore, the linguistic demand of spoken test directions from the following commonly used cognitive test batteries was examined and compared: Wechsler Intelligence Scale for Children, Fifth Edition (WISC-V), Woodcock–Johnson IV Tests of Cognitive Abilities (WJ IV COG), Cognitive Assessment System, Second Edition (CAS2), and Kaufman Assessment Battery for Children, Second Edition (KABC-II). On average, the linguistic demand of the standard test directions was greater than the linguistic demand of the supplementary test directions. When examining individual test characteristics, very few individual tests were identified as outliers with respect to the linguistic demand of their test directions. This finding differs from previous research and suggests that the linguistic demand of the required directions for most tests included in commonly used cognitive batteries is similar. Implications for future research and test development are discussed

Saturday, June 07, 2014

More research on the validity of the C-LIM framework in intelligence testing

Another article adding to the Cultural-Linquistic Interpretive Matrix research literature. Click on images to enlarge. A copy of the article can be found here.

"Conclusion

The primary conclusion drawn from this study and previous research is that linguistic demand is an important consideration when selecting and interpreting tests of cognitive abilities. The implications of this study go beyond a re-classification of the C-LIM to emphasizing one of the underlying motivations of the C-LIM's initial inception—the importance of considering a student's linguistic background and abilities prior to selecting, administering, and interpreting tests of cognitive abilities. A comprehensive evaluation that takes a student's linguistic ability into consideration should consider that a student's language ability (i.e., conversational proficiency) might not be an accurate representation of a student's academic language abilities (Cummins, 2008). Thus, it would be beneficial to gather information on a student's academic language abil-ity, due to the relationship between education and IQ (Matarazzo & Herman, 1984). A student's receptive and expressive language abilities may be a worthwhile pursuit in future research, as student's level of conversational proficiency in the classroom may mislead educators and psy-chologists to assume that the student has been exposed to English with the same frequency and depth as his or her peers (Cummins, 2008). Moreover, as suggested by the results of this study, considering the influence of linguistic ability when assessing cognitive abilities should continue to be supported by empirical evidence, instead of school psychologists continuing to rely on informal measures of linguistic ability through language samples and student interviews to gain information on language ability (Ochoa, Galarza, & Gonzalez, 1996).

A second conclusion is that it is unclear how cultural loading can be represented quantitatively in a way that is meaningful both theoretically and practically. An important, albeit unanswered question is, "What variables do practitioners take into account when making decisions about the cultural influences that may affect the selection and interpretation of tests from cognitive batteries?" Flanagan and Ortiz (2001) define cultural loading as "the degree to which a given test requires specific knowledge of or experience with mainstream culture" (p. 243). However, this broad definition does not identify specific variables that practitioners may consider in practice to make these decisions about whether a student's experiences are significantly different from mainstream culture. Given these unanswered questions, it is possible that the underlying reasoning that led to the creation of the C-LIM and its categorization system needs to be re-thought (as also suggested by Styck & Watkins, 2013), particularly with respect to cultural loading. Specifically, it would be important to consider what is occurring and possible in practice, as this is the intended use of the C-LIM."

 

 

Is biculturalism a three dimensional construct?

Interesting article suggesting that biculturalism (and not degree of acculturation and enculturation) may be a three dimensional construct. Click on images to enlarge.

 

 

 

 

 

Tuesday, March 03, 2009

Culture-Language Interpretive Matrix for IQ tests: New research

[double click on image to see enlarged and clearer image]


How good are current models for evaluating the cultural loading and linguistic complexity of individual tests in individually administered intelligence batteries?

To date, the most visible work has been based on the Flanagan, Ortiz et al. groups cross-battery work and their presentation of the Culture-Language Interpretive Matrix (C-LIM) in which individual tests in IQ batteries are categorized in terms of their perceived linguistic demand and cultural loading.

I've always believed that the C-LIM made logical and theoretical sense, but was in sore need of some empirical research evidence support.

Previously I presented an attempt by myself and Jeff Evans to quantify the linguistic demands of individually administered tests. That research effort was conducted in the spirit of stimulating others to attempt more sophisticated methods for validating the quantification of the two dimensions of the C-LIM. That post and unpublished research report is available here. Until my attendance at NASP 2009 in Boston this past week, I was unaware of any other empirical attempts to investigate the validity of this model (for example, I've routinely searched the ProQuest digital dissertation abstract service for any CHC, XBA, etc. related dissertations, and to date have not found one that has investigated this matrix-- check out those found at WMF Dissertation Abstracts Project). [Note - if such disserations exist and I've missed them, please contact me to rectify my lack of knowledge and I'll make the appropriate post]

At NASP, John Kranzler and students presented a poster (click here for single page PPT image of poster) (albeit based on a small n---thus caution is urged in generalizing from the results) that investigated the C-LIM classifications of certain Woodcock-Johnson III (WJ III) tests [conflict of interest disclosure - I'm a coauther of the WJ III). The purpose of Kranzler et al's study "was to empirically examine the relationship between English-language proficiency and linguistic demands on the cognitive test performance of a sample of culturally and linguistically diverse children in the English for Speakers of Other Languages (ESOL) program". The authors concluded that:

Results of this study for an ESOL sample do not follow the predicted pattern based on the C-LIM for the WJ-III. It is important to note that this is a small sample and replication is needed. Nonetheless, these results do not support the use of the C-LIM. These results are consistent with the general conclusion that, on IQ tests, there is very little evidence to suggest that any of the cognitive factor, scale, or subtest profile differences can be used to improve decisions about individuals.

Of course, an alternative explanation is that the C-LIM model may be correct/valid but the Flanagan, Ortiz et al. classifications of some of the WJ III tests are not accurate.

We need more research on this matrix and related interpretations. It is nice to see that some folks are attempting to do this. Kudos to Kranzler and his students for adding a small piece of empircal data to this literature.


Technorati Tags: , , , , , , , , , , , , , ,