Research Byte: Beyond Individual #Tests: Youths’ #Cognitive Abilities, Basic #Reading, and #Writing—relevant to #schoolpsychologists #CHC

An impressive multiple test-battery CHC theory cognitive and achievement (cross-battery) confirmatory factor analysis study based on a research design first conceptualized by Jack McArdle (planned missing data reference variable design) that finds that multiple broad CHC abilities are important in explaining reading and writing achievement above and beyond psychometric g.  Of course, the results would be different if a bi-factor model were run (see McGrew et al., 2023 for discussion of three major classes of cognitive-achievement CFA/SEM research designs).  

Click here to download/read this open access articles. 

This research group, IMHO, does some of the best CFA/SEM modeling in the assessment and school psychology literature.

Click on images to enlarge for easier viewing.

Beyond Individual Tests: Youths’ Cognitive Abilities, Basic Reading, and Writing 

by 

Jacqueline M. Caemmerer

 1,*, 

Audrey M. Scudder

 1, 

Timothy Z. Keith

 2 and 

Matthew R. Reynolds

 3

1

Department of Educational Psychology, University of Connecticut, Storrs, CT 06268, USA

2

Department of Educational Psychology, University of Texas at Austin, Austin, TX 78712, USA

3

Department of Educational Psychology, University of Kansas, Lawrence, KS 66045, USA

*

Author to whom correspondence should be addressed. 

J. Intell. 2024, 12(11), 120; https://doi.org/10.3390/jintelligence12110120

Abstract

Broadly, individuals’ cognitive abilities influence their academic skills, but the significance and strength of specific cognitive abilities varies across academic domains and may vary across age. Simultaneous analyses of data from many tests and cross-battery analyses can address inconsistent findings from prior studies by creating comprehensively defined constructs, which allow for greater generalizability of findings. The purpose of this study was to examine the cross-battery direct effects and developmental differences in youths’ cognitive abilities on their basic reading abilities, as well as the relations between their reading and writing achievement. Our sample included 3927 youth aged 6 to 18. Six intelligence tests (66 subtests) and three achievement tests (10 subtests) were analyzed. Youths’ general intelligence (g, large direct and indirect effects), verbal comprehension–knowledge (large direct effect), working memory (large direct effect), and learning efficiency (moderate direct effect) explained their basic reading skills. The influence of g and fluid reasoning were difficult to separate statistically. Most of the cognitive–basic reading relations were stable across age, except the influence of verbal comprehension–knowledge (Gc), which appeared to slightly increase with age. Youths’ basic reading had large influences on their written expression and spelling skills, and their spelling skills had a large influence on their written expression skills. The directionality of the effects most strongly supported the direct effects from the youths’ basic reading to their spelling skills, and not vice versa.

Keywords: 

intelligence; academic; achievement; spelling; moderation; mediation

CHC model of intelligence is mainstream: Woodcock (1990) and McGrew (2009) both #1

I was searching through some journals tonight and discovered that Dr. Richard Woodock's seminal 1990 article, "Theoretical foundations of the WJ-R measures of cognitive ability", is the #1 cited article in the Journal of Psychoeducational Assessment (see image below...click to enlarge)

This article is not just about the WJ-R. The article used the then Gf-Gc based WJ-R special study data files to conduct a series of joint CFA with all the major intelligence tests. The most important insight was a mapping of all major IQ tests as per the major broad abilities of what is now call the CHC model of intelligence. It gave birth to the concept of CHC cross-battery assessment. It was a major influential tipping point article in the evolution of the CHC model; primarily by bridging theory and assessment practice. I have written about this important seminal Woodcock paper in McGrew (2005) and Schneider and McGrew (2012). This is also relfected in the CHC Timeline project, a portion which is presented below (click image to enlarge)

Couple this with the fact that my invited editorial on CHC theory, in the journal Intelligence, is the #1 cited article since 2009 in that journal, and it is my conclusion that CHC theory is now part of mainstream psychology and theories of intelligence.

The tipping point has passed...the CHC taxonomy is now mainstream. Further evidence is the revision or development and analysis of most all intelligence theories as per CHC theory (see the stellar research of Keith and Reynolds and others...here...and here.

Posted with Blogsy

Dissertation Dish: Gf cognitive test analysis via CFA and task analysis

A comparison of confirmatory factor analysis and task analysis of fluid intelligence cognitive subtests by Parkin, Jason R., Ph.D., University of Missouri - Columbia, 2010 , 132 pages; AAT 3488814

Abstract

Cross-battery assessment relies on the classification of cognitive subtests into the Cattell- Horn-Carroll (CHC) theory's broad and narrow ability definitions. Generally, broad ability classifications have used ability data analyzed through factor analytic methods, while narrow ability classifications have used data about subtest task demands. The purpose of this investigation is to determine whether subtest similarity judgments based on task demands data, and judgments based on ability measurement provide similar results. It includes two studies. First, middle school students (N = 63) completed six target fluid reasoning subtests that were subjected to confirmatory factor analyses to analysis subtest similarities. Second, school psychology practitioners (N = 32) sorted subtest descriptions into similarity groups. Their judgments were analyzed with multiple non-hierarchical cluster analyses. Results partially confirmed that the six target subtests were classified similarly using both data types, though need to be interpreted cautiously due to limitations. Implications for assessment practices are discussed.




Posted via DraftCraft app

Dissertation Dish: CFA of WJ III AND SB5 IQ tests in a preschool population

Joint confirmatory factor analysis of the Woodcock-Johnson Tests of Cognitive Abilities, Third Edition, and the Stanford-Binet Intelligence Scales, Fifth Edition, with a preschool population by Chang, Mei, Ph.D., Ball State University, 2011 , 126 pages; AAT 3466801

Abstract

Significant evidence from the legislation, medical/clinical, or professional practice perspective all points to the advantages and necessity of conducting comprehensive assessment of cognitive abilities, especially in young children, to identify cognitive deficits, arrive at an accurate diagnosis, and establish bases for developing interventions and recommending services. Cross-battery assessment approach provides school psychologists a useful tool to strengthen their preferred cognitive battery by adopting and comparing subtests from other batteries to build up a comprehensive and theoretically sound evaluation of an individual's cognitive profile to increase the validity of test interpretation. Using joint confirmatory factor analysis, this study explored the combined underlying construct validity of the Woodcock-Johnson Tests of Cognitive Abilities, Third Edition (WJ-III COG) and the Stanford-Binet Intelligence Scales, Fifth Edition (SB5) with an independent sample of preschool children. Seven models were examined and the results showed that relatively, the underlying construct of the two tests was best represented by a three-stratum alternative CHC model in which the Gf factor and subtests had been removed. This indicates that not all the CHC constructs shared by both tests can be reliably identified among young children. Constructs of the CHC theory may be represented differently on preschool cognitive batteries due to developmental influences. Although WJ-III COG and SB5 tests as a whole did not demonstrate good results for purposes of cross-battery assessment, certain subtests (e.g., subtests representing crystallized intelligence) from each battery offer interpretative value for individual broad ability factors, providing school psychologists an in-depth understanding of a preschooler's crystallized knowledge. Exploratory factor analyses were conducted with subtests from WJ-III COG and SB5 representing the four shared broad factors ( Gc, Gf, Gv, and Gsm ). Results revealed that a 4-factor solution is a better model fit to the data. Future research includes recruiting young children with disabilities or special needs to explore best representative underlying construct of combined WJ-III COG and SB5, allowing for cross-battery assessment



- iPost using BlogPress from Kevin McGrew's iPad

IAP Applied Psychometrics 101 Report #10: "Just say no" to averaging IQ subtest scores

Should psychologists engage in the practice of calculating simple arithmetic averages of two or more scaled or standard scores from different subtests (pseudo-composites) within or across different IQ batteries? Dr. Joel Schneider and I, Dr. Kevin McGrew say "no."

Do psychologists who include simple pseudo-composite scores in their reports, or make interpretations and recommendations based on such scores, have a professional responsibility to alert recipients of psychological reports (e.g., lawyers, the courts, parents, special education staff, other mental health practitioners, etc.) of the potential amount of error in their statements when simple pseudo-composite scores are the foundation of some of their statements? We believe "yes."

Simple pseudo-composite scores, in contrast to norm-based scores (i.e., composite scores with norms provided by test publishers/authors--e.g., Wechsler Verbal Comprehension Index), contain significant sources of error. Although they have intuitive appeal, this appeal cloaks hidden sources of error in the scores---with the amount of error being a function of a combination of psychometric variables.

IAP Applied Psychometrics 101 Report #10 addresses the psychometric issues involved in pseudo-composite scores.

In the report we offer recommendations and resources that allow users to calculate psychometrically sound pseudo-composites when they are deemed important and relevant to the interpretation of a person's assessment results.

Finally, understanding the sources of error in simple pseudo-composite scores provides an opportunity for practitioners to understand the paradoxical phenomenon frequently observed in practice where norm-based or psychometrically sound pseudo-composite scores are often higher (or lower) than the subtest scores that comprise the composite. The "total does not equal the average of the parts" phenomenon is explained conceptually, statistically, and via an interesting visual explanation based on trigonometry.



Abstract

The publishers and authors of intelligence test batteries provide norm-based composite scores based on two or more individual subtests. In practice, clinicians frequently form hypotheses based on combinations of tests for which norm-based composite scores are not available. In addition, with the emergence of Cattell-Horn-Carroll (CHC) theory as the consensus psychometric theory of intelligence, clinicians are now more frequently “crossing batteries” to form composites intended to represent broad or narrow CHC abilities. Beyond simple “eye-balling” of groups of subtests, clinicians at times compute the arithmetic average of subtest scaled or standard scores (pseudo-composites). This practice suffers from serious psychometric flaws and can lead to incorrect diagnoses and decisions. The problems with pseudo-composite scores are explained and recommendations made for the proper calculation of special composite scores.


- iPost using BlogPress from my Kevin McGrew's iPad

intelligence IQ tests IQ testing IQ scores CHC intelligence theory CHC theory Cattell-Horn-Carroll human cognitive abilities psychology school psychology individual differences cognitive psychology neuropsychology psychology special education educational psychology psychometrics psychological assessment psychological measurement IQs Corner general intelligence standard scores IQ subtests Wechsler IQ subtests IQ part scores IQ composite scores cross-battery assessment applied Psychometrics psychological measurement

Generated by: Tag Generator

Dissertation Dish: Extension of CHC theory-SB5 and Bender Gestalt factor study

Structural extension of the Cattell-Horn-Carroll cross-battery approach to include measures of visual-motor integration by Brooks, Janell Hargrove, Ph.D., Georgia State University, 2009 , 117 pages; AAT 3401596

Abstract

In spite of the long-standing tradition of including measures of visual-motor integration in psychological evaluations, visual-motor abilities have not been included in the Cattell-Horn-Carroll (CHC) theory of cognitive abilities or its complementary cross-battery approach to assessment. The purpose of this research was to identify the shared constructs of a popular test of visual-motor integration and a test of intellectual functioning, and to investigate how a test of visual-motor integration would be classified within the CHC model. A large normative sample of 3,015 participants that ranged in age from 5 to 97 years completed the Bender Visual-Motor Gestalt Test, Second Edition (Bender-Gestalt II; Brannigan & Decker, 2003) and the Stanford-Binet Intelligence Scale, Fifth Edition (SB5; Roid, 2003). Correlational analyses indicated positive moderate correlations across all age ranges between the Bender-Gestalt II Copy measure and the SB5 Nonverbal Visual-Spatial Processing subscale and between the Bender-Gestalt II Recall measure and the SB5 Nonverbal Visual-Spatial Processing and Nonverbal Working Memory subscales. Exploratory factor analyses revealed a three-factor model for four age groupings and four-factor model for one age grouping, suggesting factors which represent crystallized ability, fluid reasoning, and visual-motor ability. The results of this study suggest that the Bender-Gestalt II measures abilities that are not included in the SB5. Therefore, the Bender-Gestalt II would complement an intelligence test such as the SB5 in order to form a CHC Visual Processing ( Gv ) broad ability factor. These findings also address the need for further research to validate the constructs measured by newer versions of widely-used tests of cognitive ability.

Technorati Tags: Psychology, school psychology, educational psychology, forensic psychology, neuropsychology, special education, intelligence, cognitive abilities intelligence theories, CHC theory, Cattell-Horn-Carroll, factor analysis, cross battery assessment, IQs Corner, IQ, IQ tests, IQ scores, Gv, Gc, Gf, Gp, Bender Gestalt, motor abilities, SB5, Stanford Binet

Flanagan's PPT slides re: Theory and Research-Based Approaches to SLD Identification

The PPT slides for Dr. Dawn Flanagan's Wiley Webinar on theory and research-based approaches to SLD identification is now available for on-line viewing (and downloading) at IQ's Corner SlideShare home.  Thank you to Dr. Flanagan for making this available for others.

Technorati Tags: psychology, school psychology, educational psychology, special education, LD, SLD, learning disabilities, Wiley Webinar, Flanagan Webinar, CHC, Cattell-Horn-Carroll, WJ III, SBV, KABC-II, DAS-II, WISC-IV, WAIS-IV, neuropsychology, school neuropsychology

WAIS-IV and WIA-III CHC test classifications at XBA On-line

Flanagan et al. have now made available their CHC cross-battery assessment classifications for download at their XBA On-Line web site. Check 'em out here.   Click here for link to IAP AP101 report on WAIS-IV CHC analysis.

Technorati Tags: psychology, school psychology, educational psychology, LD, special education, neuropsychology, CHC, CHC theory, Cattell-Horn-Carroll, XBA, cross-battery, IQ, IQ tests, intelligence, cognitive abilities

CHC theory: Prior cognitive-achievement relations research summaries

The first attempt to summarize the cognitive-achievement relations research vis-à-vis a CHC lens (CHC COG-ACH) was presented in McGrew and Flanagan's (1998) Intelligence Test Desk Reference: Gf-Gc Cross-Battery Assessment (ITDR). The closest other effort of note is the blended CHC-neuropsychological research integration efforts of Fiorello and colleagues (Fiorello, Hale & Snyder, 2006; Fiorello & Primerano, 2005; Hale & Fiorello, 2004).


According to Flanagan et al. (2006), studies were identified for potential inclusion in their CHC COG-ACH summaries via three search methods. First, research studies that investigated the relations between cognitive abilities and reading, math, and writing achievement were identified via a search of the PsycINFO electronic database. Second, an ancestral search strategy of the references identified in step one revealed potentially useful other articles for review. Third, the original McGrew and Flanagan (1998) CHC COG- ACH summary table results were incorporated with the new-found research literature.

Studies deemed to have reported a significant COG-ACH relation by the Flanagan research group were then listed in tabular form in one of three categories: (a) key CHC studies—CHC- organized studies that included markers for most broad CHC cognitive abilities; (b) reviews—non- CHC organized narrative or meta-analytic research syntheses reporting significant relations between cognitive abilities and school achievement; and (c) individual studies—single non-CHC empirical studies that investigated the relations between cognitive abilities and school achievement. For most of the reviews and individual studies the Flanagan group translated the described non-CHC cognitive abilities as per the nomenclature of the CHC taxonomy (e.g., phonemic awareness = narrow ability of phonetic coding under the broad domain of Ga).

Flanagan et al.’s (2006) summary is impressive. A total of 138 references were presented for reading (8 key CHC studies; 23 reviews; 107 individual studies) and 37 references were presented for math (3 key CHC studies; 5 reviews; 29 individual studies).

In contrast, the Fiorello research group, in the process of promoting the use of CHC assessment in school neuropsychology practice (Fiorello & Primerano, 2005; Fiorello et al., 2006), reviewed many of the same key CHC-reading studies included in the Flanagan research group’s summary tables. The Fiorello group’s most important contribution was their “cross- walk” of neuropsychology research on reading disorders to the CHC framework. For example, Fiorello et al. (2006) related one component of the “double deficit” reading disability (rapid automatized naming or RAN; Wolf & Bowers, 1999) to CHC when they stated that “ideational fluency, together with Gs, is related to Rapid Automatized Naming” (p. 839). Since the Fiorello research group’s information was not based on an explicit research literature search and synthesis strategy, their interpretations are presented as supplemental to the more comprehensive Flanagan research group’s CHC COG-ACH synthesis efforts.

Stay tuned. The results of these prior research syntheses, as well as limitations of these syntheses will be posted next,followed by the description of the methods used to provide a more systematic and organized literature review (McGrew & Wendling, manuscript submitted for publication)

Click here for other posts in this series.

CHC theory: Emergence, test instruments and school-related research brief

Contemporary Cattell-Horn-Carroll (CHC) intelligence test development, interpretation and applied research can be traced to a fortuitous meeting of Richard Woodcock, John Horn, and John “Jack” Carroll in the fall of 1985, a meeting also attended by the first author of this web-resource ( McGrew, 2005). This meeting resulted in the 1989 publication of the first individually-administered, nationally standardized CHC-based intelligence battery, the Woodcock- Johnson- Revised (Woodcock, McGrew, & Mather, 1989). This landmark event, which occurred 20 years ago, provided the impetus for the major CHC- driven evolution of school- based intelligence testing practice.

Subsequent important CHC events followed during this 20 year period, and included: (a) the first set of CHC- organized joint test battery factor analysis studies (Woodcock, 1990) which planted the seeds for the concept of CHC cross-battery (CB) assessment, (b) the first attempt to use the WJ-R, via a Kaufman-like supplemental testing strategy (Kaufman, 1979), to implement the yet to be named and operationalized CHC CB approach to testing ( McGrew, 1993), (c) the articulation of the first integrated Cattell-Horn-Carroll model and classification of the major intelligence batteries as per the CHC framework (McGrew, 1997), (d) the first description of the assumptions, foundations, and operational principles for CHC CB assessment and interpretation (Flanagan & McGrew, 1997; McGrew & Flanagan, 1998), (e) the publication of the first intelligence theory and assessment book to prominently feature CHC theory and assessment methods (Contemporary Intellectual Assessment: Theories, Tests, and Issues; Flanagan, Genshaft & Harrison, 1997; click here for link to 2nd edition), (f) the publication of the CHC CB assessment series ( Flanagan, McGrew & Ortiz, 2000; Flanagan, Ortiz, Alfonso & Mascolo, 2006; Flanagan, Ortiz & Mascolo, 2001, 2007; McGrew & Flanagan, 1998), (g) the completion of a series of CHC-organized studies that investigated the relations between CHC cognitive abilities and reading, math, and writing achievement (what you are reading now), (h) the articulation of CHC-grounded SLD assessment and eligibility frameworks (see Flanagan & Fiorello, manuscript in preparation) and (h) the subsequent CHC- grounded revisions or interpretations of a number of comprehensive individually administered intelligence test batteries ( Differential Abilities Scales—II, DAS-II;Stanford- Binet—5^thEdition, SB5;Kaufman Assessment Battery for Children—2^ndEdition, KABC- II). Although not overtly stated, the impact of CHC theory can be seen in the recent revisions of the venerable Wechsler trilogy ( WPPSI-III; WISC- IV; WAIS- IV) as well as the presentation of CHC CB procedures for interpreting the three Wechsler batteries ( Flanagan et al., 2000).

Click here for other posts in this series.

Culture-Language Interpretive Matrix for IQ tests: New research

[double click on image to see enlarged and clearer image]

How good are current models for evaluating the cultural loading and linguistic complexity of individual tests in individually administered intelligence batteries?

To date, the most visible work has been based on the Flanagan, Ortiz et al. groups cross-battery work and their presentation of the Culture-Language Interpretive Matrix (C-LIM) in which individual tests in IQ batteries are categorized in terms of their perceived linguistic demand and cultural loading.

I've always believed that the C-LIM made logical and theoretical sense, but was in sore need of some empirical research evidence support.

Previously I presented an attempt by myself and Jeff Evans to quantify the linguistic demands of individually administered tests. That research effort was conducted in the spirit of stimulating others to attempt more sophisticated methods for validating the quantification of the two dimensions of the C-LIM. That post and unpublished research report is available here. Until my attendance at NASP 2009 in Boston this past week, I was unaware of any other empirical attempts to investigate the validity of this model (for example, I've routinely searched the ProQuest digital dissertation abstract service for any CHC, XBA, etc. related dissertations, and to date have not found one that has investigated this matrix-- check out those found at WMF Dissertation Abstracts Project). [Note - if such disserations exist and I've missed them, please contact me to rectify my lack of knowledge and I'll make the appropriate post]

At NASP, John Kranzler and students presented a poster (click here for single page PPT image of poster) (albeit based on a small n---thus caution is urged in generalizing from the results) that investigated the C-LIM classifications of certain Woodcock-Johnson III (WJ III) tests [conflict of interest disclosure - I'm a coauther of the WJ III). The purpose of Kranzler et al's study "was to empirically examine the relationship between English-language proficiency and linguistic demands on the cognitive test performance of a sample of culturally and linguistically diverse children in the English for Speakers of Other Languages (ESOL) program". The authors concluded that:

Results of this study for an ESOL sample do not follow the predicted pattern based on the C-LIM for the WJ-III. It is important to note that this is a small sample and replication is needed. Nonetheless, these results do not support the use of the C-LIM. These results are consistent with the general conclusion that, on IQ tests, there is very little evidence to suggest that any of the cognitive factor, scale, or subtest profile differences can be used to improve decisions about individuals.

Of course, an alternative explanation is that the C-LIM model may be correct/valid but the Flanagan, Ortiz et al. classifications of some of the WJ III tests are not accurate.

We need more research on this matrix and related interpretations. It is nice to see that some folks are attempting to do this. Kudos to Kranzler and his students for adding a small piece of empircal data to this literature.


Technorati Tags: psychology, educational psychology, school psychology, neuropsychology, special education, learning disabilities, LD, cross-battery, XBA, CHC, Cattell-Horn-Carroll, WJ III, WJ III NU, cultural oading, linguistic complexity

WISC-III/WJ III cross-battery Guttman 2-D Radex analysis

One more WISC-III/WJ III cross-battery analysis--this time a 2-D Guttman Radex MDS model (click here).  As readers have noted, I've been on a bit of a data analysis binge this past week (in preparation for writing a manuscript---and after being refreshed by an actual 2+ week vacation) and have reported:  (a) WISC-III/WJ III cross-battery g+specific cog-ach relations SEM, (b) WJ III 2-D Guttman Radex MDS of WJ III norm sample ages 6-8, and (c) WJ III 3-D Guttman Radex MDS of ages 9-13 of norm sample.  It is hoped these analyses provide useful information in understanding the characteristics of the tests in the WJ III and Wechsler intelligence batteries.

Unfortunately this analysis is based on the WISC-III and not the more recent WISC-IV.  Nevertheless, the results still provide useful information on the WISC-III tests that are still present in the WISC-IV.

Given all I've written regarding the various MDS models, I'm going to only make a few comments and hope others take the presentation of these data to engage in additional discussion, interpretion, etc.----have some fun.

A few observations/comments:

• Gv tests (both WISC-III and WJ III) continue to surface on the more outer rings of the MDS models---suggesting that they are more lower-level perceptual/processing measures and do not capture complex Gv cognitive processing.  See my Gv comments on this the other day.  The same can be said for Ga tests.

• WJ III Understanding Directions is consistently one of the more cognitive complex tests.  And, it is largely a language-based measure of working memory (Gsm-MW).  Remember that as per the Radex model, cognitive complexity deals with the amount of elements/components that are processed.....and is not the same as abstract thinking (Gf-ish stuff).  WJ III Numbers Reversed also shows up close to the center, with WISC-III Digit Span not far behind.  Does this support the popular working memory=Gf/g research hypothesis?

• The Gc tests from both batteries appear similar in placement.

• As would be expected, the WJ III Gf tests (Concept Formation, Numerical Reasoning [which is a combo of Number Series and Number Matrices], and Analysis-Synthesis are within the center "cognitive complexity" circle.

I'm sure there is much more that can be gleaned, but I'll leave that to the readers to discover, debate, and discuss.  I actually think a 3-D MDS model is necessary to capture the characteristics of the measures...but I've run out of time and steam on these analysis.  Maybe at a later date.

A couple caveats I provided the other day are also relevant here--(a) I'm a coauthor of the WJ III (conflict of interest disclosure) and (b) these results have not been peer-reviewed

WISC-III/WJ III cross-battery g+specific cog-ach abilities findings

WISC-III/WJ III cross-battery g+specific abilities research reinforces "just say maybe" program of CHC g+specific abilities research.

I'm just starting the process of drafting a manuscript to summarize the results of the IAP CHC COG-ACH Correlates Meta-Analysis project (click here).  In that on-line EWOK (Evolving Web of Knowledge) I list a McGrew (2007a) study in the reading and math summary tables.  That reference reflects unpublished re-analysis I completed (last fall) with the Phelps et al. (2007) joint (cross-battery) WISC-III/WJ III dataset.  In order to include the findings in the synthesis manuscript, I felt it appropriate to at least informally publish the final models for reference. 

Two important caveats.  I'm a coauthor of the WJ III (conflict of interest disclosure).  The second caveat is outlined in the brief report of the results that I have posted.  When I have time I will update the synthesis EWOK. I will change the reference to McGrew (2008) reflecting posting.

Technorati Tags: psychology, educational psychology, school psychology, neuropsychology, intelligence, IQ testing, IQ, Wechsler, WISC-III, WJ III, general intelligence, g, g+specific abilities

Dissertation dish: K-ABC II, SB5, WJ III CHC factor analysis studies

Two new CHC-based dissertations I stumbled across this weekend.

A joint-confirmatory factor analysis using the Woodcock-Johnson III Tests of Cognitive Ability and the Stanford-Binet Intelligence Scales: Fifth Edition with high-achieving children by Williams, Tasha H., Ph.D., Ball State University, 2005, 206 pages; AAT 3176652

• Abstract: A considerable about of research has concentrated on studying the performance of high achieving children on measures of intellectual functioning. Findings have indicated high achieving children display differences in performance patterns as well as in the cognitive constructs measured when compared to their average peers. The conceptualization of intelligence has evolved over time and contemporary theories of intelligence have described cognitive ability as consisting of multiple constructs which are often interrelated. Currently, one of the most comprehensive and empirically supported theories of intelligence is the Cattell-Horn-Carroll (CHC) theory (Cattell, 1941; Horn, 1968; Carroll, 1993). The multidimensional and hierarchical CHC theory has served as the foundation for the development and recent revisions of cognitive ability measures such as the Woodcock-Johnson Tests of Cognitive Ability - Third Edition (WJ-III COG; McGrew & Woodcock, 2001) and the Stanford-Binet Intelligence Scales - Fifth Edition (SBS; Roid, 2003b). The purpose of this study was to explore the construct validity of the WJ-III COG and SBS with a sample of high achieving children. Individual confirmatory factor analyses were conducted using the WJ-III COG and SBS. Additionally, a joint confirmatory factor analysis was conducted using both the WJ-III COG and SBS. The results indicated an alternative six-factor WJ-III COG and four-factor SBS models provided the best fit to the data of a high achieving sample, supporting previous research suggesting high achieving children display differences in cognitive constructs when compared with their average counterparts. The joint-confirmatory factor analysis indicated the best measures for the CHC factors measured by both the WJ-III COG and SB5 to help guide cross-battery assessments with high functioning children. Clinical applications of the findings are discussed.

The validity of intelligence tests using the Cattell-Horn-Carroll Model of intelligence with a preschool population by Morgan, Kimberly Elaine, Ph.D., Ball State University, 2008, 219 pages; AAT 3303357

• Abstract: Individual differences in human intellectual abilities and the measurement of those differences have been of great interest to the field of school psychology. As such, different theoretical perspectives and corresponding test batteries have evolved over the years as a way to explain and measure these abilities. A growing interest in the field of school psychology has been to use more than one intelligence test in a "cross-battery" assessment in hopes of measuring a wider range (or a more in-depth but selective range) of cognitive abilities. Additionally, interest in assessing intelligence began to focus on preschool-aged children because of initiatives to intervene early with at-risk children. The purpose of this study was to examine the Stanford-Binet Intelligence Scales, Fifth Edition (SB-V) and Kaufman Assessment Battery for Children, Second Edition (KABC-II) in relation to the Cattell-Hom-Carroll (CHC) theory of intelligence using a population of 200 preschool children. Confirmatory factor analyses (CFAs) were conducted with these two tests individually as well as in conjunction with one another. Different variations of the CHC model were examined to determine which provided the best representation of the underlying CHC constructs measured by these tests. Results of the CFAs with the SB-V revealed that it was best interpreted from a two-stratum model, although results with the KABC-II indicated that the three-stratum CHC model was the best overall design. Finally, results from the joint CFA did not provide support for a cross-battery assessment with these two particular tests.

Technorati Tags: psychology, educational psychology, school psychology, neuropsychology, KABC II, SB5, WJ III, IQ tests, intelligence, factor analysis, CHC, CHC theory, cross-battery, Woodcock-Johnson, Stanford-Binet

Wechsler Arithmetic test: Measure of Gq or Gf?

What does the Wechsler Arithmetic test measure? Why has it's interpretation been so variable over the decades? Why is it now classified (as per CHC theory) as a mixed measure of Gsm (Short-term Memory - Working memory; MW) and Gf (Fluid Reasoning - Quantitative reasoning; RQ) in the latest Essentials of Cross-Battery Assessment book (Flanagan, Ortiz and Alfonso, 2007)? [Click here if you need more information on CHC theory and the major abilities, definitions, and abbreviations]

While preparing for my recent presentation at the Third National School Psychology Neuropsychology Conference, I consulted the 2nd edition of the Essentials of Cross-Battery book. I noticed on page 310 that, in contrast to prior cross-battery classifications of the Arithmetic test as a primary measure of Gq (Quantitative Knowledge-Math Achievement; A3) and Gf (Fluid Reasoning-Quantitative Reasoning; RQ) [Note - I was involved in these prior classifications as a coauthor of the first cross-battery book (ITDR: McGrew and Flanagan, 1998) and the Wechsler-specific spin-off cross-battery book (Flanagan, McGrew & Ortiz, 2000), it had now changed to Gsm and Gf.

First, a historical note. In the ITDR and Wechsler cross-battery books the primary Gq classification was based on a series of CHC/Gf-Gc designed cross-battery (joint) factor studies. The secondary Gf classification was a logical content analysis based classification, for which no available CHC/Gf-Gc cross-battery factor analysis supported the classification.

A review of page 310 in the 2nd Edition of the Essentials of CB indicates that the Gsm and Gf classifications are based on "factor analysis from the WISC-IV technical and interpretive manual (Psychological Corporation, 2003) and from the results of factor analysis reprted in Keith, Fine, Taub, Reynolds, and Kranzler (2006)." My problem with this change is that these supporting analyses are all within-battery (WISC-IV only) confirmatory factor analysis studies (CFA), and thus do not include the complete range of CHC indicators in the analysis, especially other Gq markers. If you want to see a prior post I made about my disagreement with the Keith et al. Arithmetic classification, click here. Personal communication with one of the XBA Essentials books authors indicates that the Gf classification is also intended to reflect content validity evidence.

Why do I take issue with the use of within-battery CFA to make CHC test classifications? If you want the long story you can read about the strengths and limitations of within- and cross-battery CFA studies in the first two XBA books listed above. If you want a short-course on the issue, you can check out an on-line PPT show I just uploaded that conceptually illustrates the limitations of within-battery CFA studies....and...more importantly, the advantage of CHC-designed cross-battery (joint) CFA studies classifying tests as per CHC theory.

To date I know of 9 different Wechsler/Woodcock-Johnson CHC/Gf-Gc designed cross-battery studies. All 9 studies include other markers of Gq (math achievement tests). In ALL of these analysis the Wechsler Arithmetic test has a large a significant loading on Gq (average loading in the mid .70's)....none on Gf. In one study (Phelps et al. 2005) there is a small secondary loading on Gs. In none of these cross-battery studies does Arithmetic display a significant Gsm factor loading. You can view a summary of these CHC-designed cross-battery findings, as well as the Keith et al. (2006) model upon which the current (and I believe inaccurate) Arithmetic Gf classification is based (due to a loading of .79 on a Gf factor), at the following link.

My two cents. I believe the best available analysis argue for the Wechsler Arithmetic test being interpreted primarily as a measure of Gq (Math Achievment-A3). I believe practitioners should not interpret this test as a primary or strong measure of Gf (Fluid Reasoning - Quantitative Reasoning; RQ). This does not mean that RQ is probably not involved. What the data indicate that if RQ is involved, the amount of variance is trivial and dwarfed by Gq-A3.

Conflict of interest disclosure - I'm a coauthor of the WJ III and thus have a financial interest in a competitor to the Wechsler batteries. I no longer receive any royalties from the two respective cross-battery books I coauthored.

Perceptual speed (Gs-P): Time to revisit CHC test classifications?

I recently read an article by Ackerman and Beier (2007) that reminded me of something I wrote in my Cattell-Horn-Caroll CHC (Gf-Gc) Theory: Past, Present and Future CIA book chapter (click here for on-line earlier version of the chapter; click here for specific section of the chapter I'm referencing).

The literature I reviewed in 2004/2005 suggested that those of us involved in the development and interpretation of intelligence tests should recognize an updated taxonomy of human speed abilities. A tentative taxonomy was presented, based on the work of others (I was just the synthesizer), that included Broad Cognitive Speed (Gs), Broad Decision Speed (Gt), and Broad Psychomotor Speed (Gps). Of particular interest to me was the suggestion that Perceptual Speed (P) may better be conceptualized as an intermediate speed ability between the broad (stratum II) and narrow (stratum I) speed abilities.

The research of Ackerman et al. suggested that the domain of Perceptual Speed might be best broken down into the sub-abilities of Perceptual Speed:Pattern Recognition (Rpr), Perceptual Speed: Scanning (Ps), Perceptual Speed: Memory (Pm), and Perceptual Speed: Complex (Pc). Definitions of these four narrow P abilities can be found at the 2004 link above, as well by visiting one of the original Ackerman et al. (2000) articles where this structure was first articulated.

In 2005, in an updated/revised CHC classification of the WJ III tests, I classified the WJ III Visual Matching and Cross Out tests as possible Ps measures. I further suggested that Pair Cancellation may be primarily a Pc test.

I would like to lay down a challenge to my fellow CHC intelligence users - how about people attempt to reclassify all P tests across the major intelligence batteries (as listed in the extensive CHC Cross-Battery publications of Flanagan et al.) as per the Ackerman four sub-P abilities? Maybe the results might help us better understand why performance on P tests often vary within test profiles and research studies. If nothing else, I'd like some feedback, challenges, revisions, suggestions, etc. to my reclassification of the three WJ III tests listed above.

Just a "Friday-afternoon-stuck-in-my-home-office-due-to-a-late-winter-storm-in-MN" idea.


Technorati Tags: psychology, school psychology, educational psychology, neuropsychology, Perceptual Speed, Gs, cognitive speed, CHC, Cattell-Horn-Carroll, intelligence, IQ, IQ tests, cross-battery, WJ III, Woodcock-Johnson

Dissertation Dish: WJ III K-ABC preschool CHC cross-battery factor study

Yet another WJ III CHC-organized dissertation has found its way to IQ's Corner (see Disseratation Dish index for others). This dissertation is a cross-battery confirmatory factor analysis of the K-ABC and WJ III in a preschool sample. The abstract is below.

A joint confirmatory factor analysis of the Kaufman Assessment Battery for Children, Second Edition, and the Woodcock-Johnson Tests of Cognitive Abilities, Third Edition, with preschool children by Hunt, Madeline S., Ph.D., Ball State University, 2007, 238 pages; AAT 3288307

Abstract

• The purpose of this study was to explore the construct validity of the Kaufman Assessment Battery for Children, Second Edition (KABC-II; Kaufman & Kaufman, 2004a) and the Woodcock-Johnson Tests of Cognitive Abilities, Third Edition (WJ-III COG; Woodcock, McGrew, & Mather, 2001) with a sample of 200 preschool children, ranging in age from 4 years, 0 months to 5 years, 11 months, and attending preschool and daycare programs in and around a Midwestern city. This study attempted to determine if the Cattell-Horn-Carroll (CHC) factor structure represented on these tests can be identified with young children. Individual confirmatory factor analyses were conducted separately with the KABC-II and WJ-III COG. Moreover, a joint confirmatory factor analysis was conducted using both the KABC-II and WJ-III COG. The results of the individual KABC-II factor analyses indicated a two-tiered Gf-Gc model provided the best fit to the data, although the three-tiered CHC model also fit the data well. This suggests the underlying factor structure of the KABC-II is well represented by the CHC theory. The WJ-III COG was best represented by an alternative CHC model, in which the Gf factor and subtests had been removed, indicating not all CHC constructs represented on the WJ-III COG can be reliably identified among young children. The joint confirmatory factor analysis indicated the strongest measures of the shared CHC factors on the KABC-II and WJ-III COG, which can help to guide cross-battery assessment with preschool children. Overall, the results confirmed multiple CHC abilities can be assessed with young children, implying clinicians should be using preschool tests that provide scores for several cognitive abilities. This study also revealed the constructs of the CHC theory may be represented somewhat differently on preschool tests due to developmental influences. Strong correlations were evident between unrelated tasks, primarily because the verbal and linguistic demands of many subtests caused them to load unexpectedly on the Gc factor. Suggestions for future research include conducting the same study using preschool children with suspected disabilities, as well as with older children, examining other instruments that include a Gf factor, and conducting exploratory factor analysis with subtests from the KABC-II and WJ-III COG that contain significant components of more than one ability.

Technorati Tags: psychology, educational psychology, school psychology, neuropsychology, K-ABC, WJ III, Woodcock-Johnson, CHC, CHC theory, cross-battery, preschool, intelligence, IQ tests, IQ scores, Ball State University

CHC and WJ III dissertations 11-16-07

The other day I posted a list of 19 CHC (Cattell-Horn-Carroll) based dissertations (titles and abstracts) that have been published in the past five years. I've now expanded this list to include WJ III dissertations published during the past five years. The total n is now approximately 30. Title pages and abstracts can be found by clicking here.

I've set an email alert with my university to notify me of new dissertations in these two areas (CHC and WJ III) and will pass along FYI's on new and notable dissertations as they come in.

Technorati Tags: psychology, educational psychology, neuropsychology, school psychology, dissertation, CHC theory, Cattell-Horn-Carroll, intelligence, cross-battery, IQ, IQ test, WJ III

Powered by ScribeFire.