Some CHC specific abilities are important in school learning: Reflections on the g+specific abilities research

[Double click on image to enlarge]

Back in 1997, I, together with Dawn Flanagan, Tim Keith and Mike Vanderwood, published our first g+specific--->achievement article in School Psychology Review. I recently searched high and low for a pdf copy of this article via the usual library sources but came up blank. I finally scanned a copy into a pdf file. A copy of this article can be viewed by clicking here.

Included in that article was the figure to the right (double click to enlarge the figure). I've always liked this figure as it laid out the reasoning why the "just say no" (to intelligence test subtest analysis) research needed to be revisited in light of advances in: (a) theories of intelligence (CHC theory), (b) measurement of intelligence constructs (Gf-Gc or CHC-grounded batteries), and (c) statistical methodology (SEM vs multiple regression). The figure summarizes much of the introductory text in the 1997 article. Frankly, this publication is one of those for which I'm the most proud. Why?

First, the results demonstrated that some specific abilities are important in understanding reading and math achievement above and beyond the effect of g (general intelligence). IMHO this was a very important finding...and led us to argue for the "just say maybe" position (re: interpretation of strength and weakness profiles of cognitive tests---where the S/W measures where composites of at least 2 tests---cluster scores). Even Dr. Dan Reschly, someone typically associated with the anti-IQ movement in school psychology, in a response article in the same special SPR issue, stated (with regard to our 1997 article) :

• “The arguments were fairly convincing regarding the need to reconsider the specific versus general abilities conclusions. Clearly, some specific abilities appear to have potential for improving individual diagnoses. " (Reschly, 1997, p.238).

Second, this "mother" study lead to a systematic program of g+specific abilities research that continues to suggest that certain broad/narrow CHC abilities are important in understanding reading and math achievement above and beyond the effect of g (general intelligence). A listing of articles that followed can be found by clicking here. Click here if you want to view all g+specific abilities posts that have been made at IQ's Corner. There are now a sufficient number of these studies, that when combined with CHC no-g model multiple regression studies, that I was able to recently complete the formative work on a research synthesis (CHC Cognitive-Achievement Correlates Meta-Analysis Project) of the collective findings (which I will eventually be summarizing in a manuscript). A review of all studies continue to support the "just say maybe" position.

Interestingly, although not dealing with reading and math achievement, a recent article in the prestigious journal Intelligence provided additional support for the g+specific "just say maybe" position. The CHC-organized article by Reeve (2004) demonstrated that specific cognitive abilities are indeed important (above and beyond the effect of g) in understanding and explaining the development of domain-specific knowldge (Gkn). The Reeve article provides an excellent review of the literature and, in many respects, reflects (and extends and augments) the arguments we made in our original 1997 article. I urge those interested in the g+specific abilities debate to carefully read Reeve's literature review...as well as his findings. The Reeve findings are important given the extremely large size of the sample (n = 300,000+ from the famous PROJECT TALENT project)

Finally, I would be remiss if proper credit was not given to the "grandmother" article that first stimulated our (McGrew et al., 1997) initial g+specific abilities project, and which also appeared to play a noticeable role in the Reeve (2004) study. Our original research was started after reading the following article:

• Gustafsson, J. -E., & Balke, G. (1993). General and specific abilities as predictors of school achievement. Multivariate Behavioral Research, 28, 407–434. (click here to view)
  

Enough said for now. I believe that a sufficient body of research evidence now exists that demonstrates there is more to understanding acquired knowledge acquisition and school achievement than g (general intelligence). When utilizing an appropriately valid and comprehensive model of intelligence (CHC theory) and appropriately designed and analyzed models, certain specific broad or narrow specific abilities are found to be important...and their effect sizes are not trivial. There is more to understanding school functioning than a simple full scale g-type score.

Of course, if one does not believe in the construct of g (e.g., John Horn's adamant position), then narrow and broad CHC abilities are found to be even more important (as reflected in the non-g CHC organized multiple regression studies integrated in the above mentioned research synthesis). However, that is another post (or series of posts).

Another g+specific abilities and rdg. ach. study

Yet another g+specific CHC-to-rdg achievement study has been published. This Journal of Psychoeducational Assessment "in press" article, by Nick Benson, also uses the WJ III norm data [conflict of interest disclosure - I'm a coauthor or the WJ III]. This research differs significantly from most all prior g+specific abilities causal SEM modeling investigations by including not only effects between CHC cognitive abilities and reading, but causal effects among specific subskills of reading (reading fluency, basic reading skills, reading comprehension). The abstract is below.

Blogmaster comment - the evidence continue to mount that indicates, when using a comprehensive cognitive ability taxonomy (CHC theory), construct valid measures from the major broad CHC domains, and methodology that allows the simultaneous effects of both general intelligence (g) and specific cognitive abilities in the analysis, that some specific cognitive abilities are important in understanding school achievement above an beyond the influence (effect) of g. Other studies that support this position are listed below the current featured article abstract.

Abstract

• Structural equation modeling procedures are applied to the standardization sample of the Woodcock–Johnson III to simultaneously estimate the effects of a psychometric general factor (g), specific cognitive abilities, and reading skills on reading achievement. The results of this study indicate that g has a strong direct relationship with basic reading skills until about sixth grade. Also, g is found to have a strong indirect effect on reading fluency and comprehension across grade levels. Basic reading skills has a strong direct effect on reading fluency across grade levels. The effect of cognitive processing speed (Gs) on reading fluency increase with age. Reading fluency initially has a strong direct effect on reading comprehension, but this effect is reduced with age. Conversely, the direct effect of crystallized intelligence or knowledge (Gc) on reading comprehension increase with age.

Other supporting g+specific abilities research studies.

• Bensen, N. (2007, in press). Cattell-Horn-Carroll cognitive abilities and reading achievement. Journal of Psychoeducational Assessment (this is the article featured above)
  

• Flanagan, D. P. (2000). Wechsler-based CHC cross-battery assessment and reading achievement: Strengthening the validity of interpretations drawn from Wechsler test scores. School Psychology Quarterly, 15(3), 295-329.

• Floyd, R. G., Keith, T. Z., Taub, G. E., & McGrew, K. S. (2007). Cattell–Horn–Carroll cognitive abilities and their effects on reading decoding skills: g has indirect effects, more specific abilities have direct effects. School Psychology Quarterly, 22, 200-233.

• Keith, T. Z. (1999). Effects of general and specific abilities on student achievement: Similarities and differences across ethnic groups. School Psychology Quarterly, 14(3), 239-262.

• McGrew, K. S., Flanagan, D. P., Keith, T. Z., & Vanderwood, M. (1997). Beyond g: The impact of Gf-Gc specific cognitive abilities research on the future use and interpretation of intelligence tests in the schools . School Psychology Review, 26(2), 189-210.

• Taub, G., Floyd, R. G., Keith, T. Z., & McGrew, K. S. (in press). Effects of general and broad cognitive abilities on mathematics achievement from kindergarten through high school. School Psychology Quarterly.

• Vanderwood, M. L., McGrew, K. S., Flanagan, D. P., & Keith, T. Z. (2002). The contribution of general and specific cognitive abilities to reading achievement. Learning and Individual Differences, 13, 159-188.

Technorati Tags: psychology, school psychology, educational psychology, neuropsychology, education, achievement, intelligence, IQ tests, IQ scores, g, general intelligence, specifc abiliites, domain specific, SEM, WJ-R, WJ III, CHC theory, Cattell-Horn-Carroll, Gf-Gc

Powered by ScribeFire.

Research alert: Specific cognitive abilities: A discussion of advantages and disadvantages of measurement methods - ScienceDirect

Important article for CHC g, broad, narrow cognitive ability and achievement research.

 

Specific cognitive abilities: A discussion of advantages and disadvantages of measurement methods - ScienceDirect 
https://www.sciencedirect.com/science/article/abs/pii/S0160289626000425

 

Abstract

Several models of cognitive abilities emphasize both general and specific abilities. While there have been several studies of ways to create measures of general cognitive ability, little information has been consolidated about methods for measuring specific abilities. In this article several models found in the literature are presented and discussed. Of particular importance, the reliance on face validity to develop measures of specific ability is discouraged for being unscientific, prone to personal bias and not necessarily repeatable. Factor analytic methods, hierarchical factor analysis, bifactor analysis, and principal factors and principal components methods are discussed and the need for statements about decisions made while conducting the analysis is strongly encouraged. Lastly, an examination of the methods, and advantages and disadvantages are presented.

 

Introduction

Over the past five decades, there has been a resurgence of interest in the study of cognitive abilities, particularly in relation to Spearman's two-factor theory of intelligence and factor analytic models of cognitive structure. In his seminal work, Spearman (1904) using the tetrad differences method, factor-analyzed data from schoolchildren and identified a common source of variance underlying performance across diverse cognitive tasks, which he termed the general factor, or g. The empirical observation that cognitive tests tend to correlate positively—a phenomenon known as positive manifold—has been consistently cited as evidence for the existence of g, a finding further supported by hierarchical and bifactor factor analytic approaches (e.g., Jensen, 1998). Several studies have addressed methodological approaches to measuring g (e.g., Jensen & Weng, 1994; Ree & Earles, 1991; Reeve & Blacksmith, 2009). Each cognitive test also captures unique variance not attributable to g, creating specific factors, or s. Despite their theoretical importance, there is a notable paucity of research focused on the measurement of s, with the work of Coyle and Greiff (2021) and Kell and Lang, 2017, Kell and Lang, 2018 representing examples of the few contributions in this area. While specific variance is typically unique to individual measures, in cases where multiple tests share a common specific source of variance, the resulting construct is classified as a group factor.

Whereas g represents broad, generalized cognitive functioning, specific abilities denote more narrowly defined cognitive abilities. These specific abilities are typically assessed using cognitive tests or tasks that measure both g and specific abilities. Specific abilities are rarely measured without also measuring general cognitive ability.

 

Specific abilities can be thought of as a continuum going from narrow versus broader factors. A good example is a broad spatial factor (spatial working memory, mental rotation, spatial reasoning) versus a narrowly defined spatial factor focusing on just visualization.

 

Thomson (1916) proposed an ability sampling model (also called the bonds model) of cognitive ability as an alternative to Spearman's two-factor theory. His model proposed that intelligence is related to the number and complexity of neural patterns in the brain rather than the existence of a general factor. Analyzing the same data as Spearman, Thomson found no need for a general factor, instead proposing that test scores are the sum of specific factors only. Thomson proposed that intelligence is not the consequence of a single pervasive factor. Rather, it emerges from the sampling of multiple independent mental elements, which he called “bonds.” The theory posits that the human brain learns to associate various bonds. The bonds are described very abstractly, so it is unclear what they correspond to psychologically or neurologically (e.g., cognitive processes, connections, skills, etc.). Because the model is cast mainly as a statistical sampling scheme, critics have argued that it is not a worked-out theory of cognitive architecture, but more a how-you-could-get-the-correlations story (Thomson, 1916). Bartholomew et al. (2009) contend that Thomson's model deserves more consideration. They argue that modern factor analysis cannot distinguish between Spearman's and Tomson's models; that is, they are mathematically equivalent in terms of fit to the data.

Tredoux (2025) contends that there is a lack of substantive evidence for Thomson's model and that it relies heavily on probabilistic reasoning and arguments based on chance. Tredoux further notes that Thomas' model serves as a method to illustrate cognitive functioning rather than how the mind actually works.

 

There are some competing process models of human ability. For example, the construct “working memory” has been proposed as a central organizing system for learning and action (Baddeley & Hitch, 1974; Miller et al., 1960).

 

A range of cognitive processes has been proposed to account for the observed relationships, including processing speed, executive functioning, levels of processing theory (Craik & Lockhart, 1972), and the Cattell-Horn-Carrol (CHC) three-stratum theory of intelligence (Carroll, 1996). These perspectives have also been discussed in relation to Spearman's contribution to theories of human abilities (see Dennis & Tapsfield, 1996).

 

The model selected by a researcher to represent general and specific cognitive abilities carries significant implications for the interpretation of cognitive structure. To properly interpret the results of a cognitive structure factor analysis, it is essential to remove the variance attributable to general cognitive ability (g) from specific (s) factors and ensure that they are orthogonal to one another. Without orthogonality, the analysis cannot effectively distinguish whether the variance is predominantly associated with g or with specific cognitive abilities. For example, hierarchical factor analysis may implicitly support the view that specific abilities are subsumed under g due to the inherent dependence structure of the model.

 

In contrast, bifactor models propose that all observed indicators (e.g., test scores) are directly influenced by a general factor (g), while specific factors (s) directly influence only a subset of indicators. Importantly, these specific factors are orthogonal both to g and to each other, thus precluding any assumptions regarding their origin or causal relationship to g.

 

Despite their mathematical distinction, unrotated principal components analysis and unrotated principal factors analysis yield orthogonal components or orthogonal factors and do not allow attributing causal origins. Consequently, differences in analytical approaches can lead to divergent theoretical interpretations of cognitive structure, particularly regarding the nature and independence of specific abilities.

 

There are several reasons for assessing specific cognitive abilities. Notable among them is the need to evaluate whether one or more of the specific abilities provide incremental predictive validity beyond general cognitive ability for important outcomes such as educational attainment, training success, career success, or job performance. Furthermore, the development and refinement of theoretical models of the structural organization of cognitive abilities or the possible origins of specific abilities necessitate precise measurement and empirical evaluation. The validity of conclusions drawn from such investigations depends both upon the appropriateness of the model and the reliability of the measures used. The accurate measurement of specific abilities facilitates the construction of detailed cognitive compendia, which are essential for theory development, individualized educational planning, and development of personnel measurement and selection instruments. There is no guarantee that tests or cognitive tasks created by researchers are isomorphic with brain structure and function. External tasks do not necessarily reveal internal brain structures and processes (Ree et al., 2024).

 

The purpose of this article is to present a critical analysis of key considerations in the assessment of specific abilities. The article presents six distinct approaches to defining specific abilities, followed by an analysis of the strengths and limitations of each. The discussion concludes with an evaluation of the implications these approaches have for both theoretical model development and applied practice. The approaches examined include: (a) Spearman's (1904) model, (b) face validity, (c) the use of unrotated factors and unrotated components (Hotelling, 1936), (d) hierarchical factor analysis (Holzinger & Swineford, 1937), (e) orthogonalized hierarchical factor analysis (Schmid & Leiman, 1957), and (f) bifactor analysis (Holzinger & Harman, 1938; Holzinger & Swineford, 1937; Mansolf & Reise, 2016).

Pardon typos and spelling errors-Message may be sent from iPhone and I've always had spelling problems :)

*****************************************
Kevin S. McGrew, PhD
Educational & School Psychologist
Director
Institute for Applied Psychometrics (IAP)
https://www.themindhub.com
******************************************

The genetics of g and specific cognitive #CHC abilites. A must read. Includes a commentary rant directed at my fellow #schoolpsychologist #IQ test researchers

https://www.biorxiv.org/content/10.1101/2022.02.05.479237v1

The genetics of specific cognitive abilities
Francesca Procopio, Quan Zhou, Ziye Wang, Agnieska Gidziela,  
View ORCID Profile
Kaili Rimfeld,  
View ORCID Profile
Margherita Malanchini, Robert Plomin
doi: https://doi.org/10.1101/2022.02.05.479237 
This article is a preprint and has not been certified by peer review [what does this mean?].
0000000

• Abstract

Abstract
Most research on individual differences in performance on tests of cognitive ability focuses on general cognitive ability (g), the highest level in the three-level Cattell-Horn-Carroll (CHC) hierarchical model of intelligence. About 50% of the variance of g is due to inherited DNA differences (heritability) which increases across development. Much less is known about the genetics of the middle level of the CHC model, which includes 16 broad factors such as fluid reasoning, processing speed, and quantitative knowledge. We provide a meta-analytic review of 863,041 monozygotic-dizygotic twin comparisons from 80 publications for these middle-level factors, which we refer to as specific cognitive abilities (SCA). Twin comparisons were available for 11 of the 16 CHC domains. The average heritability across all SCA is 55%, similar to the heritability of g. However, there is substantial differential heritability and the SCA do not show the dramatic developmental increase in heritability seen for g. We also investigated SCA independent of g (g-corrected SCA, which we refer to as SCA.g). A surprising finding is that SCA.g remain substantially heritable (53% on average), even though 25% of the variance of SCA that covaries with g has been removed. Our review frames expectations for genomic research that will use polygenic scores to predict SCA and SCA.g. Genome-wide association studies of SCA.g are needed to create polygenic scores that can predict SCA profiles of cognitive abilities and disabilities independent of g. These could be used to foster children's cognitive strengths and minimise their weaknesses.

Blog masters comment.  Although there are constant critical battles re the relevance of intelligence testing and CHC theory in school psychology (my professional roots), many of the critics fail to recognize that intelligence theory and research is not owned by the small applied sandbox of school psychology.  Mainstream psychological researchers (i.e., serious scholars of intelligence research, cognitive psychologists, neuropsychologists, genetic psychologists, etc.) are finding tremendous value in using the CHC taxonomy to organize research, meta-analysis, and commentaries.  The use of the CHC taxonomy to organize research and literature reviews is one of the most important contributions of CHC theory.  I have repeatedly articulated the importance of this contribution in many publications since 1997…the most recent being my chapter with Joel Schneider in Contemporary Intellectual Assessment (4th edition).  I was pleased to see our contribution reflected in this current paper by the authors using one of our taxonomic organizations to ground their research.  The image is below, which is from our prior 2012 chapter.

What I find interesting is that many CHC specific cognitive abilities demonstrated significant heritabilities after g-heritability variance is statistically removed, and, the specific cognitive abilities do NOT show the same developmental heritability trend as g.  These findings, in part, are consistent with John Horn's long standing arguement that psychometric g is just a statistical abstraction and that differential developmental, heritability, neurocognitive, and outcome relations evidence provides strong support for the validity of the broad CHC factors…with Horn going as far as saying g does not really exist as a real brain or biological based mechanism.  And, recent psychometric network-based theories (dynamic mutualism, wired intelligence, process overlap; brain network research) can account for the positive definite nature of correlation matrices (without involving a latent g variable), which, along with the failure to find a brain or biological based theoretical or psychological g despite over 100 years of research ("g is the Loch Ness Monster of Psychology", McGrew, 2021) suggests that psychometric g is just that…a statistical abstraction.  The findings in this paper need to be "chewed on" by those SP IQ test structural researchers who seem to be "stuck on psychometric g"….I call them "g-centrists" (McGrew et al., 2022 manuscript in preparation).  

Yes, my fellow SP's, our strong "staticism" dust-bowel empirical factor studies of IQ test batteries (often with the same blunt hammer every time…….bi-factor or EFA with SL orthogonalization; I was guilty of being similarly enamored with CFA as the ultimate "truth test" early in my career before I learned better, through my ongoing consultation and guidance by two important mentors—John Horn and especially Jack McArdle, who unfortunately passed away last week) are almost completely devoid of the integration of substantive mainstream research and theory.  Much of our structural research may be statistically strong….but it is substantively very weak and nearly void of substantive research and theory.  We are contributing to the theory crises in psychology (Fried, 2020), especially via the conflation of psychometric/statistical g with theoretical/psychological g.  Intelligence and IQ tests are not the sole property of SP…research on CHC and other theories of intelligence, and providing valid psychometric measures of these constructs, is important for moving all of psychologies (and related fields) intelligence-related theoretical research forward.  Thanks for letting me vent.  Click on images to enlarge.




Some findings.

******************************************
Kevin S. McGrew, PhD
Educational & School Psychologist
Director
Institute for Applied Psychometrics (IAP)
https://www.themindhub.com
******************************************

g, working memory, specific CHC abilities and achievement

[Another set of “hidden on my hard disk analyses" that now see the light of day.]

Much has been written, both in the theoretical and applied intelligence theory/assessment literature, about

• The definition of g (general intelligence),
• Whether g exists,
• The importance of g in the prediction of a wide variety of outcomes, and
  
• If g exists, the importance of specific broad and narrow stratum abilities above and beyond g in the prediction of outcomes

By way of background, I have previously summarized the primary issues in “Betwixt, Behind, and Beyond g.” (Editorial note - please read this prior information before going further in this post.) In those prior writings I presented SEM analyses (of the WJ III norm data) that supported the hypothesis that working memory (actually, cognitive efficiency as defined by Gs and working memory) has a strong causal relationship to g (i.e., if you believe g is a workable and sound construct.) I also summarized research suggesting that a number of broad and narrow CHC abilities are significant above and beyond the influence of g.

So....what is this topic of this current post? Simple. To combine the working memory--> g information processing causal hypothesized model with the more traditional psychometric g+specific abilities research model.

Using the same norm samples as described in Betwixt, Behind, and Beyond g, I ran SEM causal models that defined g via the broad CHC latent abilities of Gv, Ga, Glr, Gf, and Gc. Additionally, adhering to a general information processing causal CHC model (click here to see prior relevant post), Gs and memory span (MS) are specified to be casually related to working memory, and working memory is in turn causally related to g.

The new twist is the specification of a causal path from g (as defined above) to achievement variables (letter-word identification; word attack) plus the inclusion of any significant paths from the cognitive efficiency variables (Gs and working memory) and/or the CHC abilities (that define the measurement model for g) to the dependent achievement latent variables.

All plausible models are presented and can be viewed by clicking here (Editorial note – for each age group presented, the models reported are all equally plausible. The model fit statistics did not identify any of the models in the two age groups as being better fits than the others).

Enjoy. I’d like to encourage the readers of this blog to offer interpretations and hypothesis for the various models. My only comment is that if you believe in g, the hypothesis that g is strongly related to working memory is supported (actually, to the broader notion of cognitive efficiency). Furthermore, some specific CHC abilities are again found to provide potentially important insights above and beyond the effect of g within a CHC-based information processing model.

For all who immediately shout “but what about parsimony?....what about Occam’s Razor?”….I defer to Stankov, Boyle & Cattell, R. (1995; Models and paradigms in personality and intelligence research. In Saklofske, D. & Zeidner, M. (Eds.) International Handbook of Personality and Intelligence. New York: Plenum Press), who state, within the context of research on human intelligence, that:

• “While we acknowledge the principle of parsimony and endorse it whenever applicable, the evidence points to relative complexity rather than simplicity. Insistence on parsimony at all costs can lead to bad science. “

CHC g+specific abilities and math ach study published

Another CHC g+specific abilities research study has been published (School Psychology Quarterly) by the research team of Taub, Floyd, Keith and McGrew (that be me, the blogmaster). This study was listed as "in press" in a prior posting announcing the latest CHC g+specific abilities reading publication from our little group. The abstract is below. A copy can be viewed by clicking the "study" link above. [Conflict of interest note - I'm a coauthor of the WJ III, the norm database analyzed for the study]

This program of research continues to suggest that certain specific CHC (Gf-Gc) abilities are important in understanding school achievement above and beyond the effect of g (general intelligence). This study is already included in the ongoing CHC Cognitive Abilities-Achievement Meta-Analysis Project.

Abstract

• This study investigated the direct and indirect effects of general intelligence and 7 broad cognitive abilities on mathematics achievement. Structural equation modeling was used to investigate the simultaneous effects of both general and broad cognitive abilities on students’ mathematics achievement. A hierarchical model of intelligence derived from the Cattell–Horn–Carroll (CHC) taxonomy of intelligence was used for all analyses. The participants consisted of 4 age-differentiated subsamples (ranging from ages 5 to 19) from the standardization sample of the Woodcock–Johnson III (WJ III; Woodcock, McGrew, & Mather, 2001). Data from each of the 4 age-differentiated subsamples were divided into 2 data sets. At each age level, one data set was used for model testing and modification, and a second data set was used for model validation. The following CHC broad cognitive ability factors demonstrated statistically significant direct effects on the mathematics achievement variables: Fluid Reasoning, Crystallized Intelligence, and Processing Speed. In contrast, across all age levels, the general intelligence factor demonstrated indirect effects on the mathematics achievement variable.

Technorati Tags: psychology, educational psychology, education, school psychology, neuropsychology, WJ III, CHC, Cattell-Horn-Carroll, g+specific, general intelligence, IQ, IQ tests, IQ scores, mathematics, math, Gq

WISC-III/WJ III cross-battery g+specific cog-ach abilities findings

WISC-III/WJ III cross-battery g+specific abilities research reinforces "just say maybe" program of CHC g+specific abilities research.

I'm just starting the process of drafting a manuscript to summarize the results of the IAP CHC COG-ACH Correlates Meta-Analysis project (click here).  In that on-line EWOK (Evolving Web of Knowledge) I list a McGrew (2007a) study in the reading and math summary tables.  That reference reflects unpublished re-analysis I completed (last fall) with the Phelps et al. (2007) joint (cross-battery) WISC-III/WJ III dataset.  In order to include the findings in the synthesis manuscript, I felt it appropriate to at least informally publish the final models for reference. 

Two important caveats.  I'm a coauthor of the WJ III (conflict of interest disclosure).  The second caveat is outlined in the brief report of the results that I have posted.  When I have time I will update the synthesis EWOK. I will change the reference to McGrew (2008) reflecting posting.

Technorati Tags: psychology, educational psychology, school psychology, neuropsychology, intelligence, IQ testing, IQ, Wechsler, WISC-III, WJ III, general intelligence, g, g+specific abilities

Intelligence (IQ) and work

I just stumbled across a very nice review article (by Linda Gottfredson in the Journal of Career Assessment, 2003) that provides an overview of the state-of-the art of cognitive ability assessment in vocational psychology. Vocational/career assessment is not my cup of tea, but I have high regard for the work of Gottfredson, a top scholar in the field of intelligence (see link to her web page under "IQ Scholars" section of blog). Below is the abstract, followed by a few featured conclusions.

Abstract

• Abilities are as important as interests in career choice and development. Reviving cognitive assessment in career counseling promises to help counselees better understand their career options and how to enhance their competitiveness for the ones they prefer. Nearly a century of research on human cognitive abilities and jobs’ aptitude demands in the U.S. economy reveals that the two domains are structured in essentially the same way. The author describes that common structure and how it can be used in assessing person-job match in terms of general ability level and ability profile. She also suggests ways of resolving various technical and professional questions, such as which cognitive abilities to assess, how to assess them, what the most useful aptitude-based occupational classification would be, and how to use cognitive assessments in a broader “reality-based exploration” process intended to expand people’s career opportunities.

What I particularly liked was Gottfredson's use of the Carroll tri-stratum model (aka, CHC theory) as the preferred model for understanding cognitive abilities in a vocational/career setting. The literature review is a nice overview of the major conclusions re: the effectiveness/utility of general, broad, and narrow cognitive abilities in vocational and career assessment research. A few interesting tidbits from her article related to utility of cognitive abilities as a function of breadth (stratum level) are noted below. There is much more to the article, so I'm only featuring a few points I found of interest as it relates to my interest and research in CHC theory. Gottfredson provides an excellent overview of the role of non-cognitive traits (personality, interests, emotional intelligence, etc.), but I'll let the reader decide if they want to read this information (space constraints of this blog format). In most cases the statements below are direct quotes.


Narrow abilities: The more specific or narrow an ability is, the more trainable and subject to shared family influences it seems to be. These facts are exceedingly important. They mean that there may be much scope for changing our narrower skills but that we must work with (rather than expect to change) our most general abilities.

General intellignece (g): The most general abilities are the best overall predictors of job performance... Meta-analyses have shown that the most general ability, g, predicts performance to some extent in all jobs In contrast, Stratum II (broad) abilities add little to the prediction of performance above and beyond g and then only in selected groups of jobs (e.g., spatial ability in certain technical and artistic jobs, speeded tests of clerical ability in clerical jobs). Stratum II abilities tend to predict job performance fairly well but generally only to the extent that they also reflect g...only that more general abilities are more broadly useful across the great variety of tasks and settings that we encounter in the workplace....low
general abilities greatly constrain the range of a person’s options in the workplace.

Broad abilities: Stratum II abilities are highly correlated among individuals of below average IQ (g), but the correlations weaken in higher IQ populations. In other words, brighter individuals tend to have more jagged Stratum II ability profiles. This suggests that profile shape may add more to the prediction of performance, beyond profile level, in higher IQ populations.....The broad cognitive abilities tend not to correlate much with either vocational interests or personality (e.g., Ackerman &Heggestad, 1997). The relative independence of the cognitive and noncognitive domains means that cognitive assessments provide useful information that cannot be obtained from the noncognitive inventories of personal traits. There are important points of overlap, as we shall see, but they are fairly localized.

Why cognitive tests are useful: Contrary to initial expectation, mental tests predict performance to some extent in all jobs, best in the most cognitively complex jobs, best when performance is measured objectively and relates to the most core technical duties of a job, and almost always better than any other type of predictor. Moreover, specifically tailored cognitive test batteries do little better than a lone measure of g. Tests of spatial and clerical aptitudes add slightly to variance predicted but only in certain groups of occupations. Assessments of less cognitive traits (temperament, interests, and the like) also add little to the prediction of core job performance, although they do outpredict g when organizational “citizenship” (helpful to coworkers, professional demeanor, and such) is the performance criterion. Psychomotor abilities and length of experience in a job predict performance best where g predicts it least, namely, in the lowest level, most routine jobs. The advantages of relatively greater experience fade at higher average levels of experience, but the advantages of higher g do not.

Broad and g relative importance: Thus, when we look across the full landscape of occupations, two features of ability requirements stand out. First, the single Stratum III ability, g, is relevant to all jobs but especially so in the most cognitively demanding and prestigious ones. Second, differences in ability profiles at the Stratum II level also matter, but they are secondary in importance. Their effects are smaller and more narrowly
targeted.

Figure 2 (Occupational aptitude patterns map) is a neat way to demonstrate the relations between cognitive ability and 12 clusters of work. Check it out. I love "big picture" summary charts and figures.

I'd have to examine the research studies reviewed by Gottfredson, but I wonder if the power of "g" may be overestimated in this literature due to the same type of methodological limitations that, for years, tended to suggest the same "g"-dominance in understanding school achievement. More recent g+specific abilities school achievement research which has used comprehensive measures of the major CHC abilities and employed SEM methods have suggested that more specific (broad or narrow abilities) are important "above and beyond" the influence of "g" and that the influence of "g" may be more of an indirect effect mediated via broad and narrow abilities. I'd like to see similarly designed studies in this field.

Technorati Tags: psychology, educational psychology, school psychology, vocationnal psychology, careers, career assessment, IQ, intelligence, IQ scores, general intelligence, CHC, CHC theory, Cattell-Horn-Carroll, interests

IAP101 Brief #12: Use of IQ component part scores as indicators of general intelligence in SLD and MR/ID diagnosis

   
            Historically the concept of general intelligence (g), as operationalized by intelligence test battery global “full scale” IQ scores, has been central to the definition and classification of individuals with a specific learning disability (SLD) as well as individuals with an intellectual disability (ID).  More recently, contemporary definitions and operational criteria have elevated intelligence test battery composite or part scores to a more prominent role in diagnosis and classification of SLD and more recently in ID.

            In the case of SLD, third-method “consistency” definitions prominently feature component or part scores in (a) the identification of consistency between low achievement and relevant cognitive abilities or processing disorders and (b) the requirement that an individual demonstrate relative cognitive and achievement strengths (see Flanagan, Fiorello & Ortiz, 2010).  The global IQ score is de-emphasized in the third-method SLD methods.

            In contrast, the 11^th edition of the AAIDD Intellectual Disability: Definition, Classification, and Systems of Supports manual (AAIDD, 2010) placed general intelligence, and thus global composite IQ scores, as central to the definition of intellectual functioning.  This has not been without challenge.  For example, the AAIDD ID definition has been criticized for an over-reliance on the construct of general intelligence and for ignoring contemporary psychometric theoretical and empirical research that has converged on a multidimensional hierarchical model of intelligence (viz., Cattell-Horn-Carroll or CHC theory).

The potential constraints of the “ID-as-a-general-intelligence-disability” definition was anticipated by the Committee on Disability Determination for Mental Retardation, in its National Research Council report “Mental Retardation:  Determining Eligibility for Social Security Benefits” (Reschly, Meyers & Hartel, 2001).  This national committee of experts concluded that “during the next decade, even greater alignment of intelligence tests and the IQ scores derived from them and the Horn-Cattell and Carroll models is likely.  As a result, the future will almost certainly see greater reliance on part scores, such as IQ scores for Gc and Gf, in addition to the traditional composite IQ.  That is, the traditional composite IQ may not be dropped, but greater emphasis will be placed on part scores than has been the case in the past” (Reschly et al., 2002, p. 94).  The committee stated that “whenever the validity of one or more part scores (subtests, scales) is questioned, examiners must also question whether the test’s total score is appropriate for guiding diagnostic decision making.  The total test score is usually considered the best estimate of a client’s overall intellectual functioning.  However, there are instances in which, and individuals for whom, the total test score may not be the best representation of overall cognitive functioning.” (p. 106-107).

            The increased emphasis on intelligence test battery composite part scores in SLD and ID diagnosis and classification raises a number of measurement and conceptual issues (Reschly et al., 2002).  For example, what are statistically significant differences?  What is a meaningful difference?  What appropriate cognitive abilities should serve as proxies of general intelligence when the global IQ is questioned?  What should be the magnitude of the total test score? 

Appropriate cognitive abilities will only be the only issue discussed here.  This issue addresses  which component or part scores are more correlated with general intelligence (g)—that is, what component part scores are high g-loaders?  The traditional consensus has been that measures of Gc (crystallized intelligence; comprehension-knowledge) and Gf (fluid intelligence or reasoning) are the highest g-loading measures and constructs and are the most likely candidates for elevated status when diagnosing ID (Reschly et al., 2002).  Although not always stated explicitly, the third method consistency SLD definitions specify that an individual must demonstrate “at least an average level of general cognitive ability or intelligence” (Flanagan et al., 2010, p.745), a statement that implicitly suggests cognitive abilities and component scores with high g-ness.

Table 1 is intended to provide guidance when using component part scores in the diagnosis and classification of SLD and ID (click on images to enlarge and use the browser zoom feature  to view; it is recommended you click here to access a PDF copy of the table..and also zoom in on it).  Table 1 presents a summary of the comprehensive, nationally normed, individually administered intelligence batteries that possess satisfactory psychometric characteristics (i.e., national norm samples, adequate reliability and validity for the composite g-score) for use in the diagnosis of ID and SLD.

The “Composite g-score” column lists the global general intelligence score provided by each intelligence battery.  This score is the best estimate of a person’s general intellectual ability, which currently is most relevant to the diagnosis of ID as per AAIDD.  All composite g-scores listed in Table 1 meet Jensen’s (1998) psychometric sampling error criteria as valid estimates of general intelligence.  As per Jensen’s number of tests criterion, all intelligence batteries g-composites are based on a minimum of nine tests that sample at least three primary cognitive ability domains.  As per Jensen’s variety of tests criterion (i.e., information content, skills and demands for a variety of mental operations), the batteries, when viewed from the perspective of CHC theory, vary in ability domain coverage—four (CAS, SB5), five (KABC-II, WISC-IV, WAIS-IV), six (DAS-II) and seven (WJ III) (Flanagan, Ortiz & Alfonso, 2007; Keith & Reynolds, 2010).   As recommended by Jensen (1998), “the particular collection of tests used to estimate g should come as close as possible, with some limited number of tests, to being a representative sample of all types of mental tests, and the various kinds of test should be represented as equally as possible” (p. 85).  Users should consult sources such as Flanagan et al. (2007) and Keith and Reynolds, 2010) to determine how each intelligence battery approximates Jensen’s optimal design criterion, the specific CHC domains measured, and the proportional representation of the CHC domains in each batteries composite g-score.

Also included in Table 1 are the component part scales provided by each battery (e.g., WAIS-IV Verbal Comprehension Index, Perceptual Reasoning Index, Working Memory Index, and Processing Speed Index), followed by their respective within-battery g-loadings.[1]  Examination of the g-ness of composite scores from existing batteries (see last three columns in Table 1) suggests the traditional assumption that measures of Gf and Gc are the best proxies of general intelligence may not hold across all intelligence batteries.[2] 

In the case of the SB5, all five composite part scores are very similar in g-loadings (h² = .72 to .79).  No single SB5 composite part score appears better than the other SB5 scores for suggesting average general intelligence (when the global IQ score is not used for this purpose).  At the other extreme is the WJ III where the Fluid Reasoning, Comprehension-Knowledge, Long-term Storage and Retrieval cluster scores are the best g-proxies for part-score based interpretation within the WJ III.  The WJ III Visual Processing and Processing Speed clusters are not composite part scores that should be emphasized as indicators of general intelligence.  Across all batteries that include a processing speed component part score (DAS-II, WAIS-IV, WISC-IV, WJ III) the respective processing speed scale is always the weakest proxy for general intelligence and thus, would not be viewed as a good estimate of general intelligence. 

            It is also clear that one cannot assume that composites with similar sounding names of measured abilities should have similar relative g-ness status within different batteries.  For example, the Gv (visual-spatial or visual processing) clusters in the DAS-II (Spatial Ability), SB5 (Visual-Spatial Processing) are relatively strong g-measures within their respective battery, but the same cannot be said for the WJ III Visual Processing cluster.  Even more interesting are the differences in the WAIS-IV and WISC-IV relative g-loadings for similarly sounding index scores. 

For example, the Working Memory Index is the highest g-loading component part score (tied with Perceptual Reasoning Index) in the WAIS-IV but is only third (out of four) in the WISC-IV.   The Working Memory Index is comprised of the Digit Span and Arithmetic subtests in the WAIS-IV and the Digit Span and the Letter-Number Sequencing subtests in the WISC-IV.  The Arithmetic subtest has been reported to be a factorially complex test which may tap fluid intelligence (Gf-RQ—quantitative reasoning), quantitative knowledge (Gq), working memory (Gsm), and possible processing speed (Gs; Keith & Reynolds, 2010; Phelps, McGrew, Knopik & Ford, 2005).   The factorially complex characteristics of the Arithmetic subtest (which, in essence, makes it function like a mini-g proxy) would explain why the WAIS-IV Working Memory Index is a good proxy for g in the WAIS-IV but not in the WISC-IV. The WAIS-IV and WISC-IV Working Memory Index scales, although named the same, are not measuring identical constructs.

A critical caveat is that the g-loadings cannot be compared across different batteries.  g-loadings may change when the mixture of measures included in the analyses change.  Different "flavors" of g can result (Carroll, 1993; Jensen, 1998). The only way to compare the g-ness across batteries is with appropriately designed cross- or joint-battery analysis (e.g., WAIS-IV, SB5 and WJ III analyzed in a common sample).

The above within and across intelligence battery examples illustrates that those who use component part scores as an estimate of a person’s general intelligence must be aware of the composition and psychometric g-ness of the component scores within each intelligence battery.  Not all component part scores in different intelligence batteries are created equal (with regard to g-ness).  Also, not all similarly named factor-based composite scores may measure the same identical construct and may vary in degree of within battery g-ness.  This is not a new problem in the context of naming factors in factor analysis, and by extension, factor-based intelligence test composite scores, Cliff (1983) described this nominalistic fallacy in simple language—“if we name something, this does not mean we understand it” (p. 120). 

---

[1] As noted in the footnotes in Table 1, all composite score g-loadings were computed by Kevin McGrew by entering the smallest number (and largest age ranges covered) of the published correlation matrices within each intelligence batteries technical manual (note the exception for the WJ III) in order to obtain an average g-loading estimate.  It would have been possible to calculate and report these values for each age-differentiated correlation matrix for each intelligence battery.  However, the purpose of this table is to provide the best possible average value across the entire age-range of each intelligence battery.  Floyd and colleagues have published age-differentiated g-loadings for the DAS-II and WJ III.  Those values were not used as they are based on the use of the principal common factor analysis method, a method that  analyzes the reliable shared variance among tests.  Although principal factor and principal component loadings typically will order measures in the same relative position, the principal factor loadings typically will be lower.  Given that the imperfect manifest composite scale scores are those that are utilized in practice, and to also allow uniformity in the calculation of the g-loadings reported in Table 1, principal component analysis was used in this work. The same rationale was used for not using the latent factor loadings on a higher-order g-factor in SEM/CFA analysis of each test battery.  Loadings from CFA analyses represent the relations between the underlying theoretical ability constructs and g purged of measurement error.  Also, frequently the final CFA solutions reported in a batteries technical manual (or independent journal articles) allow tests to be factorially complex (load on more than one latent factor), a measurement model that does not resemble the real world reality of the manifest/observed composite scores used in practice.  Latent factor loadings on a higher-order g-factor will often differ significantly from principal component loadings based on the manifest measures, both in absolute magnitude and relative size (e.g., see high Ga loading on g in WJ III technical manual which is at variance with the manifest variable based Ga loading reported in Table 1) 

[2] The h² values are the values that should be used to compare the relative amount of g-variance present in the component part scores within each intelligence battery.

Is associative learning/memory (Glr-MA) a causal mechanism of g (general intelligence)?

What are basic elementary cognitive mechanisms/processes that contribute to (cause?) g (general intelligence).

As has been discussed many times previously on this blog, there has been a significant amount of recent research suggesting that working memory (Gsm-MW) and processing speed (Gs) may be causal mechanisms of g (click here for a brief summary and some CHC-based causal models I've run). In the article below, Kaufman et al (hereafter referred to as the "authors") investigate whether associative learning may be another causative mechanism...above and beyond the contribution of working memory and processing speed.

• Kaufman, S. B, DeYoung, C. G, Gray, J. R., Brown, J. & Mackintosh, N. (2009). Associative learning predicts intelligence above and beyond working memory and processing speed. Intelligence, 37, 374–382 (click here to view)
  

Abstract

Recent evidence suggests the existence of multiple cognitive mechanisms that support the general cognitive ability factor (g). Working memory and processing speed are the two best established candidate mechanisms. Relatively little attention has been given to the possibility that associative learning is an additional mechanism contributing to g. The present study tested the hypothesis that associative learning ability, as assessed by psychometrically sound associative learning tasks, would predict variance in g above and beyond the variance predicted by working memory capacity and processing speed. This hypothesis was confirmed in a sample of 169 adolescents, using structural equation modeling. Associative learning, working memory, and processing speed all contributed significant unique variance to g, indicating not only that multiple elementary cognitive processes underlie intelligence, but also the novel finding that associative learning is one such process.

ARTICLE HIGHLIGHTS

(italics indicates direct quotes; underline indicates emphasis added by blogmaster)

the existence of general intelligence (g), in the sense of a statistical feature (a “positive manifold”), is a robustfinding, it is less clear what the mechanisms are that support g. The best established candidate processes, as mechanistic substrates of g, are processing speed (Deary, 2001) and working memory (Conway, Jarrold, Kane, Miyake, & Towse, 2007)

The current study investigated associative learning as a potential additional candidate process, which might contribute to g over and above processing speed and working memory. Only recently has associative learning become a serious contender as a substrate of g (Alexander & Smales, 1997; Tamez, Myerson, & Hale, 2008; Williams, Myerson, & Hale, 2008; Williams & Pearlberg, 2006). There is good reason, however, to suspect that the ability to learn associations might support g. Intelligent behavior seems certain to require memory for patterns of associations among stimuli, and one of the original purposes of intelligence tests was to assess students' ability to learn (Binet & Simon, 1916). Relations among associative learning, general cognitive ability, and cognitive mechanisms that subserve general cognitive ability are thus of interest for both theoretical and historical reasons.

The authors define associative learning as "the ability to remember and voluntarily recall specific associations between stimuli" In this regard, it is similar to associative memory (MA) under Glr as per the CHC taxonomy.

Although early studies found a weak or no relation between associative learning and general cognitive ability (Malmi, Underwood, & Carroll, 1979; Underwood, Boruch, & Malmi, 1978; Woodrow, 1938, 1946), the failure to find a relation seems likely to be due to the fact that the associative learning tests that were used in these studies were easy and thus unlikely to be related to complex cognition

The sample was "169 participants (54 males and 115 females) included in the analysis were aged 16–18 years, and attended a selective Sixth Form College (which takes high-achieving students who are in their last 2 years of secondary education) in Cambridge, England."

The authors operationally defined an associative learning (AL) latent factor (Glr-MA as per CHC as interpreted by the blogmaster) via multiple associative learning indicators (see article). Gf was operationally measured by the Raven's advanced progressive matrices (RAPM), DAT verbal reasoning test and the Mental rotations test. Mulitple indicators of Gs included the Verbal speed test (Speed-V) from the Berlin model of Intelligence Structure, the Numerical speed test (Speed-N), and the Speed of Information Processing sub-test from the British Ability Scales. The working memory measurement model consisted of the Operation span task. The authors used structural equation method to test predictions about the independent prediction of g by associative learning (Glr-MA), working memory (Gsm-MW), and processing speed (Gs). The final causal model can be seen in the included visual-graphic causal model diagram. Note that, as per the CHC taxonomy, I would relabel the latent factors as Glr-MA (not AL), Gsm-MW (not WM), and Gs (the same as the authors).

[double click on image to enlarge]

The primary results, as represented in the above causal model diagram:

showed that associative learning, working memory, and processing speed all made statistically independent contributions to g. This finding suggests that each of these elementary cognitive processes may represent a mechanism that contributes differentially to general intelligence

Conclusions of authors

The results of the current study add to a growing literature on the existence of multiple cognitive mechanisms that support general cognitive ability (Sternberg & Pretz, 2005). Our findings suggest that multiple cognitive processes — including the abilities to process information quickly, to maintain, update, and manipulate information in working memory, and to learn specific associations between stimuli — should contribute to performance on any highly g-loaded task. Identification of separable elementary cognitive mechanisms that support g should further attempt to develop neurobiological theories of intelligence. Such theories may help to resolve current debates regarding the nature of the mechanisms underlying g (e.g., Colom, Franciso, Quiroga, Shih, & FloresMendoza, 2008). Evidence exists already that working memory and associative learning rely on different regions of the PFC (Petrides, 1995, 2000; Petrides et al., 1993), and processing speed seems likely to be determined by a distinct set of biological parameters that are not yet known. The investigation of the precise number and nature of the mechanisms that underlie g remains a promising line of research.

Technorati Tags: psychology, school psychology, educational psychology, neuropsychology, cognition, intelligence, g, general intelligence, ISIR, ISIR corner, associative memory, fluid intelligence, working memory