Research Byte: Intellectual Differences Between Boys and Girls, 35 Years of Evolution in France from WISC-R to WISC-V

 Abstract

The French adaptation of the Wechsler Intelligence Scale of Children, 5th edition (WISC-V) was an opportunity to examine if some common representations of gender differences in intellectual abilities are supported by empirical evidence. The WISC-V standardization sample provided data on a wide range of cognitive tests in a large sample of 6- to 16-year-old children representative of the French population. This sample included 517 boys and 532 girls. The WISC-V data were compared to those of the French standardization samples of three previous versions of the WISC (WISC-R, WISC-III, and WISC-IV). These four standardization samples span a 35-year period. The data analysis of the WISC-V standardization sample and the three previous versions of this intelligence scale showed that the performance gaps on intellectual tests between girls and boys have gradually narrowed over time. Almost no gender differences were observed in the WISC-V standardization sample, not only in IQ but also in key facets of intelligence. Data do not support the stereotype that girls are better at verbal tasks and boys are better at visuospatial tasks. However, some statistically significant differences remain, but the magnitude was generally small with no practical implications. The only important difference is in favor of girls and concerns performance on processing speed tasks that require visual discrimination, attentional control, and writing.

Keywords: 

intelligence; gender; WISC-III; WISC-IV; WISC-V

Click to view or download article at Journal of Intelligence.

The Wechsler Arithmetic subtest measures quantitative reasoning...another study

A new study on the now "old" WISC-III which still provides insights into the debate regarding what the Wechsler Arithmetic subtest measures. Consistent with research I have coauthored and my analysis of other studies (click here to view), this new study is consistent with the classification of Arithmetic as primarily a measure of quantitative reasoning.

Click on images to enlarge.







- Posted using BlogPress from my iPad

Cluster analysis of the WJ III/WISC-III intelligence tests: OBG post

This is a OBG (oldie but goodie) post that has new updated links

In a prior shameless plug, I briefly summarized the results of a recently published CHC-based confirmatory factor analysis study of a WJ-III/WISC-III cross-battery data set (Phelps, McGrew, Knopik & Ford, 2005). Following a favorite quantoid mantra ("there is more than one way to explore a data set"), I couldn't resist but conduct a more loosey-goosey (sp?) exploratory analysis of the data.

One of my favorite exploratory tools, given the Gv presentation of the multivariate structure of the data, is hierarchical cluster analysis (sometimes referred to as the "poor man's" factor analysis). Without going into detail, I subjected the data set previously described to Ward's clustering algorithm. As a word of caution, it is important to note that cluster analysis will provide neat looking cluster dendograms for random data....so one must be careful not to over-interpret the results. Yet, I find the looser constraints of cluster analysis and, in particular, the continued collapsing of clusters of tests (and lower-order clusters) into ever increasing broad higher-order clusters very thought provoking---the results often suggest different broad (stratum II) or intermediate level strata (as per Carroll's 3-stratum model).

I present the current results "as is" (click here to view or download). Blogsters will need to consult prior posts to glean the necessary pieces of information to interpret the CHC factor codes and names, the abilities measured by the WJ III tests, etc.

To say the least, some interesting hypothesis are suggested. In particular, I continue to be intrigued by the possibility of a higher-order dual cognitive processing model structure (within the CHC taxonomy) --that is, a distinction between automatic vs controlled/deliberate processing

Historical review of WISC to WISC-IV

This article is an open access article and thus I am providing a link to a copy.

The article helps users understand the various changes that have occurred in this series, changes which may help explain IQ score differences in individuals who may have taken various versions over time.

 

Posted with Blogsy

Another Flynn Effect study: Impact on LD diagnosis

This article will be added to the Flynn Effect Archive Project when it is updated next. Click on image to enlarge.


Posted using BlogPress from Kevin McGrew's iPad
www.themindhub.com

CHC theory and the Wechsler IQ scales and test development

In 1998 Dr. Dawn Flanagan and I published the Intelligence Test Desk Reference book which was the first thorough treatment of CHC theory (then called Extended Gf-Gc theory). This book is now out-of-print.


We then took the concepts from the ITDR and, together with Dr. Sam Ortiz, presented a cross-battery approach to interpreting the Wechsler batteries.


And again, this book is no longer in print. This also means we no longer receive any $ for sales (conflict of interest disclosure).


The research, theory, and conceptual material in the second book is nearly identical to the first, but it was presented in the context of how to upgrade interpretation and understanding of the Wechsler batteries according to the CHC framework. Since then the same CHC overview material has been tweaked and updated in a series of CHC cross-battery books by Flanagan et al. But, the foundation of CHC theory, and how it can be integrated within a conceptual framework of test development and interpretation, is largely the same in these newer CHC cross-battery books.

Thus, given that these "mother and father" books are no longer in print, I took the liberty of copying the first three chapters of the Wechsler oriented book and am now making them available for my readers (click here). I make this material available to provide psychologists who have not done much reading regarding CHC theory an opportunity to have access to the basic foundation of CHC theory to help them see how it can be applied to the interpretation of an intelligence battery (in this case the Wechslers). By choosing the Wechsler material this also helps understand how the Wechsler batteries are evolving (either implicitly or explicitly--see Keith and Reynolds, 2010) when viewed from the lens of CHC theory.

But, one must recognize that this material is a bit dated. An update of CHC theory was later published in 2005 (click here to access...plus some other chapters), and was again updated this year by Schneider and McGrew (click here).

However, the CHC chapter I provide in this blog post, particularly when placed in the context of the Wechsler batteries, provides a solid foundation for understanding CHC theory and its impact on contemporary intelligence test development and interpretation. My goal is to increase awareness of CHC theory and its relevance to psychological assessment and interpretation. My goal is to spur others to become more current re: this now dominant framework in the field of applied IQ testing.


- Posted using BlogPress from Kevin McGrew's iPad

iPost: New WISC-R to WISC III Flynn Edfect study

The Flynn Effect in the WISC Subtests Among School Children Tested for Special Education Services

Tomoe Kanaya and Stephen Ceci

Journal of Psychoeducational Assessment published 19 May 2010, 10.1177/0734282910370139
http://jpa.sagepub.com/cgi/content/abstract/0734282910370139v1

Abstract

The Flynn effect, a secular rise in IQ seen throughout the world, was examined on the WISC-R and WISC-III subtests in a longitudinal sample of more than 2,500 school children who were tested between 1974 and 2002. Multivariate analysis of variance and multiple regression analyses revealed that all the subtests experienced significant decreases in scores on the introduction of the WISC-III, as expected because of the Flynn effect, with the exception of Information and Digit Span.(Mazes was not included in the analyses because of a limited sample size.) On Picture Arrangement and Coding, however, children who were repeatedly tested on the WISC-III also experienced significant decreases compared with children who were repeatedly tested on the WISC-R. These findings add to the growing literature comparing the magnitudeof the Flynn effect on crystallized versus fluid measures. Implications for special education testing and the current WISC-IV are discussed.

Kevin McGrew PhD

Educational Psychologist 

IAP (www.iapsych.com)

FInd via Google:  IQs Corner

Sent from KMcGrew iPhone (IQMobile). (If message includes an image-double click on it to make larger-if hard to see) 

Research Bytes 4-24-10: WISC-IV & TBI and WAIS-IV factor study research studies

Allen, D. N., Thaler, N. S., Donohue, B., & Mayfield, J. (2010). WISC-IV Profiles in Children With Traumatic Brain Injury: Similarities to and Differences From the WISC-III. Psychological Assessment, 22(1), 57-64.

The Wechsler Intelligence Scale for Children—Fourth Edition (WISC–IV; D. Wechsler, 2003a) is often utilized to assess children with traumatic brain injury (TBI), although little information is available regarding its psychometric properties in these children. The current study examined WISC–IV performance in a sample of 61 children with TBI. As compared to the standardization sample, results indicated that the TBI group exhibited relative deficits on all subtest and index scores, with the greatest deficits on the Processing Speed Index (PSI) and Coding subtest scores. However, the Perceptual Reasoning Index score was not uniquely sensitive to brain injury, and the Cognitive Processing Index score was less sensitive to TBI than the PSI score. Also, the PSI did not uniquely predict learning and memory abilities, as had been reported in previous studies of the Wechsler Intelligence Scale for Children—Third Edition (WISC–III; D. Wechsler, 1991). The present findings indicate substantive differences between the WISC–III and WISC–IV profiles of children with TBI.

Benson, N., Hulac, D. M., & Kranzler, J. H. (2010). Independent Examination of the Wechsler Adult Intelligence
Scale-Fourth Edition (WAIS-IV): What Does the WAIS-IV Measure? Psychological Assessment, 22(1),
121-130.

Published empirical evidence for the Wechsler Adult Intelligence Scale—Fourth Edition (WAIS–IV) does not address some essential questions pertaining to the applied practice of intellectual assessment. In this study, the structure and cross-age invariance of the latest WAIS–IV revision were examined to (a) elucidate the nature of the constructs measured and (b) determine whether the same constructs are measured across ages. Results suggest that a Cattell–Horn–Carroll (CHC)–inspired structure provides a better description of test performance than the published scoring structure does. Broad CHC abilities measured by the WAIS–IV include crystallized ability (Gc), fluid reasoning (Gf), visual processing (Gv), short-term memory (Gsm), and processing speed (Gs), although some of these abilities are measured more comprehensively than are others. Additionally, the WAIS–IV provides a measure of quantitative reasoning (QR). Results also suggest a lack of cross-age invariance resulting from age-related differences in factor loadings. Formulas for calculating CHC indexes and suggestions for interpretation are provided.

Technorati Tags: Psychology, educational psychology, school psychology, clinical psychology, forensic psychology, neuropsychology, intelligence, IQ testing, IQ tests, WISC-IV, WISC-III, WAIS-IV, WAIS-III, CHC theory, Catell-Horn-Carroll theory, cognitive abilities, confirmatory factor analysis, TBI, traumatic brain injury

CHC theory: Emergence, test instruments and school-related research brief

Contemporary Cattell-Horn-Carroll (CHC) intelligence test development, interpretation and applied research can be traced to a fortuitous meeting of Richard Woodcock, John Horn, and John “Jack” Carroll in the fall of 1985, a meeting also attended by the first author of this web-resource ( McGrew, 2005). This meeting resulted in the 1989 publication of the first individually-administered, nationally standardized CHC-based intelligence battery, the Woodcock- Johnson- Revised (Woodcock, McGrew, & Mather, 1989). This landmark event, which occurred 20 years ago, provided the impetus for the major CHC- driven evolution of school- based intelligence testing practice.

Subsequent important CHC events followed during this 20 year period, and included: (a) the first set of CHC- organized joint test battery factor analysis studies (Woodcock, 1990) which planted the seeds for the concept of CHC cross-battery (CB) assessment, (b) the first attempt to use the WJ-R, via a Kaufman-like supplemental testing strategy (Kaufman, 1979), to implement the yet to be named and operationalized CHC CB approach to testing ( McGrew, 1993), (c) the articulation of the first integrated Cattell-Horn-Carroll model and classification of the major intelligence batteries as per the CHC framework (McGrew, 1997), (d) the first description of the assumptions, foundations, and operational principles for CHC CB assessment and interpretation (Flanagan & McGrew, 1997; McGrew & Flanagan, 1998), (e) the publication of the first intelligence theory and assessment book to prominently feature CHC theory and assessment methods (Contemporary Intellectual Assessment: Theories, Tests, and Issues; Flanagan, Genshaft & Harrison, 1997; click here for link to 2nd edition), (f) the publication of the CHC CB assessment series ( Flanagan, McGrew & Ortiz, 2000; Flanagan, Ortiz, Alfonso & Mascolo, 2006; Flanagan, Ortiz & Mascolo, 2001, 2007; McGrew & Flanagan, 1998), (g) the completion of a series of CHC-organized studies that investigated the relations between CHC cognitive abilities and reading, math, and writing achievement (what you are reading now), (h) the articulation of CHC-grounded SLD assessment and eligibility frameworks (see Flanagan & Fiorello, manuscript in preparation) and (h) the subsequent CHC- grounded revisions or interpretations of a number of comprehensive individually administered intelligence test batteries ( Differential Abilities Scales—II, DAS-II;Stanford- Binet—5^thEdition, SB5;Kaufman Assessment Battery for Children—2^ndEdition, KABC- II). Although not overtly stated, the impact of CHC theory can be seen in the recent revisions of the venerable Wechsler trilogy ( WPPSI-III; WISC- IV; WAIS- IV) as well as the presentation of CHC CB procedures for interpreting the three Wechsler batteries ( Flanagan et al., 2000).

Click here for other posts in this series.

WISC-III/WJ III cross-battery Guttman 2-D Radex analysis

One more WISC-III/WJ III cross-battery analysis--this time a 2-D Guttman Radex MDS model (click here).  As readers have noted, I've been on a bit of a data analysis binge this past week (in preparation for writing a manuscript---and after being refreshed by an actual 2+ week vacation) and have reported:  (a) WISC-III/WJ III cross-battery g+specific cog-ach relations SEM, (b) WJ III 2-D Guttman Radex MDS of WJ III norm sample ages 6-8, and (c) WJ III 3-D Guttman Radex MDS of ages 9-13 of norm sample.  It is hoped these analyses provide useful information in understanding the characteristics of the tests in the WJ III and Wechsler intelligence batteries.

Unfortunately this analysis is based on the WISC-III and not the more recent WISC-IV.  Nevertheless, the results still provide useful information on the WISC-III tests that are still present in the WISC-IV.

Given all I've written regarding the various MDS models, I'm going to only make a few comments and hope others take the presentation of these data to engage in additional discussion, interpretion, etc.----have some fun.

A few observations/comments:

• Gv tests (both WISC-III and WJ III) continue to surface on the more outer rings of the MDS models---suggesting that they are more lower-level perceptual/processing measures and do not capture complex Gv cognitive processing.  See my Gv comments on this the other day.  The same can be said for Ga tests.

• WJ III Understanding Directions is consistently one of the more cognitive complex tests.  And, it is largely a language-based measure of working memory (Gsm-MW).  Remember that as per the Radex model, cognitive complexity deals with the amount of elements/components that are processed.....and is not the same as abstract thinking (Gf-ish stuff).  WJ III Numbers Reversed also shows up close to the center, with WISC-III Digit Span not far behind.  Does this support the popular working memory=Gf/g research hypothesis?

• The Gc tests from both batteries appear similar in placement.

• As would be expected, the WJ III Gf tests (Concept Formation, Numerical Reasoning [which is a combo of Number Series and Number Matrices], and Analysis-Synthesis are within the center "cognitive complexity" circle.

I'm sure there is much more that can be gleaned, but I'll leave that to the readers to discover, debate, and discuss.  I actually think a 3-D MDS model is necessary to capture the characteristics of the measures...but I've run out of time and steam on these analysis.  Maybe at a later date.

A couple caveats I provided the other day are also relevant here--(a) I'm a coauthor of the WJ III (conflict of interest disclosure) and (b) these results have not been peer-reviewed

WISC-III/WJ III cross-battery g+specific cog-ach abilities findings

WISC-III/WJ III cross-battery g+specific abilities research reinforces "just say maybe" program of CHC g+specific abilities research.

I'm just starting the process of drafting a manuscript to summarize the results of the IAP CHC COG-ACH Correlates Meta-Analysis project (click here).  In that on-line EWOK (Evolving Web of Knowledge) I list a McGrew (2007a) study in the reading and math summary tables.  That reference reflects unpublished re-analysis I completed (last fall) with the Phelps et al. (2007) joint (cross-battery) WISC-III/WJ III dataset.  In order to include the findings in the synthesis manuscript, I felt it appropriate to at least informally publish the final models for reference. 

Two important caveats.  I'm a coauthor of the WJ III (conflict of interest disclosure).  The second caveat is outlined in the brief report of the results that I have posted.  When I have time I will update the synthesis EWOK. I will change the reference to McGrew (2008) reflecting posting.

Technorati Tags: psychology, educational psychology, school psychology, neuropsychology, intelligence, IQ testing, IQ, Wechsler, WISC-III, WJ III, general intelligence, g, g+specific abilities

Wechsler Arithmetic test: Measure of Gq or Gf?

What does the Wechsler Arithmetic test measure? Why has it's interpretation been so variable over the decades? Why is it now classified (as per CHC theory) as a mixed measure of Gsm (Short-term Memory - Working memory; MW) and Gf (Fluid Reasoning - Quantitative reasoning; RQ) in the latest Essentials of Cross-Battery Assessment book (Flanagan, Ortiz and Alfonso, 2007)? [Click here if you need more information on CHC theory and the major abilities, definitions, and abbreviations]

While preparing for my recent presentation at the Third National School Psychology Neuropsychology Conference, I consulted the 2nd edition of the Essentials of Cross-Battery book. I noticed on page 310 that, in contrast to prior cross-battery classifications of the Arithmetic test as a primary measure of Gq (Quantitative Knowledge-Math Achievement; A3) and Gf (Fluid Reasoning-Quantitative Reasoning; RQ) [Note - I was involved in these prior classifications as a coauthor of the first cross-battery book (ITDR: McGrew and Flanagan, 1998) and the Wechsler-specific spin-off cross-battery book (Flanagan, McGrew & Ortiz, 2000), it had now changed to Gsm and Gf.

First, a historical note. In the ITDR and Wechsler cross-battery books the primary Gq classification was based on a series of CHC/Gf-Gc designed cross-battery (joint) factor studies. The secondary Gf classification was a logical content analysis based classification, for which no available CHC/Gf-Gc cross-battery factor analysis supported the classification.

A review of page 310 in the 2nd Edition of the Essentials of CB indicates that the Gsm and Gf classifications are based on "factor analysis from the WISC-IV technical and interpretive manual (Psychological Corporation, 2003) and from the results of factor analysis reprted in Keith, Fine, Taub, Reynolds, and Kranzler (2006)." My problem with this change is that these supporting analyses are all within-battery (WISC-IV only) confirmatory factor analysis studies (CFA), and thus do not include the complete range of CHC indicators in the analysis, especially other Gq markers. If you want to see a prior post I made about my disagreement with the Keith et al. Arithmetic classification, click here. Personal communication with one of the XBA Essentials books authors indicates that the Gf classification is also intended to reflect content validity evidence.

Why do I take issue with the use of within-battery CFA to make CHC test classifications? If you want the long story you can read about the strengths and limitations of within- and cross-battery CFA studies in the first two XBA books listed above. If you want a short-course on the issue, you can check out an on-line PPT show I just uploaded that conceptually illustrates the limitations of within-battery CFA studies....and...more importantly, the advantage of CHC-designed cross-battery (joint) CFA studies classifying tests as per CHC theory.

To date I know of 9 different Wechsler/Woodcock-Johnson CHC/Gf-Gc designed cross-battery studies. All 9 studies include other markers of Gq (math achievement tests). In ALL of these analysis the Wechsler Arithmetic test has a large a significant loading on Gq (average loading in the mid .70's)....none on Gf. In one study (Phelps et al. 2005) there is a small secondary loading on Gs. In none of these cross-battery studies does Arithmetic display a significant Gsm factor loading. You can view a summary of these CHC-designed cross-battery findings, as well as the Keith et al. (2006) model upon which the current (and I believe inaccurate) Arithmetic Gf classification is based (due to a loading of .79 on a Gf factor), at the following link.

My two cents. I believe the best available analysis argue for the Wechsler Arithmetic test being interpreted primarily as a measure of Gq (Math Achievment-A3). I believe practitioners should not interpret this test as a primary or strong measure of Gf (Fluid Reasoning - Quantitative Reasoning; RQ). This does not mean that RQ is probably not involved. What the data indicate that if RQ is involved, the amount of variance is trivial and dwarfed by Gq-A3.

Conflict of interest disclosure - I'm a coauthor of the WJ III and thus have a financial interest in a competitor to the Wechsler batteries. I no longer receive any royalties from the two respective cross-battery books I coauthored.

CHC and WJ III dissertations 11-16-07

The other day I posted a list of 19 CHC (Cattell-Horn-Carroll) based dissertations (titles and abstracts) that have been published in the past five years. I've now expanded this list to include WJ III dissertations published during the past five years. The total n is now approximately 30. Title pages and abstracts can be found by clicking here.

I've set an email alert with my university to notify me of new dissertations in these two areas (CHC and WJ III) and will pass along FYI's on new and notable dissertations as they come in.

Technorati Tags: psychology, educational psychology, neuropsychology, school psychology, dissertation, CHC theory, Cattell-Horn-Carroll, intelligence, cross-battery, IQ, IQ test, WJ III

Powered by ScribeFire.

WISC-III/IV scatter/FS IQ study - Hale responds to Lopez

Yesterday's guest blog post by Ruben Lopez generated a lengthy response by Dr. James "Brad" Hale, a regular voice on the NASP and CHC listservs on this particular topic (interpretation of composite and test scores on intelligence tests). To those who are not members of either of these lists, Brad has regularly challenged the statistical arguments/methods behind the research illustrated by the Watkins et al. article Ruben reviewed. In fact, in the same issue of Applied Neuropsychology, Hale et al outline their arguments (click here to view/download). [Note - see previous post on this blog about this entire special issue. There are a number of positions and arguments surrounding this entire topic..Brad's position is not the only position]

In an attempt to provide some balance, I decided to lift Brad's response from the NASP listserv and post it here at IQ's Corner. This will allow others to become aware of Brad's arguments and should help save bandwith on these two lists---as Brad (and others) can then simply refer people to this more permanent post at IQ's Corner for Brad's arguments and thoughts [Note to Brad - maybe this post can save you from having to repeatedly articulate your thoughts and ideas on the listservs...just insert a URL link to this post.]. Also, in the past other voices have been heard in response to Brad's arguments, challenges and claims. I would encourage any of those voices to contact me if they want to provide a counter-response (iap@earthlink.net).

Please note that my posting of Brad's response does not mean I endorse his arguments or claims. They are presented "as is." As blogmaster I would LOVE it if authors of the other articles in the special AN issue would respond with written responses I could post in the form of blog posts.

• I mean no offense to Ruben Lopez, as he seems well-intentioned, but this is clear evidence of why the Watkins et al. results are so problematic, and could be seen as unethical. Why? Because practitioners such as Mr. Lopez read the positions and analyses of these authors, and conclude they "appear reasonable" (quote from Mr. Lopez). Then Mr. Lopez goes on to conclude "So, I won't disregard the full scale solely because of scatter". If the analyses are wrong as I suggest, this is clear evidence that at least one practitioner (and likely others) has been misinformed by this study, and this affects his (their?) practice of psychology.
• This is very sad because in the rebuttal paper of this very same special issue (Hale et al., 2007), statistical analyses are provided that clearly show the errors in the Watkin's paper. Yes, errors. Those are strong words folks, yep. I wouldn't say they were errors unless I was convinced the data shows they are errors. It is all there in black and white folks. It isn't a matter of opinion, rather one of fact, and I challenge any statistician to go on record to say that the Watkins analysis is right, and our rebuttal analyses are wrong. *Please*, statisticians only! These are complex statistical arguments and it would be best if statisticians determine who is correct. Whether you believe in the value of global IQ or not, that is not the issue. The issue has solely to do with statistical analyses.
• Again, please have any statistician who is willing to come forth and show/argue we are wrong, please do so. Please have them provide their full name and other identifying information so they may be contacted at a later date.
• As for why people would continue to value and/or support papers that have significant statistical errors, or even be willing to publish them- that is up to the reader to determine. I won't even speculate because that could be seen as an ad hominem attack. I do find it interesting that someone could read one article in the special issue and say it is good, but not mention the other articles, which show it is not!
• If I am right, and these analyses are statistically inappropriate, and they are used to inform practitioners about clinical practice (as are many of the other papers produced by this academic group - which we also show in the rebuttal paper), there is a serious ethical problem here. I have personally contacted the authors and directly informed them of the statistical errors in other papers, and I have also contacted editors.
• We have also shown why the analyses are wrong in several published works. Yet, the works by this academic group continue to be endorsed by others, even those with the statistical sophistication to know better. It is a sad day in science when people's opinions and values superseed the facts. I guess we have to ask ourselves as a profession a very important question. Are we guided by scientific fact or fancy? It is up for all of us to decide.
• Please do forward this email to anyone you think is willing to reply, including Drs.Watkins and Glutting.

Technorati Tags: psychology, educational psychology, school psychology, neuropsychology, cognitive, intelligence, IQ, IQ test, IQ score, WISC-III, WISC-IV

Powered by ScribeFire.

Test scatter and the WISC-III/IV: Recent study - Guest post by Ruben Lopez

The following is a guest post by Ruben Lopez, school psychologist with the Moreno Valley Unified School District, CA and member of the IQs Corner Virtual Community of Scholars. Rueben reviewed the following article and has provided his comments below.

Watkins, M.W., Glutting, J. J. & Lei, P. (2007). Validity of the Full-Scale IQ when there is significant variability among WISC-III and WISC-IV factor scores. Applied Neuropsychology, 14, 13-20. (click here to view/download)

• For many years now—I think I read it over 25 years ago—school psychology textbooks have told psychologists that significant scatter (“variability”) among component subscales decreased the strength of the composite/full scale score to predict academic achievement . When applied to the Wechsler Intelligence Scale for Children, Fourth Edition (WISC-IV), this would mean that you should not use the Full Scale to predict academic achievement and use a subscale, for instance, the Perceptual Reasoning Index, instead. In their article referenced above, Watkins, Glutting, and Lei “directly addressed” the question of scatter and the WISC by analyzing the scores of a substantial number of students. Although the article contains a technical argument in opposition to the use of multiple regression to address the question—an important issue, even for practitioners, nonetheless--, I’ll just talk about the direct study; yet I note that they used a “moderated multiple regression.”

• The tests studied were the third (III) (1991) and the fourth (IV) (2003) editions of the WISC and the first (1992) and the second (2) (2002) editions of the Wechsler Individual Achievement Test (WIAT). Scores on these tests from three final samples consisting of 412, 412, and 136 students obtained during standardization were analyzed. Two of the samples were described as “linking samples”, samples of students whose scores on the WISC and WIAT were analyzed for standardization to determine the relationship between intelligence and academic achievement. One of the samples consisted of students who had actually been evaluated and found eligible for special education under specific learning disability, emotional disturbance, or mental retardation. The data appear to be substantial.
• The statistical analysis and interpretation of the data appear reasonable. All the samples were divided into two groups, a flat profile group and a variable profile group. The variable profile group consisted of students who had “at least one statistically significant factor score difference.” About the statistical analysis, the authors said, “moderated multiple regression was used to detect any bias in the predictive validity of FSIQ scores [Full Scale scores] between participants with and without significant factor score variability.” Regarding the analysis, the researchers reported, “For all samples, FSIQs were significant predictors of performance on reading and math tests, but neither factor profile group (flat versus variable) nor the interaction between the FSIQ and factor profile group significantly added to the prediction.” From this, Watkins, Glutting, and Lei concluded, “Our results challenge the practice of discounting the global IQ as a predictor of academic achievement when factor scores significantly vary.”
• At least for now, for me, the application is to not disregard the full scale score if there is too much scatter. So, I won’t disregard the full scale solely because of scatter. On the other hand, I note that recent Woodcock-Johnson-III studies provide strong evidence that some component scale scores—reflecting broad CHC factors—are significant predictors of academic achievement that definitely should be considered.

Technorati Tags: psychology, educational psychology, school psychology, neuropsychology, cognitive, intelligence, IQ, IQ test, IQ score, WISC-III, WISC-IV

Powered by ScribeFire.

Validity of global IQ (g) - invitation for commentary

Over on the NASP listserv reference has been made to a special issue of Applied Neuropsychology that includes a number of articles (pro/con) surrounding the issue of the utility of global (g) IQ scores. Reynolds, Hale, Fiorello and others weigh in.

NASP listserv members have been asking for copies of the articles.


According to Cecil Reynolds (based on his NASP response post),

• "the Coalition of Clinical Practitioners in Neuropsychology (CCPN) is open to anyone licensed to practice psychology or retired from practice who at some point was licensed. It holds an annual convention with CE WSs, posters, symposia, and the like. Last year, the convention was held in Orlando...the Journal, Applied Neuropsychology, is a publication of Taylor and Francis, one of the five largest publishing companies in the world. Applied Neuropsychology has also been adopted as the official journal of the Hispanic neuropsychological society as well as the Philadelphia neuropsychological society...a recent mailing of an advertising brochure for Applied Neuropsychology by Taylor and Francis that contained a CCPN membership application when to about 100,000 psychologists around the world.

• The Journal issue that is part of this discussion is also available for sale as a single copy, special issue."

I just downloaded pdf copies of all the articles. The titles of the articles can be viewed by inspecting the table of contents of this special issue (click here)

Now......I'm not going to violate copyright laws to make these available for all to see. However, following the "fair use" doctrine, I could make these articles available for viewing if my blog posts were educational and provided criticisms, reviews, comments, etc., that would be educational in nature.

I personally don't have time to do this...I'm traveling and and am swamped. However, I'm now making an offer. If individuals will contact me via email (iap@earthlink.net) and agree to/promise to review select articles from this issue, and provide quest blog post commentary, I'll post their comments along with a link to each article (for others to view).

Maybe we can get a blog carnival going on this topic. Any takers?

Technorati Tags: psychology, educational psychology, school psychology, neuropsychology, neuroscience, IQ score, IQ, IQ test, intelligence, g, general intelligence, Applied Neuropsychology, NASP

Powered by ScribeFire.

Intelligence & achievement: Chicken or the egg

The following is a post by the blogmaster (Kevin McGrew), who is also a member of IQs Corner Virtual Community of Scholars project.

Historically there has been considerable debate re: the causal relations between psychometric intelligence and academic achievement. Positions have ranged from (a) current achievement causes future intelligence, (b) intelligence and achievement are reciprocal in causality, (c) both constructs are simply difference slices of the same domain and are not distinct constructs, and (d) current intelligence causes future achievement. A real chick-egg causal issue.

An article to appear in the journal Intelligence, using cross-lagged panel analysis methods with the WISC-III and achievement tests, supported the position that intelligence predicts (causes?) achievement and not vice-versa.

Article information is presented below.

• Watkins, M. & Canivez, G. (2006, in press). Psychometric intelligence and achievement: A cross-lagged panel analysis. Intelligence. [click here to view article]

Abstract

• There has been considerable debate regarding the causal precedence of intelligence and academic achievement. Some researchers view intelligence and achievement as identical constructs. Others believe that the relationship between intelligence and achievement is reciprocal. Still others assert that intelligence is causally related to achievement. The present study addressed this debate with a cross-lagged panel analysis of WISC-III and achievement test scores of 289 students assessed for special education eligibility with a test–retest interval of 2.8 years. The optimal IQ–achievement model reflected the causal precedence of IQ on achievement. That is, the paths from IQ scores at time 1 to IQ and achievement scores at time 2 were significant whereas the paths from achievement scores at time 1 to IQ scores at time 2 were not significant. Within the limits imposed by the design and sample, it appears that psychometric IQ is a causal influence on future achievement measures whereas achievement measures do not substantially influence future IQ scores.

Conclusion of authors

• With due consideration of these caveats, the present study provides evidence that psychometric intelligence is predictive of future achievement whereas achievement is not predictive of future psychometric intelligence. This temporal precedence is consistent with the theoretical position of Jensen (2000) that intelligence bears a causal relationship to achievement and not the other way around. press).

Technorati Tags: psychology, educational psychology, cognition, intelligence, IQ, achievement, education, psychometrics, IQ testing, WISC-III, Wechsler, cross-lagged panel analysis, g, general intelligence, school psychology

powered by performancing firefox

What does the WISC-IV measure? CHC-organized CFA study

[Double click to make image larger.....sorry for the poor quality. It is the best I can do at this time]

The latest issue of School Psychology Review has an excellent article on confirmatory factor analysis (CFA) of the WISC-IV norm data, by a research team led by Tim Keith. As I've mentioned before, I always read anything written by Tim Keith, particularly if it involves structural equation modeling (SEM) of cognitive batteries. In the language of the PGA golf tour.....Tim is "dah man" in my book for good analyses (with SEM/CFA methods) in the arena of applied psychometrics.

Currently a pdf copy of this journal issues is not available...so no link to the article is provided. Below is the formal reference citation, journal abstract, select important comments/conclusions by the researchers, one editorial comment by this blogmaster, and an invitation to anyone at Psych Corp to provide a guest blog post response.

• Keith, T., Fine, J., Taub, G., Reynolds, M. & Kranzler, J. (2006). Higher-Order, Multi-Sample, Confirmatory Factor Analysis of the Wechsler Intelligence Scale for Children—Fourth Edition: What Does it Measure? School Psychology Review, 35 (1), 108-127.

Abtstract

• The recently published fourth edition of Wechsler Intelligence Scale for Children represents a considerable departure from previous versions of the scale. The structure of the instrument has changed, and subtests have been added and deleted. The technical manual for the WISC-IV provided evidence supporting this new structure, but questions about consistency of measurement across ages and the nature of the constructs measured by the test remain. This research was designed to determine whether the WISC-IV measures the same constructs across its 11-year age span and to explicate the nature of those constructs. The results suggest that the WISC-IV indeed measures consistent constructs across ages. The scoring structure of the test was not supported in these analyses, however. Comparison of theory-derived alternative models suggests a model more closely aligned with Cattell-Horn-Carroll theory provides a better fit to the WISC-IV standardization data than does the existing WISC-IV structure. In particular, it appears that the WISC-IV measures crytallized ability (Gc), visual processing (Gv), fluid reasoning (Gf), short-term memory (Gsm), and processing speed(Gs); some abilities are well-measured, others are not. We recommend that users regroup the Perceptual Reasoning tests, and Arithmetic, to better reflect the constructs measured by the WISC-IV. Specific suggestions are also provided for interpretation of WISC-IV scores.

Select Author Comments/Conclusions

• The present research did find overall support for the invariance of the constructs across the instrument’s 11 age-differentiated groups. The theoretical and scoring model contained in the WISC-IV technical manual was not supported, however.

• For the WISC-IV, CHC theory provided an alternativeand likely a more valid understanding of construct measurement than did the structure that guided its development and the ensuing scoring structure. Results from these analyses indicate that CHC theory provided a better fit to the standardization data thandid the instrument’s four factor theoretical model. Specifically, the findings from this research indicate that a model separating PRI into measures of fluid reasoning and visual-spatial processing fit the data better than a model with a single PRI factor.
  

• The Word Reasoning subtest, although designed in part to measure fluid reasoning, measures a verbal factor, comprehension-knowledge. The results also indicate that the Picture Completion subtest measures both crystallized intelligence and visual-spatial processing. As hypothesized by the WISC-IV’s developers, the Matrix Reasoning subtest is mainly a measure of fluid reasoning. However, the Symbol Search subtest measures not only processing speed, but also the CHC factor visual-spatial processing.
  

• The results also indicate that the Arithmetic subtest is a complex measure of cognitive abilities; it measures primarily fluid reasoning, but may also measure working (or shortterm) memory and crystallized intelligence. These findings concerning Arithmetic are consistent with previous findings on the WISC-III and continue the debate concerning what this subtest measured in previous versions of the WISC (Keith & Witta, 1997; Kranzler, 1997). They also suggest that the WISC-IV Arithmetic subtest, like previous versions, provides an excellent measure of g.

Comment from blogmaster

• I take issue with the last statement above, namely, that the Arithmetic subtest may be a good measure of Gf and possibly g. As appropriately pointed out the Keith et al (in their limitations section), a significant limitation of this study is the lack of other construct indicators from important CHC domains. When cross-battery CFA studies with prior editions of the Wechsler have included indicators of Gq (primary math achievement tests), Arithmetic always demonstrates strong loadings on Gq...not Gf. For example, a recent joint CFA study of the WISC-III and WJ III (Phelps, McGrew, Knopik & Ford, 2005) found, when evaluated within the context of a relatively "complete" set of CHC indicators, the WISC-III Arithmetic test is primarily a measure of Gq (.60+ loading) , with some possible low Gs variance (.20+ loading). Additional support for this interpretation was provided in Woodcock's (1990 [warning....the Woodcock article is 3.8 MB in size] seminal (and first) CHC/Gf-Gc cross-battery organized empirical publication where the Wechsler Arithmetic test, across 8 different CFA analyses, demonstrated a median Gq loading of .753. Most of the studies that continue to suggest that Arithmetic may be a great indicator of Gf/g are typically within-Wechsler CFA studies, studies that do not include an appropriate set of Gq factor indicators.

Invitation for guest blog post response

• Given that my WJ III authorship status may be viewed as biasing my reading of the Keith et al article, I hereby extend an open invitation to anyone from Psych. Corp. to send me (in Word format) comments and or concerns they have about the original Keith et al study and/or my post. Is anyone from Psych. Corp reading this blog? I would present your comments "as is" as a guest blog post.
  

Technorati tags: SEM. structural equation modeling. neuropsychology. CHC theory. psychology. intelligence. WISC-IV. Wechsler. cognition. education. educational psychology. iq testing. psychological testing. intelligence theory.

WISC-III Arithmetic test: CFA with WJ III

Ok....I'm officially a geek. I'm having lunch at a grant-related mtg in DC and making a post to my blog in response to a thread on the IAPCHC listserv that I received via wifi during the meeting (yes...I can get a bit distracted when asked to sit all day)
.
The following question was posted to the listserv today:

• "The Arithmetic subtest on the WISC seems to be a lost child trying to find a home. On the WISC III it was in the VCI which was primarily a Gc area, then on the WISC IV it falls within the Gsm area. But when I read the WISC Essentials book, Arithmetic is used to help find Gf. Of course, it goes without saying (or does it) that Arit falls under Gq.

• Question....is Arithmetic really a lost child? Does it have a home or does it have multiple homes (a mansion, townhome, and a small studio). "

A recent joint CFA study of the WISC-III and WJ III (Phelps, McGrew, Knopik & Ford, 2005) found, when evaluated within the context of a relatively "complete" set of CHC indicators, the WISC-III Arithmetic test is primarily a measure of Gq (.60+ loading) , with some possible low Gs variance (.20+ loading).

Technorati tags: CHC theory. psychology. intelligence. WISC-III. Wechsler. iq testing. psychological testing. intelligence theory. Woodock Johnson. WJ III. quantitative. Gq.
processing speed. Gs.

WISC-III/WJ III CHC study: Shameless plug

Yes.... a shameless (but, non-commercial) plug. Be sure to read my stated conflict of interest statements on this blog's home page (WJ III coauther)

The following Wechsler/WJ3 cross-battery confirmatory factor analysis article was just published in the most recent edition of School Psychology Quarterly. Contact the journal to subscribe and get your copy now.....before supplies are depleted. Don't be the last one on your block to read it!

Phelps, L., McGrew, K. S., Knopik, S. N., & Ford, L. (2005). The general (g), broad, and narrow CHC stratum characteristics of the WJ III and WISC-III tests: A confirmatory cross-battery investigation. School Psychology Quarterly, 20(1), 66-88.

Abstract: One hundred, forty-eight randomly selected children (grades three-five) were administered the WISC-III, WJ III Tests of Cognitive Abilities, WJ III Tests of Achievement, and seven research tests selected from the WJ III Diagnostic Supplement. The validity of the existing WISC-III and WJ III broad Cattell-Horn-Carroll (CHC) test classifications was investigated via the application of CHC-organized, broad-factor, cross-battery confirmatory factor analyses (CFA). Likewise, the validity of the WISC-III and WJ III narrow CHC ability classifications was investigated via the evaluation of a three-stratum hierarchical (narrow+broad+g) CHC CFA cross-battery model. The Tucker-Lewis Index, the Comparison Fit Index, and the Root Mean Square Error of Approximation evaluated the fit for the resulting models. All statistical values indicated good to excellent fit.

Select study highlights

• This study represented the first-ever three-stratum, CFA CHC-based analyses of a jointWechsler/Woodcock data set.

• At the broad CHC factor level, prior WISC-III/WJ III test classifcations where supported.At the broad factor level, the WISC-III included a greater proportion of Gv tests (Picture Completion, Picture Arrangement, Block Design, Object Assembly) than the WJ III (Spatial Relations and Block Rotation).

• WISC-III Block Design test was the strongest single indicator of Gv (and an integrated narrow spatial relations/visualization [SR/Vz] ability factor). Consistent with prior CHC-based Wechsler studies, the results continued to NOT support the interpretation of Wechsler Block Design as a measure “reasoning” (Gf). WISC-III Object Assembly also appeared to be a strong indicator of Gv, although its interpretation at the narrow ability level was indeterminate.

• Relatively low Gv loadings for WISC-III Picture Completion and Picture Arrangement, plus an additional secondary Gc loading for Picture Completion, reinforced prior research that, in the context of CHC-defined assessments, the use of these two WISC-III tests is discouraged as scores on these two tests may confound ability profile interpretation of Gv composites.

• The two WJ III tests (Visual Matching, Decision Speed) and two WISC-III speeded tests (Coding, Symbol Search) were all identified as strong indicators of broad Gs, but differentiation at the narrow Gs ability level was not supported.

• At variance from prior suggested interpretations of WISC-III Arithmetic as an indicator of quantitative reasoning (under Gf), when included together with select WJ III math achievement measures, WISC-III Arithmetic was found to be a mixed measure of Gq (quantitative knowledge) and Gs.

• At the narrow ability level, support was found for interpreation of the WJ III Numbers Reversed and Auditory Working Memory Tests as measures of working memory (MW), whereas WJ III Memory for Words and Memory for Sentences are best interpreted as measures of memory span (MS). In contrast to apriori hypothesis, the WISC-III was found to be a measure of MS and not MS and MW.

• The WJ III Planning test was found to be less of an indicator of Gv as reported in previous studies, and, instead, loaded primarily on Gf. The possible involvement of working memory, an ability linked to Gf and g, was hypothesized as a possible reason.

• Consistent with the extant Wechsler CHC CFA research, the WJ III was found to provide valid measures of three broad CHC domains (Gf, Glr, Ga) not measued by the WISC-III.