How to evaluate the unusualness (base rate) of WJ IV cluster or test score differences: It is a pleasure to use the correct measure - A SlideShare presentation

The WJ IV provides two primary methods for comparing tests or cluster scores.  One is based on a predictive model (the variation and comparison procedures) and the other allows comparisons of SEM confidence bands, which takes into account each measures reliability.  A third method for comparing scores, one that takes into account the correlation between compared measures (ability cohesion model) is not provided, but is frequently used by assessment professionals.  The three types of score comparison methods are described and new information, via a "rule of thumb" summary slide and nomograph, are provided to allow WJ IV users to evaluate scores via all three methods.

A PDF copy of the key WJ IV base rate rule-of-thumb slide can be found here.

How to evaulate the unusualness (base rate) of WJ IV cluster or test score differences: It is a pleasure to use the correct measure from Kevin McGrew

More on The Composite Score Extremity Effect: Must see animated visualizations at Joel Schneider's blog

You must visit Joel Schneider's blog to see his awesome animations that help explain the composite score extremity effect.  They are worth viewing, even if one does not understand them :)  Click on wordpress.com and enjoy.

Photo: wordpress.com The Composite Score Extremity Effect wordpress.com · When a person scores exactly 2 standard deviations below the mean on several tests, it is intuitive that the composite score that summarizes these scores s... Read on Flipboard Read on wordpress.com

This email was sent to you by a Flipboard user via Flipboard's "Send to Friends" feature. Need help? Email support@flipboard.com. Privacy Policy    Unsubscribe ©2016 Flipboard, Inc. Made with love at 214 Homer Ave, Palo Alto, CA 94301

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Why are full scale IQ scores often lower (or higher) than the subscores? Dr. Joel Schneider on the "composite score extremity effect"

Bingo.  There is finally an excellent, relatively brief, explanation of the phenomena of why full scale IQ scores often diverge markedly from the arithmetic average of the component index or subtest scores.

This composite score extremity effect (Schneider, 2016)  has been well known by users of the WJ batteries.  Why....because the WJ has placed the global IQ composite and the individual tests on the same scale (M=100; SD=15).  In contrast, most other cognitive ability batteries (e.g., Wechslers) have the individual test scores on a different scale (M=10; SD=3).  The use of different scales has hidden this statistical score effect from users.  It has always been present.  I have written about this many times.  One can revisit my latest post on this issue here.

Now that the WISC-V measures a broader array of cognitive abilities (e.g., 5 index scores), users have been asking the same "why does the total IQ score not equal the average of the index scores?"  Why?  Because the five index scores are on the same scale as the full scale IQ score...and thus this composite score extremity effect is not hidden.  A recent thread on the NASP Community Exchange provides examples of psychologists wondering about this funky test score issue (click here to read).

As per usual, Dr. Schneider has provided intuitive explanations of this score effect, and for those who want more, extremely well written technical explanations.

The WJ IV ASB 7 can be downloaded by clicking here.  Although written in the context of the WJ IV, this ASB is relevant to all intelligence test batteries that provide a global IQ score that is the sum of part scores.

Kudos to Dr. Schneider.

Click on image to enlarge

WISC-V expanded index scores: Verbal Expanded Crystallized (Gc) and Expanded Fluid (Gf) index scores and tables

Click on image to enlarge for clearer reading

Somehow I seemed to have missed this development on my research literature monitoring radar.  More comments and a link to the technical report can be found at Joel Schneider's excellent blog.

Why does the WJ IV GIA score often appear lower (or higher) than the average of the component tests? An explanation redux

With the publication of the new WJ IV, an old score issue has again resurfaced.  Examiners have observed that often an individual's overall GIA score is lower than the arithmetic average of the scores for the component tests. This occurs when an individual consistently scores below average on the component tests.   For individuals who score significantly above average on most tests, the GIA is higher than the arithmetic average of the components (the opposite effect).

 

This is not a new phenomenon and is NOT unique to the WJ battery.  I've written about this previously with Joel Schneider.  A link to a special report can be found here.

Joel Schneider has a great explanation (with videos) of this phenomenon at his awesome blog.

Finally, I first wrote about this in 1994 and have re-posted this material (from my first WJ book) for download (PDF file).

In these explanations one will see that the “total does not equal the sum of the parts” phenomenon also occurs on the Wechslers, but it is HIDDEN from view via the fact that the Wechslers use standard scores (mean=10 plus/minus 3) for subtests.  In my 1994 explanation, I present a fictitious case where a child obtains 4's on all WISC-R subtests (-2 SD).  Since a standard score on a subtest of 4 is -2 SD, these scores would be equal to a standard score (mean=100; SD=15) of 70.  The child would have70's for all subtests.  The arithmetic average of all subtests would be 70.  So.....is the WISC-R full scale IQ approximately 70?  No.  It is 59, or 11 points lower!

Bottom line.  This phenomenon has been around for years and is present on all IQ tests.  It is more obvious on the WJ batteries were all subtests and cluster scores are on a common 100/15 scale. This is nothing new.  If  you have been using other batteries (e.g.Wechslers) you simply have not had the opportunity to observe it.

 

I am currently working with a colleague on a special Assessment Service Bulletin to explain this score issue.  I will notify readers of its availability as soon as it is completed.

How to estimate best IQ score if someone has taken multiple IQ tests: The psychometric magic of Dr. Joel Schneider

Dr. Joel Schneider has posted an excellent explanation on how to estimate a person's "true IQ score" when a person has taken multiple IQ tests at different times. Probably the most important take-away message is one should never calculated the simple arithmetic average. The median would be more appropriate, but Joel provides and even more psychometrically sound method and an Excel spreadhsheet for implementing his excellent logic and methods.



- Posted using BlogPress from Kevin McGrew's iPad

Why IQ composite scores often are higher or lower than the subtest scores: Awesome video explanation

This past week Dr. Joel Schneider and I released a paper called " 'Just say no' to averaging IQ subtest scores." The report generated considerable discussion on a number of professional listservs.

One small portion of the paper explained why composite/cluster scores from IQ tests often are higher (or lower) than the arithmetic mean of the tests that comprise the composite. This observation often baffles test users.

I would urge those who have ponder this question to read that section of the report. And THEN, be prepared to be blown away by an instructional video Joel posted at his blog where he leads you through a visual-graphic explanation of the phenomena. Don't be scared by the geometry or some of the terms. Just sit back and relax and now recognize, even if all the technical stuff is not your cup-of-tea, that there is an explanation for this score phenomena. And when colleagues ask, just refer them to Joel's blog.

It is brilliant and worth a view, even if you are not a quantitatively oriented thinker.

Below is a screen capture of the start [double click on icon to enlarge]



- iPost using BlogPress from my Kevin McGrew's iPad

Cognitive ability domain cohesion-why composite scores comprised of significantly different subtest scores are still valid

Some excellent discussion has been occurring on the NASP and CHC listservs in response to the "Just say no to averaging IQ subtest scores" blog post and report.

An issue/question that has surfaced (not for the first time) is why markedly discrepant subtest scores that form a composite can still be considered valid indicators of the construct domain. Often clinicians believe that if there is a significant and large discrepancy between tests within a composite, the total score should be considered invalid.

The issue is complex and was touched on briefly in our report and in the NASP and CHC threads by Joel Schneider. Here I mention just ONE concept for consideration.

Below is a 2-D MDS analysis of the WJ III Cog/Ach tests for subjects aged 6-18 in the norm sample. MDS also finds structure as does factor analysis. This 2D model is based on the analysis of the tests correlation matrix. What I think is a major value of MDS, and other spatial statistics, is that one can "see" the numerical relations between tests. Although the metrics are not identical, the visual-spatial map of the WJ III tests does, more-or-less, mirror the intercorrelations between tests. [Double click on image to enlarge]




So....take a look at the Gc, Grw, or Gq tests in this MDS map. All of these tests cluster closely together. Inspection of their intercorrelations finds high correlations among all measures. Conversely, look at the large amount of spatial territory covered by the WJ III Gv tests. Also look at the Ga tests (note that a red line is not connecting Auditory Attention, AA, down in the right-hand quadrant with the other Ga tests). Furthermore, even though most of the Gsm tests are relatively cohesive or tight, Memory for Sentences is further away from the other Gsm tests.

IMHO, these visual-spatial maps, which mirror intercorrelations, tell us than in humans, not all cognitive/ach domains include narrow abilities that are highly interrcorrrelated. I call it "ability domain cohesion." Clearly the different Gv abilities measured by the WJ III Gv tests indicate that the Gv domain is less cohesive (less tight) than the Gc or Grw domain. This does not suggest the tests are flawed..instead it tells us about the varying degrees of cohesiveness present in different ability domains.

Thus, for ability domains that are very very broad (in terms of domain cohesion--e.g., Gv and Ga in this MDS figure), wildly different test scores (e.g., between WJ III Spatial Relations, SR, and Picture Recognition, PR) may be valid and simply reflect that inherent lower cohesiveness (tightness) of these ability domains in human intelligence. Thus, if a person is significantly different in his/her respective Gv SR or PR scores, and these scores are providing valid indications of their relative standing on these measured abilities, then combining them together is appropriate and reflects a valid estimate of the Gv domain....which by nature is broad...and people will often display significant within-domain variability.

Bottom line. Composite scores produced by subtests that are markedly different are likely valid estimates of domains...it is just the nature of human intelligence that some of these domains are more tight or cohesive than others.
- iPost using BlogPress from my Kevin McGrew's iPad