Recently a post was made to the CHC listserv asking for clarification regarding a particular score provided by the WJ III NU norms. I thought the question provided a "teachable moment" regarding certain psychometric principles and methods used in the WJ family of instruments.
I asked David Dailey, the resident statistician and technical consultant to the WJ author team, to write a brief explanation. His well written response is below. Enjoy.
[Interested readers may also be interested in a recently published WJ III NU Assessment Service Bulletin that explains why scores may differ between the WJ III and NU norms. Also, conflict of interest disclosure - I'm a coauthor of the WJ III]
Dear Ms. Jensen (person who posed the question):
Thank you for sharing the interesting profile of reading scores with the CHC mailing list. Kevin McGrew has asked me to write a few sentences about the phenomenon exhibited by these scores-- particularly, as you ask, why this 61-month-old child's Broad Reading score is "so low". I have been heavily involved in the development of the WJ III and WJ III NU norm tables, and I hope I will be able to shed some light on your question.
You reported that your subject earned a particular set of standard scores on the reading tests and clusters. I have augmented those scores with the approximate W, W-difference, and RPI scores that would also have appeared for that subject, in the following table (best viewed in a fixed-width font):
[I apologize for the formatting of the numbers below.....I tried hard to get a nice table format but was unable to get anything to work. I'm still a relatively newbie when it comes to using blogging software]
Test/Cluster, SS, W, W-diff, RPI
Letter-Word ID, 140 , 431, +87, 100
Word Attack, 138, 463, +81, 100
Reading Fluency, 128, 477, +13, 97
Passage Comprehension , 133, 458, +56, 100
Broad Reading, 125, 455, +52, 100
Brief Reading, 149, 444, +71, 100
Basic Reading Skills, 145, 448, +85, 100
You can verify for yourself that the cluster W scores are the arithmetic means of the W scores for the tests making up the cluster. The W-differences and the RPIs show that this subject's reading development is far above that of his/her age peers-- but they also show that the Reading Fluency score is not nearly as exceptional as the remaining scores.
You were concerned that the Broad Reading cluster standard score was so much lower than the other cluster standard scores. Although this subject's scores were exceptionally high for all the clusters (in terms of proficiency relative to age peers), the Broad Reading score is not as exceptional when compared to the other clusters. Its W-difference is lower than the other clusters because it include Reading Fluency, for which the subject outperformed age peers by "only" 13 W points.
The W-difference score in the table above is one of two terms that go into calculating a subject's standard score. The other is a scaling factor (SD - standard deviation) that accounts for how widely or narrowly spread the test scores were in the reference peer group.
In Woodcock-Johnson products, the scaling factor (SD) for subjects performing below the median for the peer reference group is permitted to be, and often is, different from the scaling factor (SD) for subjects performing above the median. So the WJ scoring model has always been able to reflect different amounts of spread among high performers than among low performers.
It turns out that, for young subjects such as yours, the scaling factor (SD) for high performers on the reading clusters is quite large-- meaning it takes a very large W-difference to earn a standard score that is far away from the mean. This is because, for most of these reading skills, the scores for the above-median subjects is very widely spread out. For Broad Reading, a 61-month-old subject must earn 32 W points more than the median to receive a standard score of 115 (one standard deviation above the mean). For the other two reading clusters, the number is somewhat smaller; that, coupled with the higher W-differences your subject earned on those clusters, accounts for the standard-score pattern for your subject.
(You might notice that the scaling factor for Reading Fluency is quite small. This reflects the fact that there is very little variation among above-median subjects at this age on this task.)
So the bottom line here is that the Broad Reading score suffers a "double whammy"-- a comparatively lower W-difference (due to the lower Reading Fluency) and a larger amount of above-median variation in the norming-sample scores. And this subject earned much higher standard scores on the other clusters because their relative performance (in terms of raw ability) on those clusters was much higher, plus the variation within the norming sample was smaller.
Thank you again for your question. I hope I have been able to help you understand more about how these scores work.
Technorati Tags: psychology, educational psychology, school psychology, neuropsychology, psychometrics, WJ III, WJ III NU, reading, CHC listser
Powered by ScribeFire.
No comments:
Post a Comment