Tuesday, October 31, 2006

Random tidbits from the mind and brain blogpshere 10-31-06

  • Interesting article at BPS Research Digest on boosting Gv (visual skills) in lower SES populatons to boost their confidence in the world of work, and increase job performance.
  • A tech tidbit from the Download Squad. I'm always looking for ways to become a paper-less worker. Today they have highlighted a new light-weight e-book reader from Adobe. One of these days I'm going to try doing some e-book reading on a regular basis.
  • The Eide Neurolearning blog has posted a supposed fMRI showing the impact of chocholate on the brain...just in time for Halloween.


Technorati Tags: , , , , , , , , , , , , , ,

powered by performancing firefox

Friday, October 27, 2006

RTI and cognitive assessment--Guest post by John Garruto

The following is a guest post by John Garruto, school psychologist with the Oswego School District and member of the IQs Corner Virtual Community of Scholars. John reviewed the following article and has provided his comments below. [Blog dictator note - John's review is presented "as is" with only a few minor copy edits and the insertion of some URL links]

Hale, J.B., Kaufman, A., Naglieri, J.A. & Kavale, K.A. (2006). Implementation Of IDEA: Integrating Response To Intervention And Cognitive Assessment Methods. Psychology in the Schools, 43(7), 753-770. (click here to view)

This article (and the entire journal series in this special issue) has articulated much of what I have been saying and thinking for a long time. Hale and colleagues open up by discussing the RTI (response-to-intervention) and cognitive assessment “factions”. Although I had nothing to do with this article, I chuckled at the similarity to a PowerPoint I did for graduate study in July of 2005 (click here). I joked about these factions as having a paradigm that was analogous to “Star Wars”. I likened the idea of school psychologists who espoused both RTI and cognitive assessment as necessary requirements for the identification of SLD (Specific Learning Disability) as comprising “a rebel alliance”…primarily because it seemed we were advocating such a balanced approach. Clearly this Psychology in the Schools special issue suggests there is an increasing number of professionals who advocate this approach.

Before beginning with a general summary and sharing my overall impressions, it is important to acknowledge the obvious conflict of interest of most dissenters (in the special issue); both Kaufman (KABC-II) and Naglieri (CAS) are intelligence test authors. That said, it is important to note that two of the other authors are not test authors. In fact, Kavale (a.k.a., the intervention effect size guru) is frequently cited by many RTI-only proponents. Therefore, it is suggested that the scope of this article ending at a conflict of interest is very unlikely.

  • The Hale et al. article begins with the acknowledgment that there seem to be two factions in school psychology assessment circles--those who believe in response-to-intervention as the way to determine eligibility for SLD, and those who espouse the need for cognitive assessment. The Hale et al. article does not diminish the importance of RTI or the problem-solving model. In fact, it supports many of the changes noted in the regulations (e.g., the importance of looking at RTI as a part of the process for determining eligibility for learning disabilities.) It places emphasis on the use of empirically-based instruction and interventions. It also highlights the significance of formative assessment and ongoing progress monitoring. Such practices will illustrate the effect of interventions.
  • After supporting the importance of RTI, the authors contend that at Tier-III, a responsible individualized assessment (including cognitive assessment) needs to occur. Clearly, jumping to conclusions about a neurologically-based deficit based only on failure to RTI would lead to a significant number of false positives (Type I errors). The authors do an exemplary job of identifying the importance of cognitive processing deficits related to SLD in the problem-solving literature. This approach does not embrace the much maligned ability-achievement discrepancy LD identification procedure, but instead endorses examining those that processes are leading (if any) to the negative outcomes. The authors conclude with a case study that describes a child who seemed to have one problem on the surface, but via cognitive assessment was discovered to have an underlying latent problem (i.e., was not observably manifest.) The authors contended that this discovery, vis-à-vis appropriately designed cognitive assessment methods, facilitated the problem-solving model by allowing the team to implement new interventions. The beauty of this example is that the focus was not on eligibility as the end result, but instead, using individualized assessment to help piece the puzzle together.
  • I’ve spoken quite a bit about the authors and a possible conflict of interest. One thing I do want to mention is that I continue to be a school-based practitioner. This framework is one I have been endorsing (as a practitioner) for a long time (my presentation noted above has been online many months before this article went to press.) I’ve had many spirited debates with teachers, arguing that the spirit of formative assessment and research-based interventions has a very positive research history and we are remiss not to use these methods first. However, for those kids who are not responding, I can often complete a solid individualized assessment that provides logical reasons as to why they are not responding, and continue to provide interventions that are related to dynamics and skills that are not readily manifest. There is absolutely no doubt in my mind that combining both approaches will allow us to look beyond “eligibility” to determining what a child needs.
  • Another of my thoughts is that much of the criticism of cognitive assessment not leading to intervention has been the lack of research for establishing ATIs (aptitude-treatment-interactions). However, establishing individualized interventions based on the needs of the child (that might not have a huge history of published research) does not mean we throw it out. Many RTI-only proponents argue that we might was well go right to special education and simply intensify the research-based interventions that could be done with a special education paradigm. I argue that doing flash cards to aid sight-reading might have an empirical support base, but doing flash cards all day long (one-on-one) with a blind student isn’t going to do a thing. However, designing an intervention around the varied needs and interests of the child could (and has) lead to positive results.
  • Finally, my other concern with the RTI-only paradigm is it seems “stuck” on reading…and only on three out of the big five components of the National Reading Panel (Phonemic Awareness, Phonics, and Fluency). There is little research on using CBM for math reasoning or written expression (beyond spelling and perhaps writing fluency.) I believe the most recent edition of School Psychology Review, 35(3), which focused on CBM for reading, writing, and math might have provided practice-based school psychologists with the research we need. Quite the contrary, most all of the articles dealt with math calculations and fluency, as well as on spelling, mechanics, and writing fluency. Clearly CBM/RTI research on higher-level reasoning processes, vocabulary, induction, deduction, inferential reasoning, and writing organization, were lacking from this issue. Until RTI-only advocates start providing research and guidance in these areas, we would be remiss to discard relevant assessment techniques that provide insights into these important skills and abilities.
Technorati Tags: , , , , , , , , , , , , , , , ,

powered by performancing firefox

Thursday, October 26, 2006

Recent literature of interest 10-26-06

This weeks recent literature of interest can be found by clicking here.

I'm now experimenting with listing the references by journal title...I find it easier when looking up articles via the university library.

Technorati tags:

IQs Corner suggeston box is open

To date I, as the IQs Corner blog dictator, have been posting information that I think is exciting, informative, etc. There have been a number of guest bloggers (click here for example) who have provided additional material. I'd like to make this blog useful to a wide variety of professionals and scholars who have an interest intelligence theories and testing.

If anyone has ideas for material I should be paying attention to (articles, researchers, web pages, other blogs, etc.), ideas for posts, self-nominations for guest posts by yourself, etc., please drop me a note in the blog "comment" feature or, email me at iap@earthlink.net.

Also...if you haven't checked out my new sister blog (Tick Tock Talk: The IQ Brain Clock), maybe this is the "time." I've been posting most of the neuroscience and brain-based research I find interesting in this new little personal sand box.

Wednesday, October 25, 2006

More on mental time keeping and working memory link

Readers of IQs Corner may find a few posts I made today over on my sister blog (Tick Tock Talk: The IQ Brain Clock) of interest. One presents the predominant model of mental interval time keeping (the pacemaker accumulator model) while the other reports some very exciting linkages between the brain regions associated with cogntively controlled mental interval timing and working memory---namely, the dorsolateral prefrontal cortex.


Technorati Tags: , , , , , , , , , , , , , , , , , , ,

powered by performancing firefox

Keep your brain young: Eat your vegetables

I guess I need to jump on the blogsphere bandwagon. Every other brain, intelligence, and/or science-related blog site has posted FYI information about the recent study that suggests that eating vegetables helps to slow brain decline as you age. Click here for one representative news story. Just call me a lemming.

Technorati Tags: , , , , , , , , , , , , ,

powered by performancing firefox

Random tidbits from the brain and mind blogsphere 10-24-06


  • Science Daily has interesting news reports (from the world of research) that suggests the viral infections of the CNS can lead to memory deficits (click here), new DTI methods discover possible differences in brains of individuals with autism (click here) and the identification of a possible "memory gene" (click here)

Technorati Tags: , , , , , , , , , , , , , ,

powered by performancing firefox

Mechanical brain sculpture


Thanks to Neurofuture for the FYI post about some very interesting looking biomechanical sculptures are by artist Lewis Tardy.

Technorati Tags: , , , , , , , , , , , , ,

powered by performancing firefox

Sunday, October 22, 2006

Off task - Free hugs

This is WAY off task for this blog.

I ran across one of the more popular videos being viewed on YouTube and it hit a personal chord (Free Hugs Campaign). Don't ask me why. I think the music (by Sick Puppies) resonated to something inside me. Also, given all the partisian political bickering (here in the US) as we approach the mid-term elections and the stuff going on in the world (Iraq, N. Korea, school violence, etc.), sometimes one person can make a difference.


Technorati Tags: , , , , , , , , , , , , , , ,

powered by performancing firefox

Recent literature of interest 10-23-06

This weeks recent literature of interest can be found by clicking here.

I'm now experimenting with listing the references by journal title...I find it easier when looking up articles via the university library.

Technorati tags:

Friday, October 20, 2006

Neuroscienjce of learning (brain-based learning): Posts moving to The IQ Brain Clock

Yesterday's posts (click here, here) dealing with facts and fictions regarding brain-based learning in education, plus the response of SharpBrains, set me off digging through my electronic library of professional articles...in search of articles on the use of neuroscience in learning in education. I found a number of good overview articles that I will comment and post shortly.

However, it dawned on me that this line of research is probably more consistent with the focus of my new blog (Tick Tock Talk: The IQ Brain Clock). Thus, this note is to inform regular IQs Corner readers that I'm going to start shifting posts focused on the neuroscience of learning over to The IQ Brain Clock Blog. I will likely also broaden the definition and purpose of that blog.

Technorati Tags: , , , , , , , , , , , , ,

powered by performancing firefox

Thursday, October 19, 2006

On Intelligence: Hawkins HTM computational theory

I LOVE blogging.

Two days ago I made the FYI post "Brain boss (prefrontal cortext) acts in step-wsie manner?" Shortly thereafter, someone posted a "comment" suggesting a link between the sequential step-wise hypothesized functioning of the prefrontal cortex and Hawkin's (2005) Hierarchical Temporal Memory (HTM) computational theory of intellectual functioning (as outlined in his book "On Intelligence"). I had never heard of this book or theory.

What I find intriguing is the fact that Hawkins is no minor player on the modern technology stage. He is well known for a number of activities--namely, he has founded founded three companies: Palm Computing (who has not heard of Palm Pilots?), Handspring, and Numenta, and the non-profit Redwood Neuroscience Institute, a scientific research institute focused on understanding how the human neocortex works.

The typical reader of IQs Corner may be wondering why someone with his background is now dabbling in understanding human intelligence. I wondered this myself. The answer lies in the Prologue to On Intelligence. A portion is reproduced below.

Maybe some readers of IQs Corner will explore this area in greater detail
  • You may be wondering why a computer designer is writing a book about brains. Or put another way, if I love brains why didn't I make a career in brain science or in artificial intelligence? The answer is I tried to, several times, but I refused to study the problem of intelligence as others have before me. I believe the best way to solve this problem is to use the detailed biology of the brain as a constraint and as a guide, yet think about intelligence as a computational problem—a position somewhere between biology and computer science. Many biologists tend to reject or ignore the idea of thinking of the brain in computational terms, and computer scientists often don't believe they have anything to learn from biology. Also, the world of science is less accepting of risk than the world of business. In technology businesses, a person who pursues a new idea with a reasoned approach can enhance his or her career regardless of whether that particular idea turns out to be successful. Many successful entrepreneurs achieved success only after earlier failures. But in academia, a couple of years spent pursuing a new idea that does not work out can permanently ruin a young career. So I pursued the two passions in my life simultaneously, believing that success in industry would help me achieve success in understanding the brain. I needed the financial resources to pursue the science I wanted, and I needed to learn how to affect change in the world, how to sell new ideas, all of which I hoped to get from working in Silicon Valley.
  • In August 2002 I started a research center, the Redwood Neuroscience Institute (RNI), dedicated to brain theory. There are many neuroscience centers in the world, but no others are dedicated to finding an overall theoretical understanding of the neocortex—the part of the human brain responsible for intelligence. That is all we study at RNI. In many ways, RNI is like a start-up company. We are pursuing a dream that some people think is unattainable, but we are lucky to have a great group of people, and our efforts are starting to bear fruit.
  • The agenda for this book is ambitious. It describes a comprehensive theory of how the brain works. It describes what intelligence is and how your brain creates it


Technorati Tags: , , , , , , , , , , , , , , , , , ,

powered by performancing firefox

More on brain-based learning article: Sharp Brains response

Sharp Brains provides a nice counter-reponse to the doom-and-gloom "Brain-based" Learning: Fiction more than Fact" article that I directed folks to earlier today. You gotta love the conversations on the blogsphere.

Technorati Tags: , , , , , , , , , , , , ,

powered by performancing firefox

Brain-based learning: Facts and fictions

There is a very interesting article in the Fall 2006 issue of the American Educator where a cognitive psychologists attempts to debunk some of the fact/fiction regarding common "brain-based" educational programs. As we all know, education often runs after fads, many of which the scientists have left a long time ago.

Aside from putting the practical realities of the recent strides in neuroscience research into proper "where the rubber meets the road" practical perspective, the author does a good job of discussing three popular myths:

1. Schools are designed for lef-brain students; 2. Schools are designed to suite girl's brains; 3. Classical music is especially important in the develoment of young brains.


This might be a good article to disseminate within educational circles.

Technorati Tags: , , , , , , , , , , , , ,

powered by performancing firefox

Free open access journals

If you are looking for free access to some journals, check out DOAJ (Directory of Open Access Journals). As lifted from the home page:
  • Welcome to the Directory of Open Access Journals. This service covers free, full text, quality controlled scientific and scholarly journals. We aim to cover all subjects and languages. There are now 2423 in the directory. Currrently 709 journals are searchable at article level. As of today 118176 journals are included in the DOAJ service.


Technorati Tags: , , , , , ,

powered by performancing firefox

Tuesday, October 17, 2006

The IQ Brain Clock blog - recent posts of interest

Regular readers of IQs Corner may want to routinely visit my new sister blog (Tick Tock Talk: The IQ Brain Clock). The last two posts dealt with the role of mental time perception in ADHD and information regarding Aurhtur Jensen's new book "Clocking the Mind".

Technorati Tags: , , , , , , , , , , , , , , , , , , , , ,

powered by performancing firefox

Brain boss (prefrontal cortex) acts in step-wise manner ?

Without a doubt, the prefrontal cortex has been a very hot area of research the past few years. As most blog readers know by now, the prefrontal cortex is often considerd the "brain boss" and helps with the coordination of complex cognitive processing via executive functioning. Now there is research that suggests that the PFC may become activated in a step-wise sequential/hierarchical manner as a function of the complexity of problems presented. See news article summary from Scientific American blog.


Technorati Tags: , , , , , , , , , , , , , , , , ,

powered by performancing firefox

Train your brain: Brain gymnastics industry grows

Thanks to Mind Hacks for the Washington Post article dealing with the growing "brain gymnastics" movement. One of these ventures (Brain Age) has actually achieved some degree of notereity as it even listed in Wikipedia.

Technorati Tags: , , , , , , , , , , , , , , , , ,

powered by performancing firefox

Recent literature of interest 10-16-06

This weeks recent literature of interest can be found by clicking here.

I'm now experimenting with listing the references by journal title...I find it easier when looking up articles via the university library.

Technorati tags:

Monday, October 16, 2006

New "sister" blog: Tick Tock Talk-The IQ Brain Clock

Yes.....I've offically gone over the edge. I've now started a new "sister" blog to IQs Corner. The blog is called "Tick Tock Talk: The IQ Brain Clock.' I won't say more...go and visit this neophyte endeavor. It is an experiment given birth from my intellectual curiosity.

So many ideas and articles to read...so little time....sigh

Technorati Tags: , , , , , , , , , , , , , , , ,

Random tidbits from today's mind and brain blogsphere 10-16-06


  • Affective Teaching has some good stuff regarding peoples perceptions/beliefs about intelligence (entity vs incremental theories of intelligence/ability). The Mouse Trap also picked up on this post and has added some additional thoughts and information.
  • Chris Chatman (Developing Intelligence) continues his informative and thought-provoking blog thread regarding the acquisition of language. His most recent post deals with the issue of domain (language)-specific vs domain-general aquisition mechanisms.
  • Thanks to Mind Hacks for the tip regarding the thoughts of several cognitive scientists (from recent Future of Science conference) regarding the future of science.
  • Thanks to Omni Brain for the tip regarding the availability of an on-line visual field test, a typical part of any comprehensive neuropsychological or visual exam.

Technorati Tags: , , , , , , , , , , ,

powered by performancing firefox

Friday, October 13, 2006

Spatial intelligence (Gv) and learning center-NSF funded

The National Science Foundation recently awareded a large grant for a Spatial Intelligence and Learning Center (SILC) that brings together scientists and educators from Temple University, Northwestern University, the University of Chicago, the University of Pennsylvania, and the Chicago Public Schools (CPS) to pursue the overarching goals of:
  • Understanding spatial learning
  • Using this knowledge to develop programs and technologies that will transform educational practice, helping learners to develop the skills required to compete in a global economy.
Click here to view Temple press release regarding this grant/center

Technorati Tags: , , , , , , , , , , , , , , , ,

powered by performancing firefox

Low IQ (intelligence) = earlier death in poor countries?

Interesting (and likely controversial) post at the British Psychologcial Study re: a researcher who suggests that low IQ (intelligence) may be related to shorter life expectancy in poor countries. Links to other related studies are also included.


Technorati Tags: , , , , , , , , , , , , , ,

powered by performancing firefox

On the road again




I'm on the road to Austin, TX today (10-13-06) and will return late 10-15-06. Blog posts may be minimal. I shall return.

Wednesday, October 11, 2006

Weight gain = brain drain ?

Interesting news article regarding research linking weight and cognitive ability. Thanks to Live Science

Technorati Tags: , , , , , , , , , , ,

powered by performancing firefox

Collective intelligence?

Thanks to Positive Technology Journal for the FYI regarding MIT's "collective intelligence" project, which will investigate if people connected via computers can act more intelligently than individuals or other groups.

Technorati Tags: , , , , , , , ,

powered by performancing firefox

Excercise your frontal and parietal lobes

Interesting (and fun) post at SharpBrains re: an activity to exercise your frontal and parietal lobes...supposedly to improve attentionial processes


Technorati Tags: , , , , , , , , , , , ,

powered by performancing firefox

Tuesday, October 10, 2006

Mental time keeping: Temporal g (general intelligence)?

For those who know me well, this post may come as a bit of a shock.

I've never been a big fan of the construct of g (general intelligence; click here and here). Why?

During the past 20 years I had the fortunate privilege, via my involvement in the development of the last two editions of the Woodcock-Johnson (WJ-R; WJ III), to have observed and participated in lively debates (in the same room) between John Horn (there is no g) and Jack Carroll (g exists) . I concluded that if these two giants in the field of intelligence (as well as many other scholars since the days of Spearman--early 1900's) could not agree on the existence of this construct, who am I (an applied psychometrician and applied educational psychologist) to render any sound judgement re: this arguement. More importantly, as an applied test developer and ex-school pscyhologist, I found the notion of g (and full scale IQ scores) to be of little practical use....to me the important information was at the level of broad and narrow CHC abilities. So..I typically have opted out of the g vs no-g debates due to practical beliefs.

However, over the past few years a number of initially unrelated professional activities/events have started to merge, and this convergence has required me to give the g vs no-g debate more attention. In particular, I have recently become involved in the external evaluation [conflict of interest disclosure - I was paid for my evaluation work by the company] of an intervention program (Interactive Metronome) which is based on improving the synchronized metronome tapping (SMT) of individuals. As I've stated elsewhere, I was initially EXTREMELY skeptical about SMT, particularly as it related to possible positive effects on academic functioning. Long story short.....I, along with much of school psychology/special education, was seriously "burned" during the early days of special education vis-a-vis various non-cognitive interventions (psychomotor training; psycholinguistic training) for kids with special needs (they were all found wanting).

I have previously made a number of blog posts (click here, here, and here) regarding SMT/IM and related mental/time-tracking research. Now, in an article "in press" in Intelligence (see below), I see a possible connection between SMT/IM and the authors hypotheses that there may be a "temporal g" (based on an underlying internal master mental time keeper cognitive mechanism.) Although based on a small sample (needs replication and extension), in this investigation the reseachers found that a temporal g factor demonstrated a higher correlation (r = .56) with psychometric g than did the traditional/default/favorite approach to measuring the neural essence of g (Jensen's neural effieciency/oscillation theory), namely, Hick's reaction time paradigm (r = .-34)

What I find interesting is that SMT/IM effects, which cut across diverse domains of human performance, fit nicely with the notion of an underlying domain-general mental time keeping mechanism that may be strongly related to g (in the form of a temporal g construct). I could write much more, but instead, I'll provide a link to some PPT slides (in a pdf file; click here) that I've been crafting to explain all of this. Please recognize that these thoughts (as captured in this post and in the PPT slides) are in their formative stage of development.

Quoting from an old song from the 70's by Buffalo Springfield, "Something is happening here....what it is aint' exactly clear..." My interest is piqued and I will continue to actively reflect on, and, study these developments.

Below is the reference and abstract, as well as a link to the article.

Rammsayer, T. & Brandler, S (2006, in press). Performance on temporal information processing as an index of general intelligence. Intelligence (click here to view)


Abstract
  • The relation between general intelligence (psychometric g) and temporal resolution capacity of the central nervous system was examined by assessing performance on eight different temporal tasks in a sample of 100 participants. Correlational and principal component analyses suggested a unitary timing mechanism, referred to as temporal g. Performance on single temporal tasks and individual factor scores on temporal g were substantially correlated with factor scores on psychometric g. Additional stepwise multiple regression analysis and commonality analysis showed that performance on temporal information processing provides a more valid predictor of psychometric g than traditional reaction time measures derived from the Hick paradigm. Findings suggest that temporal resolution capacity of the brain as assessed with psychophysical temporal tasks reflects aspects of neural efficiency associated with general intelligence.

Technorati Tags: , , , , , , , , , , , , , , , , , , , ,

powered by performancing firefox

Monday, October 09, 2006

Are contemporary IQ tests being overfactored?



Are test developers (that includes me, the blog dictator) increasingly overfactoring intelligence test batteries?

According to an article by Frazier and Youngstrom "in press" in the prestigious journal Intelligence, contemporary test developers (and their publishing companies) "are not adequately measuring the number of factors they are purported to measure." Below is the reference citation and abstract (with a link to the article).

According the Frazier and Youngstrom, the purpose of their investigation was: "The present paper proposes that several forces have influenced this trend including: increasingly complex theories of intelligence (Carroll, 1993; Vernon, 1950), commercial test publishers' desire to provide assessment instruments with greater interpretive value to clinicians, publishers' desire to include minor ability factors that may only be of interest to researchers, and heavy reliance on liberal statistical criteria for determining the number of factors measured by a test. The latter hypothesis is evaluated empirically in the present study by comparing several statistical criteria for determining the number of factors present in current and historically relevant cognitive ability batteries."

As a coauthor of one of the batteries (WJ III) analyzed in this study and, in particular, the battery that measures the largest number of factors in their investigation, I feel compelled to respond to portions of this manuscript. Thus, readers should read the original article and then review my comments, fully recognizing that I have a commercial conflict of interest.

Before I present the major conclusions of the article and provide select responses, I'd like to first state that, in many respects, I think this is a well done article. Regardless of the extent to which I agree/disagree with Frazier and Youngstrom, the introduction is worth reading for at least two reasons.

  • The article provides a nice (brief) overview of development of psychometric intelligence theories from Spearman through early hierarchical theories (Vernon) to contemporary Carroll and Cattell-Horn Gf-Gc (the later two now often referred to as Cattell-Horn-Carroll [CHC] theory).
  • In addition, for individuals looking for a brief description and synopsis of the major statistical approaches to determining the number of factors to retain in factor analytic studies, pages 3-6 are recommended.

Frazier, T. & Youngstrom, E. (2006, in press). Historical increase in the number of factors measured by commercial tests of cognitive ability: Are we overfactoring? Intelligence.

Abstract

  • A historical increase in the number of factors purportedly measured by commercial tests of cognitive ability may result from four distinct pressures including: increasingly complex models of intelligence, test publishers' desires to provide clinically useful assessment instruments with greater interpretive value, test publishers' desires to include minor factors that may be of interest to researchers (but are not clinically useful), and liberal statistical criteria for determining the factor structure of tests. The present study examined the number of factors measured by several historically relevant and currently employed commercial tests of cognitive abilities using statistical criteria derived from principal components analyses, and exploratory and confirmatory factor analyses. Two infrequently used statistical criteria, that have been shown to accurately recover the number of factors in a data set, Horn's parallel analysis (HPA) and Minimum Average Partial (MAP) analysis, served as gold-standard criteria. As expected, there were significant increases over time in the number of factors purportedly measured by cognitive ability tests (r=.56, p=.030). Results also indicated significant recent increases in the overfactoring of cognitive ability tests. Developers of future cognitive assessment batteries may wish to increase the lengths of the batteries in order to more adequately measure additional factors. Alternatively, clinicians interested in briefer assessment strategies may benefit from short batteries that reliably assess general intellectual ability.

Additional comments/conclusions by the authors (followed by my comments/responses)

Frazier/Youngstrom comment: The extensive use of cognitive ability batteries in psychological assessment, an increased market for psychological assessments in general, a desire to create tests that are marketable to both clinicians and researchers, and the desire to increase the reliability of IQ measures may create a pressure on publishers to market ability tests that measure everything that other tests measure and more. This, in turn, forces other ability test publishers to try to keep pace.

  • McGrew comment/response: First, I will not attempt to comment on the "desires/pressures" of test developers/publishers of the other major intelligence batteries included in their analyses (Wechsler batteries, SB-IV, K-ABC, DAS). I restrict my comments to my experiences with the WJ-R and WJ III.
  • As a coauthor of the WJ III, and the primary data analyst for the WJ-R, I personally can vouch for the fact that there was no pressure exerted by the test publisher, nor we as co-authors, to measure more factors for the sake of just measuring more. As articulated clearly in the original WJ-R technical manual (McGrew, Werder & Woodcock, 1991), and subsequently summarized in the WJ III technical manual (McGrew & Woodcock, 2001), the driving force behind the number of factors was theory-driven, with the input of two of the most prominent psychometric intelligence theorists and factor analysts....John Horn and Jack Carroll (click here, here.) Both Horn and Carroll where intimately involved in the design and review of the factor results of the WJ-R and WJ III norm data. The driving "desire/pressure" during the revision of the WJ-R and WJ III was to validly measure, within practical constraints, the major features of the broad CHC/Gf-Gc abilities that are well established from decades of research (see Carroll's 1993 seminal work, click here, here). For additional information re: the involvement of Horn and Carroll in these deliberations, read the relevant sections of McGrew's (that be me) on-line version of CHC Theory: Past, Present, Future. If there was an underlying driving "pressure", it was to narrow the intelligence theory-practice gap.


Frazier/Youngstrom comment: Several important findings emerged from the present study. As predicted, commercial ability tests have become increasingly complex. While the length of these tests has risen only moderately, the number of factors purportedly measured by these tests has risen substantially, possibly even exponentially. It should be noted, however, that the possibility of an exponential increase in the number of factors purportedly measured may be due to inclusion of two outliers, the WJ-R and WJ-III. Possibly even more convincingly, the ratio of test length to factors purported has decreased dramatically. These trends suggest that test authors may be positing additional factors without including a sufficient number of subtests to measure these factors. When more accurate, recommended, statistical criteria were examined commercial ability tests were found to be substantially overfactored.

  • McGrew comment/response: My comment is primarily one of clarification for readers. Frazier and Youngstrom's statement that the ratio of test length to factors has decreased may be relevant to the other batteries analyzed, but is NOT true for the WJ-R and WJ III. The broad CHC factors measured by the WJ III are all represented by at least 3 or more test indicators, a commonly accepted criterion for proper identification of factors. Frazier and Youngstrom (and readers of their article) may find it informative to note that in Jack Carroll's final publication (The higher-stratum structure of cognitive abilities: Current evidence supports g and about ten broad factors. In Helmuth Nyborg (Ed.), The scientific study of general intelligence: Tribute to Arthur R. Jensen. Elsevier Science/Pergamon Press.-click here to access pre-pub copy of Carroll's chapter), Carroll stated that the WJ-R battery (which, when compared to the WJ III, has a lower test-factor ratio) was a "sufficient" set of data "for drawing conclusions about the higher-stratum structure of cognitive abilities." In describing the WJ-R dataset, he stated that "It is a dataset that was designed to test factorial structure only at a second or higher stratum, as suggested by Carroll (1993, p. 579), in that it has sufficient test variables to define several second-stratum factors, as well as the single third- stratum factor, but not necessarily any first-stratum factors." Jack Carroll is no slouch when it comes to the application of factor analysis methods. In fact, he is generally considered as one of the masters of the "art and science" of factor analysis and his contributions of the use of factor analysis methods to the study of cognitive abilities is well known (I recommend folks to read Chapter 3 in Carroll's seminal treatise on the factor structure of human cognitive abilities--"Chapter 3: Survey and Analysis of Correlational and Factor Analytic Research on Cognitive Abilities: Methodology). Frazier and Youngstrom place all of their eggs primarily in the "science" of factor analysis (emphasis on statistical tests). There is an "art" to the practice of factor analysis, something that is missing from the raw empirical approach to their investigation.

Frazier/Youngstrom comment: Results of the present study also suggest that overfactoring of ability tests may be worsening, as the discrepancy between the purported number of factors and the number indicated by MAP and HPA has risen over time and the ratio of subtests to factors purported has decreased substantially as well. While commercial pressures and increasingly complex models of human cognitive abilities are likely contributing to these recent increases, these explanations were not investigated in the present study.

  • McGrew comment/response: Where's the beef/data that supports the conclusion that "commercial pressures...are likely contributing to these recent increases?" In the absence of data, such a statement is inappropriate. Yes, increasingly complex models of human cognitive abilities are contributing to batteries that measure more abilities. Science is a process of improving our state of knowledge via the accumulation of evidence over time. The most solid empirical evidence supports a model of intelligence (CHC or Gf-Gc theory) that includes 7-9 broad stratum II abilities. Shouldn't assessment technology stay abreast of contemporary theory? I think the answer should be "yes." Since the authors state that "these explanations were not investigated in the present study" they should have refrained from their "commercial pressures" statement. I'm a bit surprised that such a statement, devoid of presented evidence, survived the editorial process of the journal.

Frazier/Youngstrom comment: Rather, evaluation centered on the hypothesis that test developers have been determining test structure using liberal, and often inaccurate, statistical criteria. This hypothesis was supported.."

  • McGrew comment/response: Aside from the failure to recognize the true art and science of the proper application of factor analysis, Frazier and Youngstrom commit a sin that is often committed by individuals (I'm not saying this is true of these two individuals) who become enamored by the magic of quantitative methods (myself included, during my early years...until the likes of John Horn, Jack McArdle, and Jack Horn personally tutored me on the limitations of any single quantitative method, like factor analysis). Briefly, factor analysis is an internal validity method. It can only evaluate the internal structural evidence of an intelligence battery. When I was a factor analytic neophyte, I was troubled by the inability to clearly differentiate (with either exploratory or confirmatory factor methods) reading and writing abilities (Grw) from verbal/crystallized (Gc) abilities. I thought the magic of factor analysis should show these as distinct factors. Both Horn and McArdle gently jolted my "factor analysis must be right" schema by reminding me that (and I'm paraphrasing from memory) "Kevin...factor analysis can only tell you so much about abilities. Often you must look outside of factor analysis, beyond the internal validity findings, to completely understand the totality of evidence that provides support for the differentiation of highly correlated abilities." In particular, Horn and McArdle urged me to examine growth curves for highly correlated abilities that could not be differentiated vis-à-vis factor analysis methods. When I examined the growth curves for Grw and Gc in the WJ-R data, I had an epiphany (note...click here for a report that includes curves for all WJ III tests....note, in particular, the differences between the reading/writing (Grw) and verbal (Gc) tests....tests that often "clump" together in factor analysis). They were correct. Although EFA and CFA could not clearly differentiate these factors, the developmental growth curves for Grw and Gc where dramatically different...so different that it would be hard to conclude that they are the same constructs. Long story short...Frazier and Youngstrom fail to recognize, which they could have in their discussion/limitations section, that construct validity is based on the totality of multiple sources of validity evidence. Internal structural validity evidence is only one form...albeit one of the easier ones to examine for intelligence batteries given the ease of factor analysis these days. As articulated in the Joint Test Standards, and nicely summarized by Horn and others, aside from structural/internal (factor analysis) evidence, evidence for constructs (and purported measures of the constructs) must also come from developmental, heritability, differential outcome prediction, and neurocognitive evidence. Only when all forms evidence are considered can one make a proper appraisal of the validity of the constructs measured by a theoretically-based intelligence battery. For those wanting additional information, click here (you will be taken to a discussion of the different forms of validity evidence as Dawn Flanagan and I discussed in our book, the Intelligence Test Desk Reference.)

I could go on and on with more points and counterpoints, but I shall stop here. I would urge readers to read this important article and integrate the points above when forming opinions regarding the accuracy/appropriateness of the author’s conclusions, particularly with regard to the WJ-R and WJ-III batteries. Also, consulting the WJ-R and WJ-III technical manuals, where multiple sources of validity evidence (internal and external) are presented to support the factor structure of the batteries are presented, is strongly recommended.



Technorati Tags: , , , , , , , , , , , , , , , ,

powered by performancing firefox