The "art and science" of psychologcal testing - we are the instrument

There has been an extremely active thread on the CHC listserv that has resurrected a quote that has been attributed to me...."we are the instrument"...with re: to the art and science of "intelligent" intelligence testing and interpretation. It reminded me of a quote that captures a good portion of the variance of my "we are the instrument" mantra. It is reproduced below. The source is also listed.

Meyer et al. (2001). Psychological testing and psychological assessment. American Psychologist, February

• "Tests do not think for themselves, nor do they directly communicate with patients. Like a stethoscope, a blood pressure gauge, or an MRI scan, a psychological test is a dumb tool, and the worth of the tool cannot be separated from the sophistication of the clinician who draws inferences from it and then communicates with patients and professionals”

Caffeine (liquid Gf?) and working memory (Gsm-MW) performance

A tip of the hat to Improbable Research for the interesting post re: the optimal number of cups of cafeine [ liquid Gf :) ] to improve working memory (Gsm-MW) performance. The backward counting task seems to me (from a logical content validity perspective) to be measuring working memory.


Technorati Tags: psychology, cognition, intelligence, caffeine, working memory, Gsm, MW, cognitive efficiency, CHC theory, education

AGS now part of Pearson Assessments

A little biz tidbit from the world of assessment companies. American Guidance Services (AGS) has long been a source of solid psychometic measures of cognitive and achievement batteries. AGS, Psych. Corp, and Riverside Publishing have been the primary trilogy for clinical measures of intelligence (Wechslers, Kaufman family of measures, Woodcock family of measures).

Today, in a press release, it was announced AGS is now officially under the broad umbrella of Pearson Assessments.


Technorati Tags: publishing, assessment, psychological assessment, IQ testing, AGS, American Guidance Services, Pearson Assessment

Definition of learning disabilities: Delay vs disorder debate

A good friend and WJ III coauthor (please note that I'm mentioning this as a potential conflict of interest disclosure), Nancy Mather, together with Noel Gregg, just published a response article to Dombrowski, Kamphaus and Reynold's (2004) article on a proposed definition of learning disabilities. The two articles, in a simplified sense, present an LD = delay vs. disorder debate. I'm not going to take sides, but instead, have provided the references and abstracts for both articles and have also provided URL links to both.

Read, think, react, debate, etc..............

Mather, N. Gregg, N. (2006). Specific Learning Disabilities: Clarifying, Not Eliminating, a Construct. Professional Psychology: Research and Practice 37(1), 99–106

Abstract

• Over the past decades, both researchers and practitioners in the field of specific learning disabilities (LD) have grappled over an appropriate definition as well as the subsequent operational diagnostic criteria. In a recent article in Professional Psychology: Research and Practice, S. C. Dombrowski, R. W. Kamphaus, and C. R. Reynolds (2004) proposed a new set of suggested diagnostic criteria that challenges many of the traditional beliefs about these disorders. This article provides a critique of their proposed criteria as well as an alternative approach for LD identification that is more aligned with current understanding of the etiology and characteristics of these disorders. (click here to read/view)
  

Dombrowski, S. C., Kamphaus, R. W., Reynolds, C. R. (2004). After the demise of the discrepancy: Proposed learning disabilities diagnostic criteria. Professional Psychology: Research and Practice, 35, 364–372.

• Pending legislation and positions taken by the U.S. Department of Education may radically alter current learning disabilities (LD) definitions and diagnostic approaches. Proposals include eliminating a discrepancy model and incorporating a more comprehensive approach to LD assessment but one based on more subjective clinical judgment. Although this effort to change is laudable, it does not address the residual problems that will continue to plague the field: the lack of specificity of the construct of LD and the inconsistent and idiosyncratic approach to diagnosis taken by practitioners and researchers across and within states. This article proposes a new LD classification model that transcends educational and psychiatric systems of diagnosis, calls for a uniform and national diagnostic system, and suggests renaming the disorder (e.g., developmental learning delay). (click here to read/view)
  

Technorati Tags: psychology, education, intelligence, IQ testing, psychological assessment, LD, learning disabilities, developmental dealy, disorder, cognition, education, special education

IQ PREPLOG: CHC abilities and effects on reading skills--beyond g--some specific abilities important

This is an IQ PREPLOG post. The following manuscript is now "in press" and can be viewed by clicking here.

Floyd, R., Keith, T., Taub, G. McGrew, K. (2006). Cattell–Horn–Carroll Cognitive Abilities and Their Effects on Reading Decoding Skills: g Has Indirect Effects, More Specific Abilities Have Direct Effects. School Psychology Quarterly (in press 2-28-06)

Abstract

• Previous research has indicated the importance of several cognitive abilities to the acquisition and development of reading decoding skills. The findings from such research may be limited because some abilities, such as general intelligence (g), frequently have been omitted from analyses. Drawing on the ability taxonomy of the Cattell–Horn–Carroll (CHC) theory, this study employed structural equation modeling to examine the effects of CHC abilities on reading decoding skills using 5 age-differentiated subsamples from the standardization sample of the Woodcock–Johnson III (Woodcock, McGrew, Mather, 2001). Using the Spearman Model including only g, strong direct effects of g on reading decoding skills were demonstrated at all ages. Using the Two-Stratum Model including g and broad abilities, direct effects of the broad abilities Long-Term Storage and Retrieval, Processing Speed, Crystallized Intelligence, Short- Term Memory, and Auditory Processing on reading decoding skills were demonstrated at select ages. Using the Three-Stratum Model including g, broad abilities, and narrow abilities, direct effects of the broad ability Processing Speed and the narrow abilities Associative Memory, Listening Ability, General Information, Memory Span, and Phonetic Coding were demonstrated at select ages. Across both the Two-Stratum Model and the Three-Stratum Model at all ages, g had very large but indirect effects. The findings suggest that school psychologists should interpret measures of some specific cognitive abilities when conducting psychoeducational assessments designed to explain reading decoding skills.

Technorati Tags: psychology, cognition, psychological assessment, iq testing, CHC theory, Cattell-Horn-Cattell, Gf-Gc theory, reading, decoding, special education, education, achievement, SEM, structural equation modeling, g, general intellignece, Spearman

Mr. Peanut and working memory

"Hi...my name is Mr. Peanut."

I ran across this interesting task for measuring working memory (Gsm-MW). It would appear to be a creative and interesting approach to measuring visual working memory. I couldn't resist introducing Mr. Peanut to the blog. A description of the task, as described in a recent journal article, is presented below.

• "The peanut task was adapted from Pascual-Leone (de Ribaupierre, Lecerf, & Bailleux, 2000; de Ribaupierre et al., 1989) and Case (1985) and measures WM. A clown figure (M. Peanut) was presented on a sheet of paper, with coloured spots on it (16 possible locations; Figure 5). This clown figure was then removed, and a second blank clown figure was immediately presented on a sheet of paper. The Figure 5. Study 1: The colour-peanut task. Example of a complexity 6 item. children's task was to recall the location of spots by placing coloured chips on this figure. Two versions were administered: a purple-peanut in which all spots were of the same colour and a colour-peanut in which spots were of different colours (seven possible colours). In the coloured version, children had to recall both the colour and the location. The number of locations or the number of colour-location associations to remember varied from two to five, with five items for each level of complexity. Trials of different complexity were randomly dis- tributed throughout the task, with the same order for all children. Overall exposure time was 1 s per spot; thus, a clown figure with three spots was presented during 3 s. The score used here was the total number of correct trials."

Technorati tags: CHC theory. psychology. intelligence. cognition. educational psychology. iq testing. psychological testing. working memory. visual-spatial. Gv. memory.

More on the Mozart effect - Can music increase intelligence?

As I've noted before, "Current Directions in Psychological Science is a must read for scholars looking for a brief, contemporary “taking stock” summary of an area of psychological research. I review the contents of this journal on a regular basis. The articles are a good starting point for getting your head into a new research area or, for ascertaining the general consensus re: a research topic in psychology (broadly defined)."

The latest issue has such a summary re: the impact of music on cognitive abilities (Schellenberg, 2005 - Music and Cognitive Abilities). This general area of research often refers to the disputed "Mozart effect" , which has had more than ample play in the popular press. This current article (click here to view) provides one scholarly take on the current state of the empirical literature, possible explanations of the benefit of music on cognitive development, etc.

Abstract (emphasis added by blogmaster)

• Does music make you smarter? Music listening and music lessons have been claimed to confer intellectual advantages. Any association between music and intellectual functioning would be notable only if the benefits apply reliably to nonmusical abilities and if music is unique in producing the effects. The available evidence indicates that music listening leads to enhanced performance on a variety of cognitive tests, but that such effects are short-term and stem from the impact of music on arousal level and mood, which, in turn, affect cognitive performance; experiences other than music listening have similar effects. Music lessons in childhood tell a different story. They are associated with small but general and long-lasting intellectual benefits that cannot be attributed to obvious confounding variables such as family income and parents’ education. The mechanisms underlying this association have yet to be determined.

Technorati Tags: Mozart effect, music, cognition, intelligence, psychology, musical abilities, Ga, auditory processing

SES and student achievement - new meta-analytic review

White's (1982) meta-analytic review of the relation between SES and student achievement has often been cited as the seminal source on the empirical relationship between SES and school achievement, issues in measuring SES in reseach, etc. Now White's seminal review has been updated by a contemporary meta-analysis by Sirin (2005) in the Review of Educational Research (click here to view entire article - note...file is 2+ MB). This article is must reading for educational researchers and those who develop and standardize psychoeducational assessment batteries.

Although you can read the entire article by clicking the URL above, a few highlights are presented below.

1. Using Cohen's (1977) guidelines, the overall ES of the present study reflects a medium level of association between SES and academic achievement at the student level and a large degree of association at the school level.
2. Of all the factors examined in the meta-analytic literature, family SES at the student level is one of the strongest correlates of academic performance. At the school level, the correlations were even stronger. This review's overall finding, therefore, suggests that parents' location in the socioeconomic structure has a strong impact on students' academic achievement. Family SES sets the stage for students' academic performance both by directly providing resources at home and by indirectly providing the social capital that is necessary to succeed in school. Family SES also helps to determine the kind of school and classroom environment to which the student has access
3. Beyond the main findings, the results from this review also show that the magnitude of the relationship between SES and academic achievement is contingent upon several factors. More specifically, methodological characteristics, such as the type of SES measure, and student characteristics, such as student's grade, minority status, and school location, moderated the magnitude of the relationship between SES and academic achievement.
4. The authors conclude that researchers must continue to assess student's SES as part of their understanding of family effects on academic performance. The decision about how to measure SES, however, is a complicated one. On the basis of the results from this meta-analysis, the following points may help researchers to better capture students' social and economic background in education research and test development plans.
   

• The unit of analysis is of critical importance.

• SES is a multi-dimensional construct, and different components yield different results. The use of participation in school lunch programs as a measure of SES, though common,is conceptually problematic.

• Only a small number of studies considered neighborhood characteristics as part of their assessment of students'social and economic background.

• SES seems to have different meanings for students from different ethnic backgrounds. One of the main findings of this review was that, for minorities, SES did not seem to be as strongly related to academic achievement as it was for Whites.

• It is important to decide from whom the SES data should be collected. As the results show, studies that collect SES data from students yield much smaller correlations with achievement than do studies that collect data from parents.

• The location of schools should be an integral part of research on students' SES and academic achievement. Without consideration of the geographical location of the school, the observed correlations between SES and academic achievement are likely to confound the differences between rural, urban, and suburban schools. [Blogmaster note - this is often one of the most important (yet one of the least controlled) variables in the development of a norming plan for the standardization of cognitive and achievement tests. Community SES information, as well as individual level SES information, should be included in the sampling design plan of any major cognitive and/or achievement battery. Community SES has always been a critical component of the sampling plans of the three editions of the Woddock-Johnson (WJ) battery (1977, 1989, 2001) - conflict of interest note----I'm a coauthor of the WJ III).
  

Technorati Tags: psychology, intelligence, achievement, SES, socioeconomic status, applied psychometrics, measurement, test development, education, cognition, psychological testing, IQ testing

Kindergarten retention effect on cognitve growth--an adverse effect

Having been a school psychologist for 10 years prior to moving on to my current endeavors, I'm not surprised to see the very active thread on the NASP listserv re: decisions to retain or promote children at this time of the year. I used to hate this time of year as teachers would refer children for possible retention....as if my magical box of tests and clinical experience would help make the decision easier. I agonized over these decisions and even participated in Gessell training to help me better make these decisions. My philosophy re: retention/promotion varied over my 10 year career. Eventually the point that started to gnaw at me was the mounting empirical evidence that argued against the practice of retention.

That being said, and no longer being in the trenches, I can now retreat to the safe world of scholarship and make a contribution by providing an FYI to a recently published (click here to read entire article) large scale study that again argues against the policy of retaining kids (I'm talking about at kindergarten...when it is most prevelant). Below is the citation, abstract, and a few select concluding statements from a non-to-shabby large scale national study (that means big n's folks)--the Early Childhood Longitudinal Study.

The results continue to argue against the benefits of retention and, in fact, suggest that retention can result in stunted cognitive growth in reading and math. Where talking about a potential adverse negative impact of the loss of 2/3 of a standard deviation in cognitive growth in reading and math 1 year after retention.....that is not small!

Yes.....each decision is an n=1 decision. Yes...I know that there will likey be cases where it makes sense...and there will be success stories. But, one cannot argue with the overwhelming empirical evidence that argues against a knee-jerk retention policy. Providing strong, convincing, reasonable evidence in favor of retaining a child in Kdgn would appear to be a burden of proof that those arguing for retention need to bear.

Hong, G. & Raudenbush, St. (2005). Effects of Kindergarten Retention Policy on Children's Cognitive Growth in Reading and MathematicsEducational Evaluation and Policy Analysis Fall 2005, Vol. 27, No. 3, pp. 205-224


Abstract

• Grade retention has been controversialfor many years, and current calls to end social promotion have lent new urgency to this issue. On the one hand, a policy of retaining in grade those students making slow progress might facilitate instruction by making classrooms more homogeneous academically. On the other hand, grade retention might harm high-risk students by limiting their learning opportunities. Analyzing data from the US Early Childhood Longitudinal Study Kindergarten cohort with the technique of multilevel propensity score stratification, we find no evidence that a policy of grade retention in kindergarten improves average achievement in mathematics or reading. Nor do we find evidence that the policy benefits children who would be promoted under the policy. However, the evidence does suggest that children who are retained learn less than they would have had they instead been promoted. The negative effect of grade retention on those retained has little influence on the overall mean achievement of children attending schools with a retention policy because the fraction of children retained in those schools is quite small. Nevertheless, the effect of retention on the retaineesis considerably large.

Select concluding statements (italics added by blogmaster)

• "Children who were retained would have learned more had they been promoted. This was true in both reading and math. After being retained for one year, the average loss in academic growth experienced by the retainees was about two-thirds of a standard deviation of the outcome in each subject area, equivalent to almost half a year's expected growth."

• "Summarizing the above empirical findings, we conclude that the kindergarten retention treatment leaves most retainees even further behind, and, therefore, impedes these children's cognitive development over the repetition year. In general, at-risk children promoted to the next grade level seemed to have a better chance of growth acceleration. The results from this study challenge the practice of routinely using kindergarten retention as a solution to the difficulties experienced by young children."

Technorati tags: psychology. cognition. education. educational psychology. retention. kindergarten. social promotion. school retention.

Recent literature of interest 03-02-06 posted

This weeks (and lasts) recent literature of interest can be found by clicking here.

Technorati tags: teaching tool. psychology.

Off task - consilience

Something to reflect on when you need a break. Some interesting thoughts re: the relations between science, philosophy, and technology over on the UberKuh blog.

Mapping Consilience (click here)



Technorati Tags: consilience, philosophy, science, psychology, technology

Cognitive automaticity

I'm starting to sound like a broken record regarding the continuing great posts over at the Eide Neurolearning blog. Kudos to ENL for the new post re: cognitive automaticity. These folks do good stuff.

In Praise of Automaticity (click here to view post)

Technorati Tags: psychology, neuroscience, neuropsychology, information processing, cognition, automaticity, executive functioning, education, fMRI

Executive function, Gf and Gc - a WAIS CFA study

Another great post over on the Developing Intelligence blog re: executive functioning (EF), this time an empirical study that investigated the relations between components of EF and indicators of Gf and Gc as measured by select WAIS-III tests and the Ravens Progressive Matrices. DI provides a link to the study abstract. To augment the DI post, I've posted a copy of the original Psychological Science article for those that want "the rest of the stofy" (click here).

As most CHC folks would predict, I do have one methodological bone to pick with the study. Gf is defined by the Ravens (good choice) and WAIS-III Block Design. As we all know from contemporary CHC-based CFA studies, and Carroll's seminal treatise on human cognitive abilities, Block Design is primarily an excellent indicator of Gv (Visualization and/or Spatial Relations) and is NOT an indicator of Gf. If it includes any Gf variance, it is small. Thus, the Gf construct in this study is confounded with Gv abilities. Take the Gf-EF findings with a grain of construct-irrelevant variance salt.

As summarized at DI (go to DI link above for complete discussion at DI - italics below added by me).

• Structural equation modeling confirmed that updating is of particular importance to the correlations between Gc, Gf, and WAIS IQ (which all shared 41 to 48% of their variance with updating). However, once this variance was included as a common underlying factor between the intelligence measures, the correlations with the other two EF subfunctions was no longer significant. This critical finding suggests that shifting and inhibition contribute to intelligence measures only insofar as they covary with updating abilities.

Technorati tags: structural equation modeling. neuropsychology. CHC theory. psychology. intelligence. WAIS-III. Wechsler. cognition. educational psychology. iq testing. psychological testing. intelligence theory. executive functioning. Gf. working memory. visual-spatial. Gv. brain. neuroscience. fluid intelligence.

Brain patterns of parents of kids with Aspergers

Hat tip to Eide Neurolearning blog for interesting article regarding brain patterns of kids with diagnosed Asperger Syndrome

Brain Differences Seen Among Parents of Kids with Aspergers (click this link)

Technorati Tags: psychology, educational psychology, special education, aspergers, intelligence, neuroscience, brain, neurospcyhology

NCLB effect on ESL and spec ed kids - an editorial comment

Warning...this is a soap box editorial from the blogmaster.

Although this blog is primarily focused on content related to cognitive abilities and intelligence, it is no secret that I also wear another hat....consultant to the U of Mn National Center on Educational Outcomes re: issues related to inclusion of kids with disabilities in NCLB (No Child Left Behing)-related state assessment accountability systems. For that matter, I should lay my cards on the table and indicate that I'm a strong advocate for including as many kids that are "on the margins" in NCLB-driven assessment systems.

Why? Although NCLB has a number features that don't jive with educational psychological research and theory, to me NCLB has one overriding significant positive feature. Namely, regular educators are now responsible, for the first time I can remember, for the performance of the "kids on the margins." Although there are a ton of logistical and psychometric issues re: how to include more kids with disabilities in groups assessments, how to make assessessments more accessible, etc., the fact that the general education system is now held responsible for these kids is, IMHO, the most important outcome of NCLB...vis-a-vis the requirement to report performance of different subgroups for schools (e.g., special education; ESL; etc). These kids can no longer be ignored.

In Education Week there is an article (click here) re: how this NCLB general-other education "hook" is working. Although the article deals with how regular education is now paying more attention to the education of ESL kids, it is my contention (and I know it is happening from what I hear via my NCEO work) that NCLB is having the same effect where general education is now paying more attention to the education of kids with disabilities. This NCLB accountability "hook" is very, very important. We need to keep it and fight against attempts to dilute the law vis-a-vis increasing the exclusion rates of more kids with disabilities in state accountability systems. If we lose this "hook", the education of kids with disabilities could be set back decades...back the the small rooms in the basements of schools...sorry, a little melodrama just hit me :)

Of course, all is not rosey. As the article points out, their are negative effects that seem to be happening to ESL kids because of this NCLB hook..namely, the narrowing of the curriculum to "teach to the test." Somehow these unintended outcomes need to be addressed.

However, the key point for this personal editorial is that the general-education/special population accountability hook is one of the best things ever to happen to kids who are on the margins in our schools. We need to fight and maintain this hook...the Education Week article illustrates that it is having a positive impact on making general education pay attention more to these kids.

In the same issue there was another interesting article re: the approval of states to use a growth model for NCLB accountability (click here to view)


Technorati tags: NCLB. ESL. special education. learning disabilities. psychology. education. educational psychology.

Mental imagery debate

Another great post over at the Developing Intelligence blog re: a recap of the debate regarding mental imagery. Below is a the URL link to the complete post (which includes links to other good stuff) and the first paragraph of the post. Thanks DI for your continued great stuff.

Pictures in the Brain (click here)

• "The new issue of Scientific American Mind has a nice recap on the mental imagery debate, which has been going strong for over 10 years now. The debate proceeds roughly as follows: when we imagine something, are we merely activating its abstract, propositional representation in long term memory, or is imagination actually a process of neurally reinstantiating this distributed information into a visual form available for inspection and spatialmanipulation?"

Technorati Tags: psychology, intelligence, imagery, cognition, mental imagery, intelligence theories, Gv, visualization

New psychology and mind and brain portals

Mind and brain portals launch on Wikipedia

Click on the URL above to visit Mind Hacks to read a post (with appropriate links) to new mind and brain and psychology portals at Wikipedia. Portals are intended to keep you up-to-date with the latest news in the topic area and also to distribuate news and messages from people interested in that specific professional community.

Technorati Tags: psychology, neuroscience, educational psychology, cognition, teaching tool, portals, Wikipedia

Executive functioning - excellent post from DI blog

I must confess that Chris Chatham's Developing Intellignence blog is becomming one of my favorites. At times the material is quite complex and deals with intelligence at a different level than the focus of my work and writings at IQs corner, but many times it is well worth the read.

Today he has one of the better posts I've recently read about executive functioning. Below is the URL to his post and the first paragraph of his rather lengthy post (full of links to articles, etc.). This is a post worth reading.

Under The Rug: Executive Functioning

• If you want to build an intelligent and biologically-plausible system, you of course need an actuator or motor subsystem, object recognition capability, several different kinds of memory capacity, and probably several other subsystems corresponding to various regions of the human brain. But what kind of subsystem would be capable of orchestrating
  these capacities and coordinating them to produce intelligent behavior? To put it another way, does intelligence consist entirely in interactions between various capacities, or is there a cognitively- and anatomically-distinct agent that coordinates them? These questions (albeit in slightly different form) are the same as those confronted in
  "executive functioning" research.

Technorati Tags: executive function, executive functioning, neuroscience, neuropsychology, brain, intelligence, fluid intelligence, Gf, working memory, attention

What to look for in a psychological evaluator--from the Spec Ed Law blog

Interesting post on the Special Education Law blog on what they recommend parents look for in professionals who may conduct a psychoeducational evaluation of their child. The readers of IQs corner who engage in applied psychological testing should take a peak.....just to know what advocates for parents are suggesting

Guidelines for Parents to Select an Evaluator

.

Technorati Tags: psychological assessment, IQ testing, special education, law, psychological testing

Neuro testing of high school athletes

Thanks to the BrainInjury blog for the interesting post regarding the use of baseline neurological concussion examination of high school athletes.

• "The Brain Injury Association of New Jersey has received funding for the administration of a grant to provide high school athletes at up to 100 schools with pre-season cognitive baseline evaluations".......click URL below for complete story

New Jersey High School Athletes To Get Base Line Concussion Testing

Of particular interest is the link to the company that does the computerized testing......ImPact

Technorati Tags: neuroscience, brain, neuropsychology, concussions, brain injury, intelligence, psychological testing, intelligence, sports injuries