Saturday, May 21, 2005

Clinical interpreation of figural Gf matrices - a "radical" cross-battery idea

Matrices tests are frequently used as primary markers of fluid intelligence (Gf) and are often included in short/brief IQ screening batteries. And, as we know from the CHC taxonomy, the reasoning involved in different measures may invoke different types of reasoning or combinations of reasoning (viz., induction-I; general sequential [deductive] reasoning-RQ; quantitative reasoning-RQ).

Astute clinicians often attempt to ascertain if an individual’s pattern of successes and failures on different item types (within the same Gf test) are systematic and suggestive of differential strengths and weaknesses within Gf.

Two studies (one previously published in Intelligence and one “in press”) suggest an interesting framework by which to analyze the task demands of figural Gf items, regardless of I, RQ, RQ demands. The framework is briefly summarized below. Those interested in the empirical studies, and the successful use of this framework in the development of an automatic figural matrices item generatator, are encouraged to read the original articles (Primi, 2001 citation can be found in Arendasy & Sommer, 2005—listed below).

  • Arendasy, M., & Sommer, M. (2005). The effect of different types of perceptual manipulations on the dimensionality of automatically generated figural matrices. Intelligence, In Press, Corrected Proof.

According to Primi, and described by Arendasy and Sommer, the component processes of figural Gf matrices can be dissected as per four main item design features (called “radicals”). These include:

  • Number of elements
  • Number of rules
  • Type of rules
  • Perceptual organization.

According to recent research, the first two radicals are associated with the amount of information that has to be stored and processed in working memory (MW), and thus, may contribute to the explanation of the strong, positive, and significant relation between Gf and MW. Embretson (1998, 2002) has also associated the type of rules with working memory capacity--more difficult rules seem to put a heavier demand on the working memory capacity than easier rules. [See prior post for more MW->Gf discussion].

Finally, the perceptual organization features of figural matrices have been studied the least. This feature category involves: (a) perceptual features of the elements of the figural matrices and (b) the impact of the Gestalt perceptual principles of proximity, similarity, common region and continuity (Rock & Palmer, 1990; Wertheimer, 1923).

Interesting stuff…don’t you all think? To me, from my table in the corner of Barnes and Nobles on a Saturday night in St. Cloud, Mn, might this suggest yet another set of dimensions by which Gf (figural) matrices tests might be analyzed in order to understand why individuals may perform differently on tests that, on face value, appear to measure the same abilities with the same general class of items?

Might CHC cross-battery principles be extended to yet another level (for this class of Gf tests) via the analysis and classification of figural Gf matrices as per these radicals [don’t you love that word?---it brings me back to the 70s’]? Inquiring minds (at least one in central MN) want to know. Dawn and Sam…what say you?

So many ideas and data…so little time. I need more caffeine.

Measurement of practical intelligence via video technology

For years (actually, since I finished my dissertation on the topic) I've found Stephen Greenspan's Model of Personal Competence, which includes (depending on which revision of his model one is examining) the broad competence domains of physical and emotional competence and conceptual, practical, and social intelligence, a useful meta-model for conceptualizing human competence. [Note - For CHC thinkers, CHC abilities would fall under the broad umbrella of Conceptual Intellignece.] However, a major stumbling block in research and applied measurement has been difficulty in developing valid measures of social and practical intelligence.

Although based on a very small sample of convenience, Greenspan and Yahon-Chamovitz recently pubished the following encouraging article. The article reports on the promising use of technology (video portrayal of everday events) in the development of potentially valid measures of the cognitive component of practical intelligence. The article reference and abstract is provided below.

Anyone interested in developing measures of the cognitive component of practical intelligence should give this brief article a quick look. I can envision subjects responding to questions based on their viewing of video clips of everyday events presented via the screen of a handheld PDA (personal digital assistant).

YalonChamovitz, S., & Greenspan, S. (2005). Ability to identify, explain and solve problems in everyday tasks: preliminary validation of a direct video measure of practical intelligence. Research in Developmental Disabilities, 26(3), 219-230.
  • Recent developments in the definitional literature on mental retardation emphasize the need to ground the concept of adaptive behavior in an expanded model of intelligence, which includes practical and social intelligence. Development of a direct measure of practical intelligence might increase the likelihood that an assessment of this domain would be included in the diagnostic process of mental retardation. The current paper reports on the preliminary exploration of the validity and utility of using a videotaped portrayal of everyday tasks, with built-in errors, as a measure of practical intelligence. A correlation of .79 was found between the practical intelligence video score and the Vineland domestic and community sub-domains score in 50 adults with mild and moderate mental retardation. This suggests that the instruments are essentially measuring the same domain of human competence. The unexplained variance may be attributed to the fact that the video measure is more directly measuring cognition.

Friday, May 20, 2005

Working memory, domain knowledge and higher-level cognitive performance

Hambrick, D. Z., & Oswald, F. L. (2005). Does domain knowledge moderate involvement of working memory capacity in higher-level cognition? A test of three models. Journal of Memory and Language, 52(3), 377-397.

What makes a person more competent on higher-level cognitive tasks? A larger reservoir of domain-relevant knowledge (software), larger working memory capacity (hardware), or the interaction of the two? Inquiring minds want to know. Contemporary cognitive research has implicated both working memory capacity (Gsm-MW) and domain knowledge (Gk) in higher-level cognition.

In their research article, Hambrik and Oswald investigated the interplay between Gk and MW. According to the authors, this study represents one of a few studies that have investigated the interplay between MW capacity and depth of Gk. The authors characterize MW and Gk in the following manner:

  • “Working memory capacity might be thought of as a stable component of higher-level cognition---a possible ‘hardware' aspect of cognition”
  • “Domain knowledge…might be thought of as a modifiable ‘software’ aspect of cognition”

Predictions regarding higher-level cognitive performance in a relatively large sample (n=381…yes…you guessed it…undergraduate students in an introductory psychology course) were based on three competing hypothesis.
  • Compensation hypothesis – Gk will attenuate the influence of MW capacity on higher-level cognition. In other words, greater Gk will reduce or eliminate the influence of MW capacity on domain-relevant tasks – there is an interaction effect.
  • The rich-get-richer hypothesis – larger MW capacity will enhance the use of Gk on higher level cognitive tasks. “In other words, people with high levels of working memory capacity tend to benefit from domain knowledge more than those with lower levels.”
  • The independent influences hypothesis – MW and Gk make independent “additive” contributions to higher-level cognition
The participants performance was measured on two tasks. One required tracking information in a dynamic environment (tracking the movements of baseball players on a baseball diamond) while the other was structurally isomorphic with arbitrary content (spaceship task).

The results revealed “greater use of baseball knowledge in the baseball task than in the spaceship task. However, even at high levels of baseball knowledge, this knowledge use did not alter the relationship between working memory capacity and task performance. This finding is inconsistent with compensation and rich-get-richer hypotheses. Instead, it suggests that working memory capacity and domain knowledge may operate independently under certain conditions”

Bottom line practical implications
  • Hardware (the working memory capacity an individual brings to a specific task) and software (domain-relevant knowledge an individual brings to a specific task) combine to facilitate higher-level cognitive performance. Simply having a larger CPU (MW) is not the answer. Having the most sophisticated software (knowledge base) is insufficient. Both hardware and software make unique contributions to performing at a high level on demanding cognitive tasks. At least in adult populations, both domains need to be assessed when trying to predict/explain a person’s performance.

Thursday, May 19, 2005

Quote to note - Theories and truth

"Professors of every branch of the sciences prefer their own theories to truth: the reason is, that their theories are private property, but truth is common stock"
  • Charles Caleb Colton, Lacon (1849)

Gottfredson on general intelligence (g) - Scientific American

While rummaging around the internet this evening I stumbled on an article in Scientific American about g (general intelligence). Linda Gottfredson presents a nice concise synthesis of the majority view on the existence of g.

As for me....I still do not know if g exists, and if it does, what it represents (although I recently posted some empirical data in support of the "working memory may be g" position). Much better minds than mind have been debating the g vs no-g position since the days of Spearman. I've personally sat in a small room and witnessed John Horn and the late Jack Carroll strongly argue (to put it in mild terms) both positions. Both made convincing points.

Regardless, as a practical manner, g-based IQ scores are indeed powerful predictors, on the average, across a wide variety of domains. However, as reported in a prior post, a number of specific (narrow and/or broad) CHC abilities have been found to provide incremental validity above and beyond g.

MS Windows of the future - a sneak peak review from NY Times

Yes...we all bemoan Microsoft's monopoly with the Windows operating system. A love-hate relationship....one nurtured because we have become addicted to it as an operating-system drug...the most powerful one we all first experimented with (not counting some of the early DOS, CPM, etc. days for those of us who go a ways back with PCs).

The NY Times Circuits section just posted a sneak peek review at the next generation of Windows (nicknamed Longhorn.

Cognitive assessment of deaf and hard of hearing

Yep...this is information stolen from the National Association of School Psychologists general membership listserv. Thanks to Guy McBride whose post on 5-19-05 brought this resource to my attention. Guy...you da man!

Recommendations regarding the cognitive assessment of individuals who are deaf or hard of hearing can be found at the Gallaudent web page.

Thanks again Guy. Stealing posted information is a compliment in the currency of the internet :)

Accessible Reading Assessments workshop

FYI - Registration for NCEO's National Accessible Reading Assessments Topical Clinic (at CCSSO's annual National Conference on Large Scale Assessment) are now being taken.

The descriptive blurb (from the NCEO web page) is reproduced below

  • "How can we ensure that our state reading assessments accurately reflect what ALL students know and are able to do, even those students with disabilities that affect reading? Accessible reading assessments that produce reading scores for all students are an essential part of inclusive accountability systems. Participants will engage in detective work to build understanding of how to best assess students with disabilities that affect reading. The National Accessible Reading Assessment Projects' definition of reading proficiency will serve as a set of clues for participants. With the assistance of NCEO's crack team of detectives (a.k.a. researchers), participants will be asked to think in and outside of the box to begin to unlock these mysteries!"

Large scale assessments & kids with disabilities

I have made it no secret that I provide consultative services (via service contracts) to the University of Minnesota National Center on Education Outcomes (NCEO), which is one of two centers (the other being ETS) dealing with the federally funded National Accessible Reading Assessment Project.

Why?....because I believe passionately in the need to support the efforts of groups advocating on the behalf of "kids on the margins" during nationally-driven (NCLB) educational reform, esp. as it relates to measurement/quantoid issues.

That being said, this is an FYI post.

NCEO has recently made the following information available via their web page. Please check the reports/links if you are interested in the issues related to the inclusion of students with disabilities in large scale accountability assessments. These are important issues.

Tuesday, May 17, 2005

Brain Candy and the Flynn Effect

FYI.

Someone recently directed me to a book review, in the New Yorker, of Malcom Gladwell's Brain Candy: Is Pop Culture Dumbing Us Down or Smartening Us Up? This is a pop review about a pop psychology book that delves into the Flynn Effect.

I must admit that I've not read the book myself.

Monday, May 16, 2005

Stanford Binet 5 (SB5) post-publication resources: 5-20-13

This is an update of a post made a number of years ago...with new information

Stanford Binet 5 Assessment Service Bulletins (info from Riverside Publishing web page)


Sunday, May 15, 2005

Quote to note - Asimov on life and eath

"Life is pleasant. Death is peaceful. It’s the transition that’s troublesome"
  • •Isaac Asimov, “How Easy to See the Future,” Natural History, April 1975

Saturday, May 14, 2005

Quote to note - da Vinci on science and mathematics

"No human investigation can be called real science if it cannot be demonstrated mathematically"
  • Leonardo da Vinci, Treatise on Painting (1651)

Friday, May 13, 2005

Quote to note - purpose of models

"The purpose of models is not to fit the data but to sharpen the questions"
  • •Samuel Karlin, 11th R. A. Fisher Memorial Lecture, Royal Society, April 20, 1983

Thursday, May 12, 2005

Working memory--TheVillage-2 more

Two more working memory abstracts from TheVillage working memory listserv sent to me today.

  • Author Name(s): Nelson Cowan, Emily M. Elliott, J. Scott Saults, Lara D.; Nugent, Pinky Bomb, and Anna Hismjatullina
  • Contact email: CowanN@missouri.edu
  • Title: Rethinking Speed Theories of Cognitive Development: Increasing the Rate of Recall Without Affecting Accuracy
  • Journal: Psychological Science
  • Abstract: Researchers have suggested that developmental improvements in immediate recall stem from increases in the speed of mental processes. However, that inference has depended on evidence from correlation, regression, and structural equation modeling. We provide counterexamples in two experiments in which the speed of spoken recall is manipulated. In one experiment, second-grade children and adults recalled lists of digits more quickly than usual when the lists were presented at a rapid rate of 2 items per second (items/s). In a second experiment, children received lists at a 1 item/s rate but half of them were successfully trained to respond more quickly than usual, and similar to adults' usual rate. Recall accuracy was completely unaffected by either of these response-speed manipulations. Although response rate is a strong marker of an individual's maturational level, it thus does not appear to determine immediate recall. There are important implications for developmental methodology.
*****
  • Author Name(s): Jefferies, E., Frankish, C., Lambon Ralph, M. A.
  • Contact email: beth.jefferies@manchester.ac.uk
  • Title: Lexical and semantic influences on item and order memory in immediate serial recognition: Evidence from a novel task
  • Journal: QJEP (A)
  • Abstract: Previous studies have reported that, in contrast to immediate serial recall, lexical/semantic factors have little effect on immediate serial recognition: this has been taken as evidence that linguistic knowledge contributes to verbal short-term memory in a redintegrative process at recall. Contrary to this view, we found that lexicality, frequency and imageability all influenced matching span. The standard matching span task, requiring changes in item order to be detected, was less susceptible to lexical/semantic factors than a novel task involving the detection of phoneme order and hence item identity changes. Therefore, in both immediate recognition and immediate serial recall, lexical/semantic knowledge makes a greater contribution to item identity as opposed to item order memory. Task sensitivity, and not the absence of overt recall, may have underpinned previous failures to show effects of these variables in immediate recognition. We also compared matching span for pure and unpredictable mixed lists of words and nonwords. Lexicality had a larger impact on immediate recognition for pure as opposed to mixed lists, in line with findings for immediate serial recall. List composition affected the detection of phoneme but not item order changes in matching span; similarly, in recall, mixed lists produce more frequent word phoneme migrations but not migrations of entire items. These results point to strong similarities between immediate serial recall and recognition. Lexical/semantic knowledge may contribute to phonological stability in both tasks.

Working memory virtual home

As a follow-up to my last post, click here to visit the home page for TheVillage, a listserv dedicated to the exchange of scholarly information regarding working memory research.

Working memory listserv - TheVillage

Below is an FYI post I received from TheVillage working memory listserv. Interested IQ blogsters may want to consider joining this list.

The directions I received to join are to send an email to:
  • majordomo@workingmemory.org
With this message in the body of the email:
  • subscribe thevillage
If it doesn't work, other contact information is listed below.

FYI message reposted from TheVillage listserv

Dear colleagues,
  • Below is TheVillage's monthly list of new, in-press papers related to WM received in April. The listings are presented in order of receipt during the month; each listing is separated by 5 asterisks. If you would like a copy of any of the papers listed and abstracted below, please email the contact author off-list.
  • If you are an author who would like to announce your own in-press work, please see the instructions at the bottom of this message.
  • If you have not yet joined the working memory listserve, but would like to, then please write the list owner, John Towse, at: j.towse@lancaster.ac.uk.
Enjoy!
*****
  • Author Name(s): Kevin Dent & Mary M Smyth
  • Contact email: k.dent@bham.ac.uk
  • Title: Capacity limitations and representational shifts in spatial short-term memory capacity
  • Journal: Visual Cognition
  • Abstract: Performance was examined in a task requiring the reconstruction of spatial locations. Previous research suggests that it may be necessary to differentiate between memory for smaller and larger numbers of locations (Postma and DeHaan, 1996), at least when locations are presented simultaneously (Igel and Harvey, 1991). Detailed analyses of the characteristics of performance showed that such a differentiation may also be required for sequential presentation. Furthermore the slope of the function relating each successive response to accuracy was greater with 3 than with 6, 8, or 10 locations which did not differ. Participants also reconstructed the arrays as being more proximal than in fact they were, however sequential presentation eliminated this distortion when there were 3 but not when there were more than 3 locations. These results support the idea that very small numbers of locations are remembered using a specific form of representation, which is unavailable to larger numbers of locations.
*****
  • Author Name(s): Kevin Dent & Mary M Smyth
  • Contact email: k.dent@bham.ac.uk
  • Title: Verbal coding and the storage of form-position associations in visual-spatial short-term memory
  • Journal: Acta Psychologica
  • Abstract: Short-term memory for form-position associations was assessed using an object relocation task. Participants attempted to remember the positions of either 3 or 5 Japanese Kanji characters, presented on a computer monitor. Following a short blank interval, participants were presented with 2 alternative Kanji, only 1 of which was present in the initial stimulus, and the set of locations occupied in the initial stimulus. They attempted to select the correct item and relocate it back to its original position. The proportion of correct item selections showed effects of both articulatory suppression and memory load. In contrast the conditional probability of location given a correct item selection showed an effect of load but no effect of suppression. These results are consistent with the proposal that access to visual memory is aided by verbal recoding, but that there is no verbal contribution to memory for the association between form and position.
*****
  • Author Name(s): Roy Allen, Peter McGeorge, David G. Pearson, Alan Milne.
  • Contact email: roy.allen@abdn.ac.uk
  • Title: Multiple-target tracking: A role for working memory?
  • Journal: Quarterly Journal of Experimental Psychology Section A
  • Abstract: In order to identify the cognitive processes associated with target tracking, a dual task experiment was carried out in which participants undertook a dynamic multiple-object tracking task first alone and then again, concurrently with one of several secondary tasks, in order to investigate the cognitive processes involved. The research suggests that after designated targets within the visual field have attracted preattentive indexes that point to their locations in space, conscious processes, vulnerable to secondary visual and spatial task interference, form deliberate strategies beneficial to the tracking task, before tracking commences. Target tracking itself is realized by central executive processes, which are sensitive to any other cognitive demands. The findings are discussed in the context of integrating dynamic spatial cognition within a working memory framework.
*****
  • Author Name(s): Deschuyteneer, M. & Vandierendonck, A.
  • Contact email: maud.deschuyteneer@UGent.be
  • Title: The role of response selection and input monitoring in solving simple arithmetical products
  • Journal: Memory & Cognition
  • Abstract: Several studies have already shown that the central executive, as conceptualised in the working memory model of Baddeley and Hitch (1974), is important in simple mental arithmetic. Recently, attempts have been made to define more basic processes that underlie the "central executive". In this vein, monitoring, response selection, updating, mental shifting, and inhibition have been proposed as processes capturing executive control. Previous research has shown that secondary tasks which require a choice decision impair the calculation of simple sums, whereas input monitoring was not found to be a sufficient condition to impair the calculation of the sums (Deschuyteneer & Vandierendonck, in press). In the present paper we report data on the role of input monitoring and response selection in solving simple arithmetical products. In four experiments subjects solved one-digit products (e.g., 5 x 7) in a single-task as well as in dual-task conditions. Just as for solving simple sums, the results show a strong involvement of response selection in calculating simple products, while input monitoring does not seem to impair the calculation of such products. These findings give additional evidence that response selection may be one of the processes needed for solving simple mental arithmetic problems.
*****
AUTHOR INSTRUCTIONS

Once a month, a list of in-press papers and their abstracts will be distributed through the WM Village listserve. If you are an author who would like your paper listed, then please contact me off-list (at mjkane@uncg.edu), as I will be collating the information.

In the subject line of the email, please write:
  • "In Press, [Month, Year], [1st Author Name]"
  • - where month and year refer to the date your email is sent. This will assist with the job of sorting and tracking messages.
  • The email itself should contain the referencing information. The format we'd prefer is given below (using a standard template will help subscribers peruse the listings for the information they desire).
  • Author Name(s):
  • Contact email:
  • Title:
  • Journal:
  • Abstract:

The value of this preprint information list depends on subscribers sharing preprint information, so we do strongly encourage you to let the list know if you have relevant articles accepted for publication.

A quote to note: Committees

"A committee is a cul-de-sac down which ideas are lured and then quietly strangled"
  • Sir Barnett Cocks, in New Scientist, 1973

Wisconsin Card Sorting Test and ADHD

FYI meta-analytic review of sensitivity of Wisconsin Card Sorting Test for ADHD.

Bottom line practical conclusion - the WCST, much like other measures of cognitive efficiency (e.g., Gs measures), might best be viewed as a thermometer of the efficiency of a person's information processing system. If the scores are poor (the temperature is high), it tells you that something is amiss in the IP processing system, but like a thermometer, it doesn't posses the diagnostic specificity to tell you what the problem is.


Romine, C. B., Lee, D., Wolfe, M. E., Homack, S., George, C., & Riccio, C. A. (2004). Wisconsin Card Sorting Test with children: a meta-analytic study of sensitivity and specificity. Archives of Clinical Neuropsychology, 19(8), 1027-1041.

Abstract
  • More and more frequently the presence of executive function deficits appears in the research literature in conjunction with disabilities that affect children. Research has been most directed at the extent to which executive function deficits may be implicated in specific disorders such as attention deficit hyperactivity disorder (ADHD); however, deficits in executive function have been found to be typical of developmental disorders in general. The focus of this paper is to examine the extent to which one frequently used measure of executive function, theWisconsin Card Sorting Test (WCST), demonstrates sensitivity and specificity for the identification of those executive function deficits associated with ADHD as well as its use with other developmental disorders through meta-analytic methods. Evidence of sensitivity of the WCST to dysfunction of the central nervous system is reviewed. Effect sizes calculated for all studies compared groups of children on differing variables of the WCST. The results of this meta-analysis suggest that across all of the studies, individuals with ADHD fairly consistently exhibit poorer performance as compared to individuals without clinical diagnoses on the WCST as measured by Percent Correct, Number of Categories, Total Errors, and Perseverative Errors. Notably, other various clinical groups performed more poorly than the ADHD groups in a number of studies. Thus, while impaired performance on the WCST may be indicative of an underlying neurological disorder, most likely related to frontal lobe function, poor performance is not sufficient for a diagnosis of ADHD. Implications for further research are presented.

Wednesday, May 11, 2005

The intelligence testing world is being "flattened" - The World is Flat

There is little doubt (IMHO) that the buzz surrounding one of the current top best selling non-fiction books, "The World is flat: A brief history of the twenty-first century" by Thomas L. Friedman, is appropriate.

Im only 70+ pages in to the book and I must say that Friedman's thesis regarding the impact of globalization on the world is very interesting. His thoughts on the "ten forces that flattened the world" are very thought provoking. In his discussion of Flattener # 3 (Work Flow Software), he quotes (in the context of the impact of the development of standard internet protocols) Joel Crawley, the head of IBM's strategic planning unit, as stating that:
  • "Standards don't eliminate innovation, they just allow you to focus it. They allow you to focus on where the real value lies, which is usually everything you can add above and beyond the standard" (p. 76).
Of course, being of unstill mind, I extrapolated the definition of standards to a much broader level and have tentatively applied it to the increasing recognition (and adoption) of the CHC framework as the leading psychometric taxonomy from which to conceptualize, measure, and interpret intelligence tests (see my prior CHC bandwagon proclamation post). That is, we now have a common standard theoretical framework and nomenclature around which to focus applied intelligence testing and development work.

Thus, if my late night extrapolation/generalization has any merit, and if I'm correct in my conclusion/prediction that most test developers are (or will) jump on some variation of the CHC bandwagon in the near future (if not already), we in the field of applied psychometric intelligence testing now have the CHC standard.

If true, the logical prediction follows that as more-and-more intelligence batteries adopt a CHC framework, the real innovation in intelligence testing will come from those who add "value" and those who produce instruments that "add above and beyond the standard."

You heard it first here...the recognition of the CHC cognitive ability framework may be a key world flattening event in the world of intelligence test development. Value added instruments which provide more than CHC construct valid measures will be what folks will be asking for and what test developers need to focus on.

Tuesday, May 10, 2005

Fourth Spearman Conference Presentations available

FYI - The Fourth Spearman Conference (“Diagnostics for Education: Theory, Measurement, Applications,”) was held in Philadelphia, Pennsylvania, USA on October 20-23, 2004.

By clicking here blogsters can visit a page and click and view/download the presentations "as is." Below is a list of the presenations for which material has been posted.
  • Aldrich, Clark, Educational Simulations - Simulation; Games; and Pedagogy
  • Flanagan, Dawn P., The Role of IQ Tests in LD Identification
  • Graesser, Ozuru, Rowe and Floyd, Enhancing the Landscape and Quality of Multiple Choice Questions
  • Graf, Steffen, Peterson, Saldivia and Wang, Designing and Revising Quantitative Item Models
  • Hunt, Earl, Patterns of Thought
  • Matthews, Gerald, Picking a Bone with Spearman: A Contrary View of Mental Energy and Attention
  • Misley, Robert J., Cognitive Diagnosis as Evidentiary Argument
  • Pennebaker, James W., What Our Words Say About Us
  • Sabatini, John, Critical Issues in Aligning Psychometrics with Current Reading Models
  • Stout, William, The Reparameterized Unified Model Skills Diagnostic System
  • Swanson, H. Lee, Memory and Learning Disabilities: Historical Perspective and Current Status
  • van der Linden, Wim J., Statistical Models for Diagnosis: A Discussion
  • VanLehn, Kurt, The Andes Intelligent Tutoring System: Lessons Learned about Assessment
  • von Davier, Mattias and Yamamoto, Kentaro, A Class of Models for Cognitive Diagnosis
  • Wiliam, Dylan, From Diagnosis to Action: Toward Instructionally Tractable Assessment
  • Wilson, Mark, Cognitive Diagnosis using Explanatory Item Response Models
  • Wittmann, Werner W., Brunswik-Symmetry: A Key Concept for Successful Assessment in Education and Elsewhere

Berlin BIS model of intelligence--material to review

I previously suggested that American intelligence scholars need to pay more attention to the Berlin BIS facted model of intelligence and how it can interface with CHC theory (see March 28, 2005 post). In a response to my recent post ("g, working memory, specific CHC abilities and achievement"), Werner Wittmann extends this recommendation and, more importantly, directs IQ blogsters to a number of on-line papers and PowerPoint presentations.

For those who did not notice Werner's comment, below are the links he provided
  • 2004 PowerPoint presented APS in Atlanta - deals with working memory, intelligence and Phil Ackerman's PPIK-theory under a Brunswik symmetry perspective. [Editorial note - I'm a big fan of Ackerman's PPIK theory, particularly his work on aptitude-trait complexes, which follow's in the footsteps of Richard Snow's work on aptitudes.]
Thanks to Werner for being an active reader and FYI contributor to this blog.

Sunday, May 08, 2005

g, working memory, specific CHC abilities and achievement

[Another set of “hidden on my hard disk analyses" that now see the light of day.]

Much has been written, both in the theoretical and applied intelligence theory/assessment literature, about
  • The definition of g (general intelligence),
  • Whether g exists,
  • The importance of g in the prediction of a wide variety of outcomes, and
  • If g exists, the importance of specific broad and narrow stratum abilities above and beyond g in the prediction of outcomes

By way of background, I have previously summarized the primary issues in “Betwixt, Behind, and Beyond g.” (Editorial note - please read this prior information before going further in this post.) In those prior writings I presented SEM analyses (of the WJ III norm data) that supported the hypothesis that working memory (actually, cognitive efficiency as defined by Gs and working memory) has a strong causal relationship to g (i.e., if you believe g is a workable and sound construct.) I also summarized research suggesting that a number of broad and narrow CHC abilities are significant above and beyond the influence of g.

So....what is this topic of this current post? Simple. To combine the working memory--> g information processing causal hypothesized model with the more traditional psychometric g+specific abilities research model.

Using the same norm samples as described in Betwixt, Behind, and Beyond g, I ran SEM causal models that defined g via the broad CHC latent abilities of Gv, Ga, Glr, Gf, and Gc. Additionally, adhering to a general information processing causal CHC model (click here to see prior relevant post), Gs and memory span (MS) are specified to be casually related to working memory, and working memory is in turn causally related to g.

The new twist is the specification of a causal path from g (as defined above) to achievement variables (letter-word identification; word attack) plus the inclusion of any significant paths from the cognitive efficiency variables (Gs and working memory) and/or the CHC abilities (that define the measurement model for g) to the dependent achievement latent variables.

All plausible models are presented and can be viewed by clicking here (Editorial note – for each age group presented, the models reported are all equally plausible. The model fit statistics did not identify any of the models in the two age groups as being better fits than the others).

Enjoy. I’d like to encourage the readers of this blog to offer interpretations and hypothesis for the various models. My only comment is that if you believe in g, the hypothesis that g is strongly related to working memory is supported (actually, to the broader notion of cognitive efficiency). Furthermore, some specific CHC abilities are again found to provide potentially important insights above and beyond the effect of g within a CHC-based information processing model.

For all who immediately shout “but what about parsimony?....what about Occam’s Razor?”….I defer to Stankov, Boyle & Cattell, R. (1995; Models and paradigms in personality and intelligence research. In Saklofske, D. & Zeidner, M. (Eds.) International Handbook of Personality and Intelligence. New York: Plenum Press), who state, within the context of research on human intelligence, that:

  • “While we acknowledge the principle of parsimony and endorse it whenever applicable, the evidence points to relative complexity rather than simplicity. Insistence on parsimony at all costs can lead to bad science. “

Thursday, May 05, 2005

"Current Challenges in Educational Testing" conference

FYI - "pass along" post from NCME listserv

ACT and the Center for Advanced Studies in Measurement and Assessment (CASMA) of the University of Iowa are sponsoring a one-day conference on Saturday, November 5, 2005, at ACT's conference facilities in Iowa City.

A Keynote Address will be given by Laura Schwalm, Superintendent, Garden Grove Unified School District, Orange County, CA, winner of the 2004 Broad Prize for Urban Education.

Sessions and speakers include:
  • (1) Welcome (Richard Ferguson, CEO, ACT; David Skorton, President, University of Iowa)
  • (2) K-12 Testing and NCLB (Robert Linn, University of Colorado and CRESST)
  • (3) Computerized Grading of Essays (eRater; KAT; Vantage Learning)
  • (4) NAEP 12th Grade Testing (Sharif Shakrani, National Assessment Governing Board; Ted Stilwill, former Director of Iowa Department of Education)
  • (5) Non-cognitive Assessment in College Admissions and the Workforce (Robert Sternberg, Yale; Michael Campion, Purdue; Paul Sackett, University of Minnesota).
  • (6) Panel Discussion on a "Hot Topic"
  • (7) Reception

The registration fee is $50 (includes breakfast, lunch, and reception). The registration deadline is October 7, 2005. To register starting June 6, and to obtain travel and hotel information, go to http://www.act.org/casma.

Contributing sponsors include the College Board, CTB/McGraw Hill, Harcourt Assessment, Measured Progress, National Evaluation Systems, Riverside Publishing Company, and Vantage Learning.

Wednesday, May 04, 2005

Conative abilities and "aptitude"

I've long been a believer that educational/school psychologists need to pay more attention to non-cognitive variables (conative abilities) when discussing a student's aptitude for school learning. A major barrier to attending to these abilities has been a shortage of empirically based conceptual/theoretical models and practical measures.

Although there is still a clear void in the area of practical measurement tools, significant progress has been made the past 10 years in the development of theoretical models of important conative abilities. Two books are suggested for those who want to learn more.

First, and probably the most difficult read, is Remaking the Concept of Aptitude: Extending the Legacy of Richard Snow. This posthumously published book, in honor of Richard Snow, provides (in one place) a summary of the "state of the art" of Snow's work on aptitude...which, in simple terms, requires an integration of cognitive and conative abilities into a coherent theoretical framework. The reader is forewarned---this book shows the signs of being authored by committee. At times it is not coherent, material is not well linked, and it becomes difficult to see the forest from the trees.

The recently published Handbook of Competence and Motivation (Elliot & Dweck, 2005) represents a monumental integration of the emirical/conceptual progress made during the past few decades regarding to the most salient conative abilities (e.g., achievement goal orientation, self-theories, self-regulatory learning processes, etc.). Having recently spent the better part of 2 years searching for contemporary research in the diverse domain of conative abilities, I believe that this edited volume provides the best contemporary integration of this vast literature. An expensive book (yes, I know..."tell me something new"), but probably the best single integrative work I've seen to date.

Hopefully these books will stimulate the development of applied measures that can narrow the theory-practice gap.

Tuesday, May 03, 2005

FYI - Used/rare books in psychology-Powells.com

FYI tidbit

If anyone likes to purchase used and/or rare books (like I do), I would suggest subscribing to a service from one of the largest used books stores I've ever seen - Powell's Books in Portland, Oregon. Book lovers could get lost in this store for days.

I receive via email (every day) a listing of new books posted in three areas - psychology, mathematics, and rare books. I often find a book I want for my collection and then simply order. The service has always been great.

If blogsters want to take a peak, click here and you will be taken to the list of currently featured psychology books. From that page you can navigate around to other topics and find the page where you can subscribe. E-mailed book reviews is another feature I enjoy

Cognitive Efficiency and achievement

The following article, which is "in press" in Intelligence, provides interesting information regarding the potential importance of measures of cognitive efficiency in predicting/explaining school achievement. The abstract is printed below along with a few highlights from the study.

Luo, D., Thompson, L. A., & Detterman, D. K. (2005). The criterion validity of tasks of basic cognitive processes. Intelligence, In Press, Corrected Proof.

Abstract
  • The present study evaluated the criterion validity of the aggregated tasks of basic cognitive processes (TBCP). In age groups from 6 to 19 of the Woodcock-Johnson III Cognitive Abilities and Achievement Tests normative sample, the aggregated TBCP, i.e., the processing speed and working memory clusters, correlate with measures of scholastic achievement as strongly as the conventional indexes of crystallized intelligence and fluid intelligence. These basic processing aggregates also mediate almost exhaustively the correlations between measures of fluid intelligence and achievement, and appear to explain substantially more of the achievement measures than the fluid ability index. The results from the Western Reserve Twin Project sample using TBCP with more rigorous experimental paradigms were similar, suggesting that it may be practically feasible to adopt TBCP with experimental paradigms into the psychometric testing tradition. Results based on the latent factors in structural equation models largely confirmed the findings based on the observed aggregates and composites.
Summary/Comments
  • The measures of TBCP in the present study were taken from two data sources, Woodcock-Johnson III Cognitive Abilities and Achievement Tests (W-J III; Woodcock et al., 2001a, 2001c; Woodcock, McGrew, & Mather, 2001b) normative data and the Western Reserve Twin Project (WRTP) data. The WJ III results will be the focus of this post.
  • Luo et al. examined (via multiple regression and SEM) the extent to which measures of what the WJ III authors (myself included – see home page conflict of interest disclosure) call “cognitive efficiency” (CE - Gs and Gsm tests/clusters) add to the prediction of total achievement, above and beyond Gc and Gf.
  • These researchers found that CE measures/abilities demonstrated substantial correlations with scholastic performance (WJ III Total Achievement). The CE-Ach correlations were similar to correlations between conventional test composites and scholastic performance. These results suggested that measures of CE provide incremental predictive validity beyond Gc. Collectively, CE+Gc accounted for approximately about 60% of the variability in achievement when observed measures were analyzed (multiple regression) and up to 70% or more of the variance when the SEM latent traits were analyzed. The authors concluded that these levels of prediction were “remarkable”
  • Gf measures did NOT contribute significantly to the prediction of achievement beyond that already accounted for by the CE measures. [Editorial note – my prior research with the WJ III and WJ-R suggests this result may be due to the authors using “total achievement” as their criterion. My published research has consistently found that Gf is an important predictor/causal variable in the domain of mathematics].
  • A potential explanation for the power of the CE measures/variables has previously been published and posted to the web (click here).
  • The current results, IMHO, fit nicely within CHC-based information model frameworks that have been suggested. Simplified schematic models (based on the work of Woodcock) can be viewed by clicking here.

Friday, April 29, 2005

Journal of Intelligence and ISIR - pre-pub "in press" feature

For those serious about staying current with contemporary intelligence research, the journal Intelligence is a must subscription. I would urge blog readers to visit the ISIR (International Society for Intelligence Research) web page to consider joining this organization. Membership provides members with a subscription to the journal (members can also download pdf copies of the articles from the web site).

Another nice feature is that "corrected proof" pdf copies of "in press" articles are made available with regularity. This feature allows members to keep abreast of contemporary developments ASAP. I think this feature is a model that should be followed by other journals.

Below is a select number of abstracts from recently published "in press-corrected proofs" that were available for download.


Ashton, M. C., & Lee, K. (2005). Problems with the method of correlated vectors. Intelligence, In Press, Corrected Proof.


  • The method of correlated vectors has been used widely to identify variables that are associated with general intelligence (g). Briefly, this method involves finding the correlation between the vector of intelligence subtests' g-loadings and the vector of those subtests' correlations with the variable in question. We describe two major problems with this method: first, associations of a variable with non-g sources of variance can produce a vector correlation of zero even when the variable is strongly associated with g; second, the g-loadings of subtests are highly sensitive to the nature of the other subtests in a battery, and a biased sample of subtests can cause a spurious correlation between the vectors.

Arendasy, M., & Sommer, M. (2005). The effect of different types of perceptual manipulations on the dimensionality of automatically generated figural matrices. Intelligence, In Press, Corrected Proof.

  • Two pilot studies (n1 = 155, n2 = 451) are presented in this article, which were carried out within the development of an item generator for the automatic generation of figural matrices items. The focus of the presented studies was to compare two types of item designs with regard to the effect of variations of the property "perceptual organization" on the psychometric properties and concurrent validity of figural matrices. The main results not only indicate a comparably high concurrent validity of the automatically generated figural matrices with regard to Raven matrices but also point to interesting differential effects of two kinds of implementations of perceptual organization. The conclusion of the two studies is that a more thorough understanding of the component processes of inductive reasoning and especially the impact of various item features on their difficulty and psychometric properties must be obtained.

Barber, N. (2005). Educational and ecological correlates of IQ: A cross-national investigation. Intelligence, In Press, Corrected Proof.

  • The new paradigm of evolutionary social science suggests that humans adjust rapidly to changing economic conditions, including cognitive changes in response to the economic significance of education. This research tested the predictions that cross-national differences in IQ scores would be positively correlated with education and negatively correlated with an agricultural way of life. Regression analysis found that much of the variance in IQ scores of 81 countries (derived from [Lynn, R., & Vanhanen, T. (2002). IQ and the wealth of nations. Westport, CT: Praeger]) was explained by enrollment in secondary education, illiteracy rates, and by the proportion of agricultural workers. Cross-national IQ scores were also related to low birth weights. These effects remained with national wealth, infant mortality, and geographic continent controlled (exception secondary education) and were largely due to variation within continents. Cross-national differences in IQ scores thus suggest that increasing cognitive demands in developed countries promote an adaptive increase in cognitive ability.


Johnson, W., & Bouchard, Jr. T. J. (. The structure of human intelligence: It is verbal, perceptual, and image rotation (VPR), not fluid and crystallized. Intelligence, In Press, Corrected Proof.

  • In a heterogeneous sample of 436 adult individuals who completed 42 mental ability tests, we evaluated the relative statistical performance of three major psychometric models of human intelligence--the Cattell-Horn fluid-crystallized model, Vernon's verbal-perceptual model, and Carroll's three-strata model. The verbal-perceptual model fit significantly better than the other two. We improved it by adding memory and higher-order image rotation factors. The results provide evidence for a four-stratum model with a g factor and three third-stratum factors. The model is consistent with the idea of coordination of function across brain regions and with the known importance of brain laterality in intellectual performance. We argue that this model is theoretically superior to the fluid-crystallized model and highlight the importance of image rotation in human intellectual function.

Johnson, W., & Bouchard, Jr. T. J. (2005). Constructive replication of the visual-perceptual-image rotation model in Thurstone's (1941) battery of 60 tests of mental ability. Intelligence, In Press, Corrected Proof.

  • We recently evaluated the relative statistical performance of the Cattell-Horn fluid-crystallized model and the Vernon verbal-perceptual model of the structure of human intelligence in a sample of 436 adults heterogeneous for age, place of origin, and educational background who completed 42 separate tests of mental ability from three test batteries. We concluded that the Vernon model's performance was substantively superior but could be significantly improved. In so doing, we proposed a four-stratum model with a g factor at the top of the hierarchy and three factors at the third stratum. We termed this the Verbal-Perceptual-Image Rotation (VPR) model. In this study, we constructively replicated the model comparisons and development of the VPR model using the data matrix published by Thurstone and Thurstone (1941) [Thurstone, L. L., & Thurstone T. G. (1941). Factorial studies of intelligence. Chicago: University of Chicago Press]. The data matrix was generated by scores of 710 Chicago eighth graders on 60 tests of mental ability.

Kempel, P., Gohlke, B., Klempau, J., Zinsberger, P., Reuter, M., & Hennig, J. (2005). Second-to-fourth digit length, testosterone and spatial ability. Intelligence, In Press, Corrected Proof.

  • Based on stimulating findings suggesting that prenatal levels of steroids may influence cognitive functions, a study with N=40 healthy volunteers of both sexes was conducted. Prenatal levels of testosterone (T) were estimated by use of the second-to-fourth digit ratio (2D:4D) which is supposed to be controlled by the same genes involved in maturation of gonadal tissue and therefore may reflect the level of prenatal T. Moreover, activational effects of T were investigated by measuring T levels in saliva directly. Subjects completed several subtests of intelligence batteries for verbal, numerical and spatial abilities. Levels of T were not related to any of the cognitive functions. The 2D:4D was lower in males as compared to females. Males outperform females on spatial ability. Moreover, females with low 2D:4D performed better on cognitive tests measuring spatial as well as numerical ability as compared to females with high 2D:4D. Results are discussed with respect to the assumed role of prenatal and present levels of T in brain development and cognitive functioning.


Luo, D., Thompson, L. A., & Detterman, D. K. (2005). The criterion validity of tasks of basic cognitive processes. Intelligence, In Press, Corrected Proof.

  • The present study evaluated the criterion validity of the aggregated tasks of basic cognitive processes (TBCP). In age groups from 6 to 19 of the Woodcock-Johnson III Cognitive Abilities and Achievement Tests normative sample, the aggregated TBCP, i.e., the processing speed and working memory clusters, correlate with measures of scholastic achievement as strongly as the conventional indexes of crystallized intelligence and fluid intelligence. These basic processing aggregates also mediate almost exhaustively the correlations between measures of fluid intelligence and achievement, and appear to explain substantially more of the achievement measures than the fluid ability index. The results from the Western Reserve Twin Project sample using TBCP with more rigorous experimental paradigms were similar, suggesting that it may be practically feasible to adopt TBCP with experimental paradigms into the psychometric testing tradition. Results based on the latent factors in structural equation models largely confirmed the findings based on the observed aggregates and composites.


McDaniel, M. A. (2005). Big-brained people are smarter: A meta-analysis of the relationship between in vivo brain volume and intelligence. Intelligence, In Press, Corrected Proof.

  • The relationship between brain volume and intelligence has been a topic of a scientific debate since at least the 1830s. To address the debate, a meta-analysis of the relationship between in vivo brain volume and intelligence was conducted. Based on 37 samples across 1530 people, the population correlation was estimated at 0.33. The correlation is higher for females than males. It is also higher for adults than children. For all age and sex groups, it is clear that brain volume is positively correlated with intelligence.

Reed, T. E., Vernon, P. A., & Johnson, A. M. (2005). Confirmation of correlation between brain nerve conduction velocity and intelligence level in normal adults. Intelligence, In Press, Corrected Proof.

  • In 1992, Reed and Jensen [Intelligence 16 (1992) 259-272] reported a positive correlation (.26; p=.002; .37 after correcting for restricted intelligence range) between a brain nerve conduction velocity (NCV) and intelligence level in 147 normal male students. In the first follow-up of their study, we report on a study using similar NCV methodologies, but testing both male and female students and using more extensive measures of cognitive abilities. One-hundred eighty-six males and 201 females, aged 18-25 years, were tested in three different NCV conditions and with nine cognitive tests, including Raven Progressive Matrices as used by Reed and Jensen. None of the 27 independent correlations in either the males or in the females are significant at Bonferroni-corrected probability levels, but 25 of 27 correlations in males and 20 of 27 correlations in females have positive signs. The exact binomial probabilities for these results are 5.6X10-6 and .002, respectively. We discuss possible reasons for the differences between the results of Reed and Jensen and our results. We also find that males have four percent faster NCVs than females with each of the three test conditions, probably due to their faster increase of white matter in the brain during adolescence.

Learning and the Brain conference

To regular blogsters...I'm back. I just returned (early) from the Learning and the Brain-Rewiring the Brain: Using Brain Research to Enhance Learning, Treatment and Teaching conference in in Cambridge, MA. This was my first time attending and presenting at this conference. I must say that my eyes were opened regarding the state-of-the art of research into neuroplasticity and learning/interventions. I think I shall keep this annual conference on my list of conferences to consider attending in the future.

Revised Code of Fair Testing Practices in Education

FYI - the Joint Committe on Testing Practices (JTCP) has released a Revised Code of Fair Testing Practices in Education. The code (in pdf format) can downloaded by visiting the JTCP page and clicking on the appropriate link.

As described on the JTCP page:


  • The Code of Fair Testing Practices in Education was initially developed by JCTP in 1988 as a statement of the primary obligations that professionals who develop or use educational tests have toward test takers. The Code has been revised to be consistent with the 1999 Standards for Educational and Psychological Testing. The Code provides guidance to professionals who develop or use educational tests.

Monday, April 25, 2005

Blogs catch mainstream medias attention-again.

Blogs are catching the attention of the mainstream media.

Recently (May 2, 2005), Business Week magazine saw fit to feature, as their cover topic, a series of articles on the importance of blogs in contemporary business. If you are still wondering about the potential impact of blogs, and whether you need "get up to speed" regarding all the hype regarding blogs, blogsters, the blogsphere, etc., these articles might be worth a peak.

John Carroll "On learning a piece on the piano" - A historical note from Jack's daughter

As I mentioned in a prior post, I was fortunate to be the last scholar to visit and work with John "Jack" Carroll during the last few months of his retirement in Fairbanks, Alaska. At the time he was living with his daughter (Melissa "Mimi" Chapin) , her husband, and their son.

During this visit I had the serendipitous opportunity to video Jack playing the piano one evening. Unfortunately, as I mention below in an editorial note, it was my first clumsy attempt at using the video feature on my new camera. I only captured a few minutes of his performance, and the picture quality was poor. But, I did capture a precious piece of history for those interested in the history of prominent psychologists.

Below is a post I'm making on behalf of Jack's daughter regarding this piano performance and subsequent private notes ( "On learning a piece on the piano") she discovered while gonig through his boxes of private papers.


Note from Jack Carroll's daughter (Mimi Chapin)


I was pleased to see that your blog readers had asked for more anecdotes about my father, In connection with your visit to Fairbanks, they may be interested in the .mpg of my father playing the piano that you made while there. He listened to it later and commented that it wasn't his best performance. Nonetheless, for me that evening when he played for you was poignant.

During my childhood, my father used to sit down at the piano every evening and play for relaxation, and for joy, usually after a concentrated stint of work, before going to bed. I hadn't heard him do this for years. I think when he played for you it was his way of expressing some of the relief he felt that night that he had passed on what he could of his programs and methods.

Perhaps your blog readers would also be interested in reading some notes my father had made about learning an Etude by Scriabin which I found in his file cabinet labeled "biography". I am fairly sure that the piece he was playing in your excerpt was an Etude by Scriabin, though I haven't found the music to confirm this. Perhaps the piece to which he alludes in the notes is the same piece we have recorded!! Perhaps someone with a keener ear than mine can follow the steps he took to identify the piece.

------------------------------------------------------
[Editorial Note to readers from K. McGrew. By clicking here, blogsters can visit a web page where you can find and view (if you have a proper media player) the mpg file of J. Carroll playing this piano piece on 5-25-03, approximately one month prior to his passing away. Unfortunately, the video is of poor quality as it was the first time I had ever used the video function on a new camera. I only wish now that I had learned the video feature before I had this serendipitous opportunity to video Jack Carroll at the piano.

Can anyone confirm Mimi's belief, as stated above, that the piece he was playing was indeed Etude by Scriabin?]
------------------------------------------------------

The notes start with a quote from Arthur Jensen about the nature of musical memory. My father describes becoming fascinated with an Etude by Scriabin, identifying it and marvelling at its composition. Then he starts to analyze the process by which it is learned, breaking it down into number of key strikings per second.

At the convention I attended with my father, I remember Arthur Jensen's description of experiments on intelligence measuring the speed of striking of keys....I wonder where my father was going with these notes. What is particularly interesting to me is the way these notes reveal the essentially romantic cast of my father's mind, which he then set about to codify and analyze.

I'll include the entire text of these notes below and have sent you a .pdf file of the original (blogsters can click here to view/download these type-written pieces of history. )

12/21/73 – the first day of winter, 1973 - notes by John "Jack" Carroll

Jensen, A.R. Social class and verbal learning. In M. Deutsch, I. Katz and A. R. Jensen. Social class, race and psychological development. NY Holt, Rinehart & Winston, 1968.

p. 121. , The pianist who has memorized a piece of music later finds he cannot recall the notes unless he sits at the piano and begins to play. His “memory” of the composition seems to lie more in his motor behavior than in any symbolic representation of the music; he can play the piece at the piano even though at his desk he would be quite unable to write out the score. And if he makes a mistake while playing, he cannot easily spot and start again at just any point. He usually has to go back to some “beginning” point in the music and continue from there.

On Learning a Piece on the Piano

It is called an Étude, and that word is to be taken literally in this case: a study, something to be studied. But it is not only that—it is something that, when faithfully studied, practiced and perfected, can become what it was intended to be—a work of art, that, unlike a painting can be realized only its performance.

A conjunction of circumstances nudged me into testing my pianistic competence against this formidable piece—composed in about his twenty-second year by Alexander Scriabin. Fashions in music came and go, but it happens that in the last several years the music of Scriabin has once more come into favor and popularity—at least among that small minority of people whose ears are best attuned to what is known as “classical” music. New recordings of Scriabin’ piano works are being played almost daily on the classical radio station that I like to listen to, and it was on one of those broadcasts that I heard a recording of the Étude in G sharp minor, Opus 8, No. 9.

I was listening only rather casually, that afternoon, to the usual mélange of selections—ranging from Bach to Stravinsky—that had been programmed to occupy the 55 minutes between news broadcasts. The announcer’s mention of the name of Scriabin alerted me, however, to the possibility that I might be about to hear a rendition of one of the pieces in the collection of Scriabin's Preludes and Etudes that I had must acquired by mail-order from a well-know reprint publishing house. From the announcement I was able to apprehend the fact that the piece about to be aired was an Etude, but I was not listening carefully enough to catch the opus number or the key. A few seconds into the piece, however, I could tell that it had be in G sharp Minor, and a rapid scanning of the table of contents of the collection, which I had ready at hand, allowed me to know that it could only be Opus 8, No. 9, for there is only one Scriabin Etude in that key. Turning rapidly to the page where the score began, I was then able to have the rare double experience of listening to a new piece of music while at the same time watching its printed representation unfold.

Perhaps it is trite to say the music “cast its spell” on me, but that is the only phrase that I can use to describe what was happening. Whether it was the artistry of the pianist, the eerie quality of the music itself, or the particular mood I myself happened to be in—I do not know: something caused the piece to make a strong impression on my mind, such that I was to hear it long after the music came to an end. I marveled at the haunting yet simple character of the main theme, the rich harmonies that accompanied it, the subtle modulations through which it coursed, the calm assurance of the second theme, the passion with which the return to the main theme was gradually approached, and the crashing sound of the final section—sounds which in the end gave way to a faint, almost whimpering, desperate flight of rapid octaves terminated by a somber, sustained minor chord played twice, slowly. Throughout, I was impressed with the technical skill, on the part of the pianist, that had to be invoked to perform the piece in any reasonable way at all. It was truly an Etude, in the best tradition of Chopin and all who have essayed this genre.

Even as an amateur pianist, or perhaps precisely because I am an amateur pianist, I am challenged when I learn of a piece of music that offers stubborn technical difficulties but at the same time promises a rich artistic reward to anyone who can overcome those difficulties. This Etude by Scriabin struck me immediately as falling in that category. I wanted to learn to perform it, as well as I could.

Perhaps it is not the most difficult piece for the piano ever composed (at this level, I don’t know how difficulty can really be gauged), but it is difficult enough. Containing 103 measures, its performance requires 3665 separate strikings of piano keys in a space of about 220 seconds (if it is to be played at the slower range of the metronome marking of a quarter note = 120-136 proposed by the composer, allowing for a somewhat slower speed in the middle section), or about 16-17 strikings per second. (The rate averages actually about 18-19 strokes per second in the first and last sections, but about 12 per second in the short, slower, middle section.) The 3665 separate strikings are not like completely random events (as they might be in some contemporary music); there is much pattern and redundancy in them. Many notes are duplicated at the octave, or even quadrupled at further octaves; phrases follow systematic patterns, and segments of them may be repeated with no variation in different parts of the piece, groups of notes played simultaneously usually constitute harmonies which are familiar to the musician.

Thursday, April 21, 2005

Clarification of value of IQ tests in education: Response to the Reid Lyon DC gang

It has come to my attention that a report I previously directed blog readers to (Expectations for Students with Cognitive Disabilities: Is the Cup Half Empty or Half Full? Can the Cup Flow Over?) (IQ Scores, Forest Gump & NCLB: Run Forrest..Run), has made it to high places....which is not always good....especially when these places are awash in a sea of politics.

Briefly, in the recent announcement from the U.S. Department of Education that there should be an increase in the percent of students with disabilities who can be excluded from state NCLB-related assessments, (Raising Achievement: Alternative Assessment for Students with Disabilities), yours truly (and, indirectly, my associate Jeff Evans), is cited in support of, what can only be called, and anti-intelligence testing position in special education. As a coauthor of the Woodcock-Johnson III battery, which includes a comprehensive CHC-based intelligence battery, the attribution of an anti-IQ position to me may strike readers as bizarre, and, if believed, possibly suggestive of myself experiencing a break from reality.

Below is the select quote (emphasis added by me) from the ED.gov web URL cited above:
  • "Research also supports the idea that IQ does not dictate achievement and, thus, cannot be used as a predictor. Kevin McGrew of the Institute for Applied Psychometrics notes that for most children with below average IQ scores, it is not possible to predict expected achievement with much accuracy. Lower-than-average IQ does not automatically translate into lower achievement or less ability to learn reading, language arts, mathematics, or other subjects. Other important variables affecting achievement appear to be interpersonal skills, motivation, engagement, and study skills, all of which can be positively influenced by high standards and expectations. Unfortunately, students are too often given a curriculum that is driven by educators' expectations of their students (based in part on a misunderstanding of IQ).
First, the later half of this statement is 100% true. As I've written elsewhere, non-cognitive factors (e.g., conative variables - motivation, self-regulated learning strategies, social and interpersonal abilities, etc.) are often ignored when making statements about a student's "aptitude" for school learning. I'm a firm believer in the Richard Snow approach to defining school aptitude as a combination of both cognitive and conative abilities. Nowhere have I ever written that valid measures of theory-based intelligence tests are not useful or predictive. In fact, I've authored/coauthored four different books on the interpretation of intelligence batteries. In the report drawn upon, I simply describe the normal curve of achievement scores that surround any specific IQ score, and this variability makes it difficult to predict, with precise accuracy, the exact level of expected current or future achievement for a specific student.

What I did write, and which is misconstrued in the ED.gov announcement, is the honest truth about our best measures of cognitive abilities. Namely, they are fallible predictors. Intelligence tests do not, nor have they ever, nor will they likely ever, account for school achievement beyond a threshold of approximately 40-60% of academic achievement variance....which is a hell of a lot in the field of psychology!

Cognitive measures, as per all respected models of school learning (e.g., Carroll's Model of School Learning; Walberg's Model of Educational Productivity), all include student characteristics, and cognitive characteristics in particular, as important contributing/predictive variables in explaining school learning. The point made in the Forrest Gump report is that even today's best available intelligence batteries are fallible predictors that do not allow educators to predict, beyond a reasonable doubt, specific expected levels of current or future achievement without a known degree of error. But, this degree of uncertainty (or error of prediction) is known and can be quantified. When combined with other variables (e.g., conative characteristics; home environment variables; quantity and quality of instruction), intelligence tests can provide valuable explanatory and predictive information.

The anti-IQ rhetoric coming out of many politically correct academic school psychologist channels, as well as the powers-that-be driving the special education reforms at the federal level, is accurate in the statement that global/full-scale IQ scores, and their use in ability-achievement formula's for LD determination, is not an empirically-based defensible procedure. I couldn't agree more.

However, cognitive measures, especially those that are based on contemporary psychometric theory (e.g., CHC theory), and those that provide reliable and construct valid measures of most of the major broad CHC ability domains, can, and do provide useful information in the hands of skilled clinicians. In fact, those advocating against IQ tests offer, as alternatives, measures or "markers" of phonemic awareness, rapid automatized naming, working memory, and vocabulary to identify students at risk for reading disabilities. Anyone with any knowledge of CHC theory recognizes that these marker/screening abilities can be directly mapped to the CHC abilities of phonetic coding (Ga-PC), naming facility (Glr-NA), working memory (Gsm-MW), and lexical knowledge (Gc-VL). AND, all of these abilities are measured on the most comprehensive CHC-based intelligence batteries. Isn't the addage "don't throw the baby out with the bathwater" applicable?

The real irony (in the misrepresentation of the information in the Forrest Gump report) in the U. S. Dept. Of Education statement, is that the primary premise of the report was that students with disabilities should not be denied high expectations, inclusion in state assessments, or access to the general education curriculum based only on a single point IQ score. The primary theme of the report was that, for too long, some educators, psychologists, and policy makers have believed in the supreme power of IQ tests, to the point that inappropriately low academic achievement expectations may be formed. Furthermore, the primary thesis of that report is that kids with disabilities (that are so classified based on intelligence test scores) should be NOT be automatically excluded from high standards and state accountability systems--the exact opposite message that the ED.gov statement uses our report to support (viz., increasing the percent of students with disabilities in NCLB accountability activities).

Whatever happened to President Bush's statement, when unveiling NCLB, that for too long, many children have been victims of "the soft bigotry of low expectations." Isn't the exclusion of more students with disabilities from high-stakes state accountability systems, systems that require schools to raise their expectations for students, promoting the same "soft bigotry of low expectations?"

As I've often said, and I don't know who to attribute the original quote to (I want to say Ralph Reiten of the Halstead-Reitan Neurospcyhological Battery fame), "if you give a monkey a Stradivarius violin and you get bad music, you don't blame the violin." By extension, if you give a "politically motivated federal bureaucrat honest scholarly information....... [never mind, you get the point].

Temporary MIA: I shall return

I apologize to blog readers for a lack of posts the past few days. Just wayyyyyy to busy to come up for air. But, fear not, I've got plenty of stuff bouncing around my cranium and will start to post new info, musings, etc. by this weekend at the latest. Thanks.

So much data....so little time

Sunday, April 17, 2005

Gs, Glr, executive function and aging

I just skimmed the interesting article below. The abstract is pretty much self-explanatory. If one uses their CHC-SL (CHC as a Second Language) skills, the CHC interpretation is that the study focuses on Glr (recognition memory), Gs and executive functions (which seems to be Gf related).

Bunce, D., & Macready, A. (2005). Processing speed, executive function, and age differences in remembering and knowing. Quarterly Journal of Experimental Psychology Section A Human Experimental Psychology, 58(1), 155-168.

Abstract
A group of young ( n = 52, M = 23.27 years) and old ( n = 52, M = 68.62 years) adults studied two lists of semantically unrelated nouns. For one list a time of 2 s was allowed for encoding, and for the other, 5 s. A recognition test followed where participants classified their responses according to Gardiner's (1988) remember-know procedure. Age differences for remembering and knowing were minimal in the faster 2-s encoding condition. However, in the longer 5-s encoding condition, younger persons produced significantly more remember responses, and older adults a greater number of know responses. This dissociation suggests that in the longer encoding condition, younger adults utilized a greater level of elaborative rehearsal governed by executive processes, whereas older persons employed maintenance rehearsal involving short-term memory. Statistical control procedures, however, found that independent measures of processing speed accounted for age differences in remembering and knowing and that independent measures of executive control had little influence. The findings are discussed in the light of contrasting theoretical accounts of recollective experience in old age.

fMRI of IQ blog readers

Below is a composite fMRI of the brain activity of readers of this blog. Cool...don't you think?



Honestly, this is just a tease. I just wanted a cool and colorful fMRI picture on my blog so I could look hip.

Saturday, April 16, 2005

WISC-III/WJ-III cluster analysis results

So much data....so little time!

In a prior shameless plug, I briefly summarized the results of a recently published CHC-based confirmatory factor analysis study of a WJ-III/WISC-III cross-battery data set (Phelps, McGrew, Knopik & Ford, 2005). Following a favorite quantoid mantra ("there is more than one way to explore a data set"), I couldn't resist but conduct a more loosey-goosey (sp?) exploratory analysis of the data.

One of my favorite exploratory tools, given the Gv presentation of the multivariate structure of the data, is hierarchical cluster analysis (sometimes referred to as the "poor man's" factor analysis). Without going into detail, I subjected the data set previously described to Ward's clustering algorithm. As a word of caution, it is important to note that cluster analysis will provide neat looking cluster dendograms for random data....so one must be careful not to over-interpret the results. Yet, I find the looser constraints of cluster analysis and, in particular, the continued collapsing of clusters of tests (and lower-order clusters) into ever increasing broad higher-order clusters very thought provoking---the results often suggest different broad (stratum II) or intermediate level strata (as per Carroll's 3-stratum model).

I present the current results "as is" (click here to view or download). Blogsters will need to consult prior posts to glean the necessary pieces of information to interpret the CHC factor codes and names, the abilities measured by the WJ III tests, etc.

To say the least, some interesting hypothesis are suggested. In particular, I continue to be intrigued by the possibility of a higher-order dual cognitive processing model structure (within the CHC taxonomy) --that is, a distinction between automatic vs controlled/deliberate processing

Bayles Scales of Infant Development-II: A review

FYI post.

Alfonso et al (2005) have published a critical review of the Bayley Scales of Infant Development-2nd Edition, the venerable standardized measure of development abilities and milestones for infants up to 42 months of age. The review is published in Vol 59 (2) of the Spring issue of APA's Division 16 "The School Psychologist Newsletter." A copy can be viewed (or downloaded) by clicking on the Spring 2005 issue at the APA Division 16 web page.

Thursday, April 14, 2005

Student progress monitoring summer institute

I'm an equal assessment methodology blogster.

Yes, although I've spent most of my career developing standardized measures of intelligence and achievement, at my core is the essence of a true quantoid---someone who believes that psychology and education benefit from the development of all kinds of good measurement tools that may serve different functions.

In this spirit, and in my belief in the importance of continuous student monitoring systems, I hereby notify readers of this blog of a potentially valuable summer training experience on continuous student progress monitoring systems.

The National Center on Student Progress Monitoring has just announced it's 2005 Summer Institute (July 7-8 in Washington, DC). Registration is now open at the web site.

Long live good measurement on behalf of students.

Tuesday, April 12, 2005

'95 CHC Carroll mountain top mtg

I have been surprised by the number of blogsters who have an interest in history. My prior "In memory of Jack Carroll" post, which included a picture of Jack approximately 1 month prior to his passing away, generated a number of emails asking for more personal stories and/or picutres.

As I've written elsewhere (CHC Theory: Past, Present and Future), I believe that the recently declared CHC bandwagon has its roots in the cross-fertlization of the theoretical/research of John Horn and John "Jack" Carroll and the applied psychometric work of Dr. Richard Woodcock, over 20 years ago (1985). Amazing...it has been twenty years.

The Horn-Carroll-Woodcock collaboration, as I describe historically in the link above, continued up through the publication of the WJ3 in 2001. Below is another bit of history. A meeting 10 years ago (1-12-95) where (from left to right--after Jack Carroll on far left) Dr. Richard Woodcock, myself (don't ask why I'm so serious), and Dr. Fred Schrank gathered for one of our ongoing work sessions in Chapel Hill, NC. Yes, Woodcock, Schrank, and McGrew made routine visits to one of the CHC mountains (the other being John Horn) to sit in a circle and ask questions and learn pearls of wisdom. These truly were once-in-lifetime opportunities to learn from one of the greatest educational psychologists of all times.

A handsome and bright looking group...don't you think?

Example