Breaking legal and research news: SCOTUS (US Supreme Court) decision re multiple IQ scores in intellectual disability (ID) death penalty case (Hamm v Smith) - A DIG decision upholding the lower court decision

Today SCOTUS ruled on the Hamm v Smith multiple IQ score Atkins intellectual disability (ID) case that was argued before SCOTUS last December. This important decision, with all concurring and dissenting opinions, is available here.  Interested parties should read all the opinions. 

All documents and the history of the case can be found in a prior IQs Corner post.  As noted in the prior lengthy post, APA submitted an Amicus brief that, based on the reading of some of the justices opinions, suggests it (together with a AAIDD Amicus Brief) played an important role in the decision.

Briefly, when SCOTUS dismisses the writ of certiorari as improvidently granted it means the justices recognized that they made a mistake in agreeing to hear the case and decided to dismiss it without issuing a ruling on the merits. In legal shorthand, this is often referred to as a "DIG" (Dismissed as Improvidently Granted). This typically happens after the Court has accepted the case, reviewed the briefs, and sometimes even heard oral arguments. 

In practice, this means that the lower court decision—which is consistent with APA’s Amicus brief—that argued that multiple IQ scores should be viewed holistically  augmented by clinical judgement—stands. This means Mr. Smith will not be executed and, more importantly, the ruling does not upset the status of prior Atkins case law.  Given the recent direction and flavor of SCOTUS, there was a fear that SCOTUS might render a formal decision that would set back prior SCOTUS Atkins ID decisions.  So, this is a win for maintaining the principles of clinical judgement, the opinions and standards established by the established medical communities (both APA’s, AAIDD), and the need for a holistic approach to the interpretation of multiple IQ scores together with adaptive behavior.

A reading of the concurring opinions indicates that APA’s brief, as well as several key APA and AAIDD ID-related publications referenced in their decisions, were influential in the decision.  Several of the dissenting opinions reveal some troubling thinking by several justices. 

As a potential conflict of interest (COI) notice, I (Dr. Kevin McGrew), together with Dr. Joel Schneider and Dr. Cecil Reynolds (as noted on page three the APA amicus brief), were consultants to APA in the drafting of that brief.

It should be noted that as a result of the work of APA Amicus brief working committee, a peer-reviewed paper outlining a psychometrically sound approach (developed by Dr. Joel Schneider) for integrating multiple IQ scores was started during the deliberations and was recently published (Schneider, Reynolds, McGrew & Salekin, 2026) after the oral argument's.  This issue will likely to be revisited in future state cases, with Schneider et al. (2026) now elevated to a scientifically sound multiple IQ composite score method.

AI Brief: Is the Intellectual Functioning Component of AAIDD’s 12th Manual Satisficing?

AI Brief:  Is the Intellectual Functioning Component of AAIDD's 12th Manual Satisficing?

 (McGrew, 2021)

 

Dr. Kevin McGrew with assist from Google NotebookLM

 

In a commentary published in Intellectual and Developmental Disabilities, Kevin S. McGrew  evaluated the intellectual functioning section (prong 1) of the AAIDD’s 12th edition manual (2021) for diagnosing intellectual disabilities (ID). He commends the organization for finally adopting the Cattell-Horn-Carroll (CHC) theory, which aligns the manual with modern scientific consensus on cognitive abilities. However, the author expresses significant concern about the manual’s contradictory guidance on part scores, arguing that its ambiguous stance could lead to legal and diagnostic confusion. McGrew also highlights various technical measurement issues and numerous copyediting errors that he believes undermine the manual's status as an authoritative resource. He suggests that while the manual is satisfactory in its theoretical shift, it does not provide the precise clarity needed for high-stakes clinical and judicial settings.

 

_______________________

 

In his review of the 12th edition of the American Association on Intellectual and Developmental Disabilities (AAIDD) manual, McGrew (2021; link for downloading article) evaluates whether the "Intellectual Functioning Component" (aka.,prong 1 of a three-prong definition of intellectual disability—ID) provides a "satisficing"—or satisfactory and sufficient—solution for practitioners and scholars.[1] McGrew draws on over 45 years of experience in school psychology and intelligence research, theory, and test development. In addition, he draws on his expert work and consultation (since 2009) on Atkins intellectual disability (ID) death penalty cases in legal settings. McGrew provides an evaluation of the AAIDD’s manual's prong 1 (intellectual functioning) theoretical grounding, technical guidance, and professional polish. He does not evaluate the other two ID prongs (adaptive behavior and age of onset).

Advancement in Intelligence Theory

McGrew awards the manual a Grade B+ for its formal adoption of the Cattell-Horn-Carroll (CHC) theory of intelligence. This shift aligns the AAIDD manual with the contemporary consensus taxonomy of cognitive abilities, moving away from outdated models. However, McGrew notes that the manual "muddies the CHC waters" by giving preferential treatment to fluid (Gf) and crystallized (Gc) intelligence while neglecting other broad CHC abilities like learning efficiency (Gl), working memory (Gwm), retrieval fluency (Gr), auditory processing (Ga), visual-spatial processing (Gv), and processing speed (Gs). He suggests that a visual-graphic model of the CHC hierarchical model would have been a beneficial addition for users.

Measurement and Organizational Challenges

The manual receives a Grade B- for its treatment of major measurement issues. While it provides adequate coverage of such measurement issues as the standard error of measurement (SEM), confidence intervals, and the Flynn effect (aka., norm obsolescence), McGrew criticizes the lack of a topic index, which makes finding specific guidance very frustrating. For instance, practice effects are obscurely placed under "progressive error" in the glossary, and the Flynn effect is curiously categorized under "Making a Retrospective Diagnosis," despite being relevant to historical and current intellectual assessments.

The Part-Score Controversy

The most critical evaluation—a Grade C—is reserved for the manual's handling of part scores. McGrew identifies three primary failures in this area:

 

●      Inconsistency: The manual contradicts itself by advising against the use of part scores as proxies for general intelligence (psychometric g) while simultaneously suggesting that their valid use requires 3–6 subtests of Gf and Gc.

●      Variance with Other Authorities: This "just say no to part scores" stance conflicts with other major authoritative sources, such as the DSM-5, which acknowledges that highly discrepant subtest scores may invalidate an overall IQ score.

●      Scientific and Legal Tensions: McGrew argues that the manual fails to address the "General-2-individual" (G2i) legal principle which acknowledges that group-based scientific research (e.g., suggesting full-scale scores are always superior) may not apply to every unique individual case—the G2i principle conundrum is that scientists generalize; but courts must particularize to an individual. He warns that without clearer guidance; legal entities may fill the void with "remedies of dubious quality.”

Editorial Quality and Professionalism

McGrew gives the manual a Grade D for style and substance, citing at least 20 copyedit errors in the sections relevant to the intellectual functioning prong alone. These include misspellings of prominent researchers, incorrect terminology like "test e-norms," and frequent "misplaced italics.” He contends that such preventable errors tarnish the manual’s status as an "authoritative" and "definitive" source for diagnosing intellectual disabilities.

Conclusion

McGrew concludes that while the endorsement of CHC theory is a significant positive revision, the manual’s obfuscation regarding part scores and its numerous editorial flaws represent major missed opportunities. He emphasizes that practitioners cannot wait another decade for the next edition to offer more robust guidance, particularly in high-stakes legal and diagnostic settings. He concludes that while he may be a "tough grader," his critiques are intended to push AAIDD toward more robust and clearer guidance in future editions or supplements

[1] Nobel laureate Herb Simon advanced the behavioral economics concept of satisficing (Simon, 1956)—the idea that, although we may aspire to optimal solutions, real-world constraints often require us to settle on what is both satisfactory and sufficient (hence, the portmanteau term satisficing).

Research Alert: #Giftedness: A Critical Analysis of #Theories and Identification Methods in Light of Contemporary #Neuroscience - #CHC #intelligence #motivation #conative

And yet another open access overview article on giftedness.

 

Giftedness: A Critical Analysis of Theories and Identification Methods in Light of Contemporary Neuroscience - PMC 
https://pmc.ncbi.nlm.nih.gov/articles/PMC13099527/

Click on images to enlarge for easy viewing

Pardon typos and spelling errors-Message may be sent from iPhone and I've always had spelling problems :)

Research alert: Life-and-death psychometrics: Generalizable best methods for combing scores in intellectual disability and other diagnostic assessments - #pschometrics #Atkins #SCOTUS #IQ #intelligence #ID #intellectualdisability #AAIDD

Click on image to enlarge for better viewing 

Click here for the Journal of Pediatric Neuropsychology article page.

Abstract

A diagnosis of intellectual disability is a momentous event that can determine eligibility for special services and supportive sources of income, and in the criminal arena, it can be a matter of life and death. For criminal defendants who might otherwise face capital punishment, it is a matter of life and death. Individuals evaluated for intellectual disability often have been given multiple intelligence tests, sometimes with results falling on both sides of the diagnostic threshold. In all cases, the diagnostic decision must be based on a rigorous examination of the totality of evidence in the context of systematic clinical judgment. When multiple IQ results are relevant and comparable, they can be combined into a properly computed composite score to assist the clinician charged with diagnostic responsibility in determining if Prong 1, deficits in intellectual functioning, of the three-prong criteria necessary for an intellectual disability diagnosis has been met. Best psychometrically grounded methods for these calculations are presented along with a discussion of inappropriate approaches for accurately combining multiple scores. To make these methods accessible to professionals outside the discipline of psychology, all calculations are fully explained in the context of foundational concepts. 

Interesting, reasonably accurate recent video on IQ/intelligence testing

I just stumbled across a relatively new video covering the history and several major issues regarding intelligence testing and IQ scores.  Two scholars that I respect (Dr. Cecil Reynolds; Dr. Stuart Ritchie) are featured in the video.  I did see some spelling errors in the subtitles (Dr. Ian Dearie instead of Dr. Ian Deary; Benet instead of Binet; using capital G when referencing Spearman's concept of general intelligence, which is always noted with an italic font small g; etc) and heard several statements that made me cringe slightly.  

Also, it left the impression that fluid and crystallized intelligence (and a lessor extent quantitative ability) are the primary recognized broad cognitive abilities measured by intelligence tests.  It did not acknowledge contemporary CHC theory as the consensus taxonomy of human cognitive abilities.  Also, it left the impression that IQ tests are "bubble in" multiple choice tests.  This may be true for group tests, but it is not the case with individually administered intelligence tests.

Overall, it is a reasonable video to share with others as an introduction, possibly in college courses where the concept of intelligence and IQ testing is being introduced.  It did a good job of covering the historical bad uses of IQ tests (e.g., discrimination; cultural bias, eugenics movement, etc.) 

The complete video is approximately 35 minutes.  It did freeze up for me at the 17 minute mark when it was going to display an ad....but I simply restarted the video and quickly moved to that point and then it continued.

 

 

 

IQs Corner. In what way are #intelligence testing (#IQ) and the US Supreme Court (#SCOTUS) alike?—SCOTUS will be hearing important case addressing #multiple IQ scores and #intellectualdisability #Dx in fall 2025 term

This fall 2025, the Supreme Court of the United States (SCOTUS) will be hearing a case related to intelligence testing in the context of Atkins intellectual disability (ID) death penalty cases. The case is Hamm v Smith.

The question before SCOTUS is :  Whether and how courts may consider the cumulative effect of multiple IQ scores in assessing Atkins claims (in the context of diagnosis ID in death penalty cases)?

Note.  In order to save space and time, instead of writing “general intelligence” or “general intellectual functioning” every time, I use the abbreviation “IQ”.

The respondent (Joseph Smith) has five IQ test scores from comprehensive IQ tests.  He obtained two scores of 75 and 74 during the developmental period (before age 22), and three scores of 72, 78, and 74 between the ages of 28 and 46.  

This case is important for assessment professionals who conduct intelligence testing in general, and potential ID diagnostic cases (Atkins cases in particular).  I find this SCOTUS case particularly interesting given that in 2021, after the 2021 release of the latest official AAIDD manual (Intellectual disability: Definition, diagnosis, classification, and systems of supports), I published a critique where I specifically stated, as one weakness of the new AAIDD manual that “…many high-stakes ID cases often include case files that include multiple IQ scores across time or from different IQ tests. Some form of guidance, at minimum in a passing reference, to the issues of the convergence of indicators and IQ score exchangeability would have been useful. Users will need to go beyond the AAIDD manual for guidance (see Floyd et al., 2021; McGrew, 2015; and Watson, 2015)” (click here to download and read this critique).

All official petitioner and respondent legal briefs (and amicus briefs) have now been published at the SCOTUS blog as of yesterday.  The number of documents posted on the SCOTUS docket are many.  To help the reader better determine which documents are most critical (the final briefs), instead of clicking away on the various links at the SCOUTUS blog, I’ve organized the petitioner and respondent brief links below.

If you prefer to not wade through all the briefs (it is not for everyone), I would encourage practicing assessment professionals read the three respondent-related briefs.  The points made are relevant to all who conduct intellectual assessments.  As a potential conflict of interest notice, I (Dr. Kevin McGrew), together with Dr. Joel Schneider and Dr. Cecil Reynolds (as noted on page three for the APA amicus brief), were consultants to APA in the drafting of that brief.  This work was performed pro bono. I, at a minimum, suggest reading all the respondent briefs.  If time permits, I would also suggest reading the petitioner’s Alabama brief and the US Justice Department Solicitor General’s brief to better understand the petitioner and respondent positions re Hamm v Smith. 

Petitioner briefs

• The state of Alabama brief.  Alabama is the petitioner.  That is, if you want to read why the State of Alabama asked SCOTUS to hear this case, click on the link provided.
• The Alabama brief also includes a very long appendix for those who want to read the prior courts related testimony from the state and various experts. This is a very long read and is not necessary for readers who only want to understand the legal and professional issues. 
• Supporting amicus brief from the US Justice Department Solicitor General.
• Two supporting briefs from legal groups—the American Legal Foundation and the Criminal Justice Legal Foundation.
• Supporting amicus briefs from other states (Idaho et al.; Kentucky)

Respondent briefs

• Smiths respondent brief.
• Supporting amicus brief from American Psychological Association (APA), American Psychiatric Assocation (ApA), and Alabama APA.
• Supporting amicus brief from AAIDD et al.

Final comment.  Those from school psychology should make note that we three consultants involved in drafting the APA/ApA,AL-APA brief all had our original educational roots in the profession of school psychology.  Furthermore, SP professionals should note the significant number of authoritative references to publications authored by school psychologists in the respondents briefs, as well as in some of the petitioners briefs.  I’ve been doing expert consultation, writing declarations, and testifying in court re: Atkins ID cases since 2009.  Joel Schneider and Cecil Reynolds have also been active in a similar capacity.  There are more psychologists who come from, or are affiliated with, the field of school psychology who have been prominent consultants/experts to lawyers and the courts re Atkins cases.  

Perhaps some of these briefs should be assigned readings (in intellectual assessment courses or special topic seminars) for graduate students being trained in the art and science of intelligence testing and interpretation.

IQs Corner: What is (and what is not) clinical judgment in intelligence test interpretation? - #IQ #intelligence #ID #intellectualdisability #schoolpsychologists #schoolpsychology #diagnosis

What is clinical judgment in intelligence testing?  

This term is frequently invoked when psychologists explain or defend their intelligence test interpretations.  Below is a brief explanation I’ve used to describe what it is…and what it is not, based on several sources.  Schalock and Luckasson’s AAIDD Clinical Judgment book (now in a 2014 revised version) is the best single source I have found that addresses this slippery concept in intelligence testing, particularly in the context of a potential diagnosis of intellectual disability (ID)—it is a recommended reading.

—————

Clinical judgment is a process based on solid scientific knowledge and is characterized as being “systematic (i.e., organized, sequential, and logical), formal (i.e., explicit and reasoned), and transparent (i.e., apparent and communicated clearly)” (Schalock & Luckasson, 2005, p.1). The application of clinical judgment in the evaluation of IQ scores in the diagnosis of intellectual disability includes consideration of multiple factors that might influence the accuracy of an assessment of general intellectual ability (APA: DSM-5, 2013).  The “unanimous professional consensus that the diagnosis of intellectual disability requires comprehensive assessment and the application of clinical judgment” (Brief of Amici Curiae American Psychological Association, American Psychiatric Association, American Academy of Psychiatry and the Law, Florida Psychological Association, National Association of Social Workers, and National Association of Social Workers Florida Chapter, in Support of Petitioner; Hall v. Florida; S.Ct., No. 12-10882; 2014; p. 8).

The misuse of clinical judgment in the interpretation of scores from intelligence test batteries should not be used as the basis for “gut instinct” or “seat-of-the-pants” impressions and conclusions of the assessment professional (Macvaugh & Cunningham, 2009), or justification for shortened evaluations, a means to convey stereotypes or prejudices, a substitute for insufficiently explored questions, or an excuse for incomplete testing and missing data (Schalock & Luckasson, 2005). Idiosyncratic methods and intuitive conclusions are not scientifically based and have unknown reliability and validity. 

If clinical judgement interpretations and opinions regarding an individual’s level of general intelligence are based on novel or emerging research-based principles, the assessment professional must document the bases for these new interpretations as well as the limitations of these principles and methods. This requirement is consistent with the Standards for Educational and Psychological Testing Standard 9.4 which states:

When a test is to be used for a purpose for which little or no validity evidence is available, the user is responsible for documenting the rationale for the selection of the test and obtaining evidence of the reliability/precision of the test scores and the validity of the interpretations supporting the use of the scores for this purpose (p. 143).

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (2014).  Standards for educational and psychological testing.  Washington, DC:  Author. 

American Psychiatric Association (2013). Diagnostic and statistical manual of mental disorders— Fifth Edition. Washington D.C.:  Author. 

Brief of Amici Curiae American Psychological Association, American Psychiatric Association, American Academy of Psychiatry and the Law, Florida Psychological Association, National Association of Social Workers, and National Association of Social Workers Florida Chapter, in Support of Petitioner; Hall v. Florida; S.Ct., No. 12-10882; 2014; p. 8.

MacVaugh, G. S. & Cunningham, M. D. (2009). Atkins v. Virginia: Implications and recommendations for forensic practice.  The Journal of Psychiatry and Law, 37, 131-187.

Schalock, R. L. & Luckasson, R. (2005). Clinical judgment. Washington, DC: American Association on Intellectual and Developmental Disabilities. 

—————

Kevin S. McGrew, PhD.

Educational Psychologist

Director 

Institute for Applied Psychometrics (IAP)

www.theMindHub.com

Research Byte: Is trying hard enough? Causal analysis of the effort-IQ relationship suggests not - #intelligence #IQ #motivation #volition #CAMML #conative #noncognitive

Is Trying Harder Enough? Causal Analysis of the Effort-IQ Relationship Suggests Not.  Timothy Bates. Intelligence and Cognitive Abilities (open access—click here to locate article to read or download)

Abstract

Claims that effort increases cognitive scores are now under great doubt. What is needed is randomized controlled trials optimized for testing causal influence and avoiding confounding of self-evaluation of performance with feelings of good effort. Here we report three large studies using unconfounded measures of effort and instrumental analysis to isolate any causal effect of effort on cognitive score. An initial study (N = 393) validated an appropriate effort measure, demonstrating excellent external and convergent validity (β = .61). Study 2 (N = 500, preregistered) randomly allocated subjects to a performance incentive, using an instrumental variable analysis to detect causal effects of effort. The incentive successfully manipulated effort (𝛽 = .18, p = .001). However, the causal effect of effort on scores was near-zero and non-significant (𝛽 = .04, p = .886). Study 3 (N=1,237) replicated this null result with preregistered analysis and an externally developed measure of effort: incentive again raised reported effort (𝛽 = .17, p <.001), but effort had no significant causal effect on cognitive score (β2 = .27 [-0.07, 0.62]), p = .15). Alongside evidence of research fraud and confounding in earlier studies, the present evidence for the absence of any causal effects of effort on cognitive scores, effort research should shift its focus to goal setting – where effort is useful – rather than raising basic ability, which it appears unable to do.

Select quote from discussion: “The present results suggest a potential ‘central dogma of cognition’: that volitional effort can direct cognitive resources but cannot fundamentally alter or bypass the efficacy of the underlying cognitive systems themselves”

These findings are consistent with my proposed cognitive-affective-motivation-model-of-learning (CAMML), grounded extensively on Richard Snows concept of aptitude trait complexes, where motivational constructs are seen as driving and directing the use of cognitive abilities (via personal investment mechanisms), but not directly having a causal effect on cognitive abilities.  See first of two figures below.  Note lack of causal arrows from conative and affective domain constructs to CHC cognitive abilities.  Paper can be accessed by clicking here.

Click on images to enlarge for easier viewing

On #factoranalysis of #IQ tests—impact of software choice—plus comments about art+science of factor analysis in #intelligence test research—#schoolpsychology

Contributing to the reproducibility crisis in Psychology: The role of statistical software choice on factor analysis.  Journal of School Psychology.  Stefan C. Dombrowski.  Click here to view article source and abstract.

This is an important article for those who conduct (and also those who consume) factor analysis results of intelligence or cognitive ability tests.  

Abstract (note - bold font in abstract has been added by me)

A potentially overlooked contributor to the reproducibility crisis in psychology is the choice of statistical application software used for factor analysis. Although the open science movement promotes transparency by advocating for open access to data and statistical methods, this approach alone is insufficient to address the reproducibility crisis. It is commonly assumed that different statistical software applications produce equivalent results when conducting the same statistical analysis. However, this is not necessarily the case. Statistical programs often yield disparate outcomes, even when using identical data and factor analytic procedures, which can lead to inconsistent interpretation of results. This study examines this phenomenon by conducting exploratory factor analyses on two tests of cognitive ability—the WISC-V and the MEZURE—using four different statistical programs/applications. Factor analysis plays a critical role in determining the underlying theory of cognitive ability instruments, and guides how those instruments should be scored and interpreted. However, psychology is grappling with a reproducibility crisis in this area, as independent researchers and test publishers frequently report divergent factor analytic results. The outcome of this study revealed significant variations in structural outcomes among the statistical software programs/applications. These findings highlight the importance of using multiple statistical programs, ensuring transparency with analysis code, and recognizing the potential for varied outcomes when interpreting results from factor analytic procedures. Addressing these issues is important for advancing scientific integrity and mitigating the reproducibility crisis in psychology particularly in relation to cognitive ability structural validity.

My additional comments

The recommendation that multiple factor analysis software programs be used when analyzing the structural validity of cognitive abilities tests makes sense.  Kudos to Dr. Dombrowski for demonstrating this need.

Along these lines, it is also important to recognize that the use and interpretation of any factor analysis software is highly dependent on the statistical and substantive expertise and skills of the researcher.  I made these points (based on the writings and personal conversations with Jack Carroll) in a recent article (McGrew, 2023; open access so you can download and read) in the Journal of Intelligence.  The salient material is reproduced below.  This article can be accessed either a the journal website or via the Research and Reports section of my MindHub web page (McGrew, 2023)

(Note - Bold font in text below, extracted from McGrew (2023), is not in the original published article)

“I was fortunate to learn important tacit EFA and CFA knowledge during my 17 years of interactions with Carroll, and particularly my private one-to-one tutelage with Carroll in May 2003. Anyone who reads Chapter 3 (Survey and Analysis of Correlational and Factor-Analytic Research on Cognitive Abilities: Methodology) of Carroll's 1993 book, as well as his self-critique of his seminal work (Carroll 1998) and other select method-focused post-1993 publications (Carroll 1995, 1997), should conclude what is obvious—to Carroll, factor analyses were a blend of art and science. As articulated by some of his peers (see footnote #2), his research reflected the work of an expert with broad and deep substantive knowledge of research and theories in intelligence, cognitive psychology, and factor analysis methods. 

In 2003, after Carroll had been using CFA to augment his initial EFA analyses for at least a decade, Carroll expressed (to me during our May 2003 work week) that he was often concerned with the quality of some reported factor analyses (both EFA and CFA) of popular clinical IQ tests or other collections of cognitive ability measures (Carroll 1978, 1991, 1995, 2003). Carroll's characteristic positive skepticism regarding certain reported factor analyses was first articulated (as far as I know) in the late 1970's, when he stated “despite its many virtues, factor analysis is a very tricky technique; in some ways it depends more on art than science, that is, more on intuition and judgment than on formal rules of procedure. People who do factor analysis by uncritical use of programs in computer packages run the risk of making fools of themselves” (Carroll 1978, p. 91; emphasis added). It is my opinion that Carroll would still be dismayed by some of the EFA and CFA studies of intelligence tests published during the past two decades that often used narrow or restricted forms of factor analysis methods and rigid formal statistical rules for decision-making, with little attempt to integrate contemporary substantive research or theory to guide the analysis and interpretation of the results (e.g., see Decker 2021; Decker et al. 2021; McGrew et al. 2023). 

Carroll's unease was prescient of recently articulated concerns regarding two aspects of the theory crises in structural psychological research—the conflation of statistical (primarily factor analysis) models with theoretical models and the use of narrow forms of factor analysis methods (Fried 2020; McGrew et al. 2023). First, many intelligence test batteries only report CFA studies in their technical manuals. EFA results, which often produce findings that vary from CFA findings, are frequently omitted. This often leads to debates between independent researchers and test authors (or test publishers) regarding the validity of the interpretation of composite or cluster scores, leaving test users confused regarding the psychometric integrity of composite score interpretations. McGrew et al. (2023) recently recommended that intelligence test manuals, as well as research reports by independent researchers, include both EFA and CFA (viz., bifactor g, hierarchical g, and Horn no-g models), as well as psychometric network analysis (PNA) and possibly multidimensional scaling analyses (MDSs; McGrew et al. 2014; Meyer and Reynolds 2022). As stated by McGrew et al. (2023), “such an ecumenical approach would require researchers to present results from the major classes of IQ test structural research methods (including PNA) and clearly articulate the theoretical basis for the model(s) the author's support. Such an approach would also gently nudge IQ test structural researchers to minimize the frequent conflation of theoretical and psychometric g constructs. Such multiple-methods research in test manuals and journal publications can better inform users of the strengths and limitations of IQ test interpretations based on whatever conceptualization of psychometric general intelligence (including models with no such construct) underlies each type of dimensional analysis” (p. 24).”

#Intelligence (#IQ) #cognitive testing in perspective: An #ecological systems brief video explanation—useful for #schoolpsychology

Click on image to enlarge for easy reading

An oldie but goodie!  This is a 19+ minute narrated video (sit down with your favorite beverage and enjoy) where I explain how intelligence (IQ) or cognitive ability testing should be better understood in the context of a larger ecological systems model perspective (Bronfenbrenner).  

I first posted the video in 2015—-9 years ago! So be gentle…I’m much better with these videos now :) Thus, some of my COI statements/disclaimers/affiliations are no longer accurte (and updated version can be found a theMindHub.com—Under About IAP: The Director: Disclosures & Bio).

If all works well, just click the start arrow on the video screen…and tap the enlarge icon in the lower right corner.  This video is now hosted on YouTube, so it may be possible that you may first encounter 1-2 very brief adds that you can skip within the first 15-10 seconds.  It is possible (it seems to vary everytime) that you might be asked to “sign in” to show you are not a bot.  All you need to do is press the message, or if images of muliptle videos appear, press the first one…if you only get the message you may need to back up and try link again (no signing in….I hate having lost control of how these work by using YouTube 9 years ago…as now the starting has these mild annoyances..but it is the price for a free service).  Be aware that some of the first 4-5 slides may have minimal or no narration and you can skip ahead to the beginning…it is the first slide shown immediately below before the video. Given the caveats above, it is possible the video might not deploy exactly how I describe…the platform seems to be a bit tempormental, at least for me.  Enjoy.

#AAIDD's #IQ Part-Score Position (with reference to diagnosing #intellectual #disabiity [#ID]) Is at Variance With Other Authoritative Sources—Important for #schoolpsychologists

In a 2021 commentary regarding the most recent official AAIDD intellectual disability definition and classification manual (2021), I raised a concern regarding AAIDD’s position that only a full scale or global IQ score can be used for a Dx of ID.  No room was left for clinical judgement for n=1 unique cases.  Click here to download and read the complete article.  Below is some select text.

“AAIDD's [IQ] Part-Score Position Is at Variance With Other Authoritative Sources”

In AAIDD’s The Death Penalty and Intellectual Disability (Polloway, 2015), both McGrew (2015) and Watson (2015) suggest that [IQ] part scores can be used in special cases. (Note that these two chapters, although published in an AAIDD book, do not necessarily represent the official position of AAIDD.) The limited use of part scores is also described in the 2002 National Research Council book on ID and social security eligibility (see McGrew, 2015; Watson, 2015). The authoritative Diagnostic and Statistical Manual of Mental Disorder—Fifth Edition (DSM-5) manual implies that part scores may be necessary when it states that ‘‘highly discrepant subtest scores may make an overall IQ score invalid'' (American Psychiatric Association, 2013, p. 37). Finally, in the recent APA Handbook of Intellectual and Developmental Disabilities (Glidden, 2021), Floyd et al. (2021) state ‘‘in rare situations in which the repercussions of a false negative diagnostic decision would have undue or irreparable negative impact upon the client, a highly g-loaded part score (see McGrew, 2015a) might be selected to represent intellectual functioning'' (emphasis added; p. 412).

“ In a unique n = 1 high-stakes setting, a psychologist may be ethically obligated to proffer an expert opinion whether the full-scale score is (or is not) the best indicator of general intelligence. There must be room for the judicious use of clinical judgment-based part scores. AAIDD's purple manual complicates rather than elucidates guidance for psychologists and the courts. In high-stakes settings, a psychologist may be hard pressed to explain that their proffered expert opinions are grounded in the AAIDD purple manual, but then explain why they disagree with the ‘‘just say no to part scores'' AAIDD position.”

More on the conflation of #psychometric #g (general #intelligence): Is g the Loch Ness Monster of psycholgy?

From McGrew et al. (2023) article (click here for prior post and access to the article in Journal of Intelligence.)”  Click here for a series of slides regarding the theoretical and psychometric conflation of g.

The Problem of Conflating Theoretical and Psychometric g

“Contributing to the conflicting g-centric and mixed-g positions (regarding the interpretive value of broad CHC scores) is the largely unrecognized common practice of conflating theoretical and psychometric g. Psychometric g is the statistical extraction of a latent factor (via factor analysis) that accounts for the largest single source of common variance in a collection of cognitive abilities tests. It is an emergent property statistical index. Theoretical g refers to the underlying biological brain-based mechanism(s) that produce psychometric g. The global composite score from IQ test batteries is considered the best manifest proxy for psychometric g. The conflation of psychometric and theoretical g in IQ battery structural research ignores a simple fact—“general intelligence is not the primary fact of mainstream intelligence research; the primary fact is the positive manifold….general intelligence is but one interpretation of that primary fact” (Protzko and Colom 2021a, p. 2; italic emphasis added). As described later, contemporary intelligence and cognitive psychology research has provided reasonable and respected theories (e.g., dynamic mutualism; process overlap theory; wired cognition; attentional control), robust methods (psychometric network analysis), and supporting research (Burgoyne et al. 2022; Conway and Kovacs 2015; Kan et al. 2019; Kievit et al. 2016; Kovacs and Conway 2016, 2019; van der Maas et al. 2006, 2014, 2019) that accounts for the positive manifold of IQ test correlations in the absence of an underlying latent causal theoretical or psychometric gconstruct.” (p.3; bold font emphasis added).