Another AI evaluation framework informed by the human CHC cognitive abilities taxonomy

According to my tracking of the literature (of course, most likely not complete), here is a third article (open access from the Journal of Intelligence) that proposes using the CHC cognitive ability taxonomy (or parts of it) to evaluate an aspect of AI performance—in this case AI-assisted organizational decision-making.  The other two proposed AI evaluation framework articles using the CHC taxonomy (alone or as part of a much grander framework) can be found here and here. 

In a related development, Kameron Green—the developer behind an ambitious and grand AI evaluation framework (in second to last link above)—today posted a link to an AI generated video summary overview of his HCQM taxonomic model—which would help most readers understand his lengthy HCQM paper.

Cognitive Performance Under AI Advice: Development and Initial Validation of a CHC-Informed Assessment for Organizational Decision-Making

Abstract

Artificial intelligence (AI) is increasingly embedded in organizational decision-making, requiring employees not only to use AI-generated recommendations but also to evaluate their quality and determine when reliance is appropriate. Although established research examines behavioral reliance on algorithmic and AI advice, fewer studies have approached performance under AI advice as an individual-differences assessment problem integrating psychometric structure, cognitive correlates, process indicators, and criterion-related evidence. This study developed and initially validated a CHC-informed, performance-based assessment of cognitive performance under AI advice using 24 organizational decision scenarios. The assessment was designed around three closely related content/performance dimensions—AI error detection, evidence integration, and cognitive control and adaptive reliance—while also capturing confidence, response time, and reliance behavior. The validation sample comprised 780 employed adults in Türkiye. Psychometric analyses included confirmatory factor analysis, multidimensional item response theory, response-time analyses, scenario-level logistic regression, measurement invariance, differential item functioning, and internal cross-validation. Results indicated a dominant general cognitive-performance component together with additional structure corresponding to the three theoretically specified dimensions. Assessment performance was positively associated with established cognitive measures, including ICAR-16 reasoning performance, working memory, processing speed, and attentional control, whereas associations with AI-related self-reports were generally weaker. Dimension-aligned analyses supported the expected associations of ICAR-16 with AI error detection and working memory with evidence integration. Performance was also moderately associated with concurrently assessed organizational decision quality (r = 0.408) and explained additional variance in this criterion beyond demographic and work characteristics, AI experience, conventional cognitive-performance measures, and AI-related self-reports (ΔR2 = 0.103, p < .001). In contrast, several hypothesized scenario-specific associations involving AI confidence, time pressure, interruptions, and resistance to confidently inaccurate advice were not supported. Overall, the findings provide initial evidence for a performance-based approach to assessing how employees evaluate and respond to AI advice, while indicating that the proposed scenario-specific mechanisms and group-comparability findings require further replication before consequential applications are considered.

Research Alert: Is It Mine or Not Mine? Cognitive Offloading to Generative AI and Concern over the Erosion of Cognitive Ownership

Email-based research alert—open access link to download

 

Is It Mine or Not Mine? Cognitive Offloading to Generative AI and Concern over the Erosion of Cognitive Ownership 
https://www.mdpi.com/2079-3200/14/8/182

 

 

Abstract

Cognitive offloading—delegating cognitive work to external tools—is basic to human cognition, but generative AI (GenAI) amplifies it radically: entire cognitive products can now be produced on request. This sharpens a question about agency over one's own thinking: when cognition is habitually offloaded, does its product still feel like one's own? We surveyed 239 pre-service teachers, operationalizing the felt loss of cognitive ownership as concern over the erosion of teaching-design subjectivity (TSC)—the metacognitive appraisal that AI-assisted work is not genuinely one's own and that independent capacity is declining. Guided by the Interaction of Person-Affect-Cognition-Execution (I-PACE) model, we tested whether AI anxiety/fear of missing out (affective) and impulsivity (self-regulatory) relate to this concern through behavioral GenAI dependency—habitual offloading. In a structural equation model with bias-corrected bootstrapping, dependency strongly predicted TSC and partially mediated the effect of AI anxiety; impulsivity raiseddependency but showed a suppression pattern (a positive indirect effect offset by a null total effect), and the model explained 35% of the variance in the concern. Habitual offloading—that is, GenAI dependency—rather than generic AI use, accompanies the metacognitive loss of cognitive ownership.

Keywords: cognitive offloading; agency; cognitive ownership; metacognition; generative AI; GenAI dependency

Click on image to enlarge for easy reading.

 

Highly recommended AI paper regarding human & AI capabilities. HCQM: A dual-use human capability assessment and prescriptive target for engineering synthetic cognitive architectures (AI)

I recently exchanged emails and zoomed with Kameron Green who works in the AI space regarding his proposed Human Capability Quotient Map (HCQM).  I was thoroughly impressed with his proposed dual-use meta-taxonomy of human competence taxonomies (e.g., CHC; executive function; self-regulation; motivation; etc) into a grand overarching ability and capacity taxonomy.  As stated in the paper “HCQM is positioned to support broad human capability assessment while also providing a partial prescriptive target, at the capacity layer, for engineering synthetic cognitive architectures.”  

IMHO his grand model is one of the most comprehensive proposed high-level integrations of human individual difference constructs and emerging AI competencies I have read.  Although long, I urge those interested in human individual difference domains and AI evaluation frameworks to read his working paper. The paper is relevant to those studying individual differences in humans and those working on models by which to evaluate AI.  

The paper can be found (and downloaded) at the link provided by Kameron (click here).  He has also shared access to his paper at his LinkedIN profile (which is the URL associated with the hyper-link with his name in the first sentence of this post).

I’m sharing this link/paper with his permission.

 

Abstract

The field of intelligence research has long been characterized by fragmentation, with distinct traditions examining general cognitive abilities (Carroll, 1993; McGrew, 2009), emotional and social competencies (Salovey & Mayer, 1990; Goleman, 1995), creativity (Guilford, 1950), metacognition (Flavell, 1979), grit and adaptability (Duckworth et al., 2007), and more recent constructs such as digital intelligence (Park, 2019) and systems thinking (Senge, 1990; Meadows, 2008). While each line of inquiry has yielded valuable insights and measurement tools, the absence of an integrated taxonomy limits holistic assessment of human potential and the principled design of synthetic cognitive architectures. This paper proposes the Human Capability Quotient Map (HCQM) as a hierarchical synthesis that organizes eight top-level domains (General Cognitive Intelligence, Executive/Self-Regulatory Intelligence, Emotional & Social Intelligence, Creative & Innovation Intelligence, Motivational & Adaptive Intelligence, Learning & Knowledge Intelligence, Digital & Technological Intelligence, and Systems & Strategic Intelligence) into a coherent framework. Each domain includes subcomponents and observable indicators drawn from established psychometric, psychological, and cognitive-science literature.

HCQM makes three contributions: the integration itself, a coverage-asymmetry observation, and an explicitly dual-use framing; the two durable differentiators are the integration and the dual-use framing. HCQM is positioned to support broad human capability assessment while also providing a partial prescriptive target, at the capacity layer, for engineering synthetic cognitive architectures, with the explicit limitation that full architectural prescriptiveness requires a companion specification document (§6.6). These are applied to a coverage asymmetry: the human-capability tradition treats motivational, affective, cultural, and adversity capability as first-class (contemporary CHC, for example, now includes emotional intelligence as a broad ability), whereas contemporary AI architecture and evaluation frameworks largely do not. Unlike purely evaluative taxonomies such as DeepMind's 2026 Measuring Progress Toward AGI: A Cognitive Framework (Burnell et al., 2026), which identifies 10 cognitive faculties for benchmarking AGI progress, HCQM brings these dimensions to bear as a specification target. Unlike engineering-oriented architectural frameworks such as CoALA (Sumers et al., 2024), which specifies memory modules, action spaces, and decision loops for language agents, HCQM specifies the capacity surface those structural slots are expected to implement. The dimensions the AI frameworks omit are not a miscellaneous remainder: they concentrate in the motivational, adaptive, and metacognitive capacities (persistence under failure, strategy revision, self-monitoring, calibration) that govern whether a long-horizon autonomous agent is reliable, as distinct from whether it is capable. HCQM's distinctive AI-facing contribution is to organize these as a first-class reliability-and-autonomy layer and specify it at the capacity level (§5.2); a worked example maps a documented long-horizon agent failure pattern to the layer it omits. HCQM consolidates and ports existing constructs rather than claiming to discover them.

Drawing on CHC theory (Carroll, 1993; Schneider & McGrew, 2018), multiple-intelligences and triarchic models (Gardner, 1983; Sternberg, 1985), cultural intelligence (Earley & Ang, 2003), computational thinking (Wing, 2006), and modern cognitive architectures for language agents (Sumers et al., 2024), HCQM offers a synthesis rather than a novel discovery. We outline design principles, detail the hierarchical structure, discuss applications in human development and AI engineering, and acknowledge limitations, including the need for empirical validation, operationalized assessment instruments, and an explicit evaluation protocol. This paper aims to stimulate cross-disciplinary dialogue and guide future instrument development and architectural design.

Keywords: human capabilities, intelligence taxonomy, cognitive architecture, AGI evaluation, capability-grounded architectures, autonomous agents, agent reliability, long-horizon agents, metacognition, grit, cultural intelligence, systems thinking, CHC theory, CoALA.

Click on image to enlarge for easy viewing

Research Alert: When Can AI Models Explain Learning? Validity Criteria for AI as Cognitive Models in Education

This is a quick email-based research alert….this is also an open access article you can download.😉

 

When Can AI Models Explain Learning? Validity Criteria for AI as Cognitive Models in Education | Educational Psychology Review | Springer Nature Link 

https://link.springer.com/article/10.1007/s10648-026-10202-y

Click on image to enlarge for easy reading

 

Abstract

Traditional verbal theories in educational psychology often remain underspecified at the mechanistic level. While they offer rich descriptive constructs and conceptual insights, they provide limited accounts of how learning processes unfold dynamically and causally. Here, “verbal” denotes theories stated in prose and qualitative relations rather than as formal, computational process models. This lack of mechanistic precision constrains rigorous theory testing, limits integration across levels of analysis, and reduces the potential to design interventions grounded in explanatory understanding of learning processes. In this Review, we synthesize an emerging paradigm that treats artificial intelligence (AI) systems not merely as predictive tools or instructional technologies, but as cognitive models of learners, explicit, runnable instantiations of theoretical assumptions about cognition and learning. Our central question is not whether AI systems can serve as cognitive models, but when they should be allowed to count as such. We therefore organize the Review around explicit validity criteria, theoretical grounding, construct validity, mechanistic transparency, alignment with human learning trajectories, error-signature matching, causal-intervention tests, ecological validity, and instructional usefulness, that an AI system must satisfy before its cognitive-model status is granted rather than assumed. We examine how major families of AI models, including neural networks, reinforcement learning agents, cognitive architectures, and large language models, have been used to operationalize core educational constructs such as memory, strategy use, motivation, self-regulation, and social learning. Across these approaches, we highlight how mechanistic transparency, interpretability, and alignment with human learning trajectories and error patterns are essential for explanatory validity. We further discuss methodological tools, such as representation analysis, ablation, and trajectory-level comparison, that enable causal inference about learning mechanisms within models. Finally, we outline key challenges and future directions, including construct validity, ecological realism, individual differences, and ethical accountability. By positioning AI as a theoretical instrument rather than solely an engineering solution, this Review argues that AI-based cognitive models can, when they satisfy these criteria, help transform abstract learning theories into precise, testable, and educationally actionable accounts of how students learn.

Research Alert: Combining Psychology With Artificial Intelligence: What Could Possibly Go Wrong?

Quick email-based Research Alert FYI.  This is an open access article👍

 

Combining Psychology With Artificial Intelligence: What Could Possibly Go Wrong? - Iris van Rooij, Olivia Guest, 2026 
https://journals.sagepub.com/doi/10.1177/09637214261438379

 

Abstract

The current AI hype cycle combined with psychology’s various crises make for a perfect storm. Psychology, on the one hand, has a history of weak theoretical foundations, a neglect for computational and formal skills, and a hyperempiricist privileging of experimental tasks and testing for effects. Artificial intelligence, on the other hand, has a history of conflating artifacts for theories of cognition, or even minds themselves, and its engineering offspring likes to move fast and break things. Many of our contemporaries now want to combine the worst of these two worlds. What could possibly go wrong? Quite a lot. Does this mean that psychology and artificial intelligence can best part ways? Not at all. There are very fruitful ways in which the two disciplines can interact and theoretically contribute to cognitive science, for instance, by studying the scope and limits of computational models of human cognition. But to reap the fruits, one needs to understand how to steer clear of potential traps.

Click on images to enlarge

Research Alert: Generation Intelligence (Gen I): A Five-Intelligence Framework for Understanding Generational Cognitive, Emotional, Social, Spiritual, and AI Readiness Profiles

Interesting food for thought.

PDF copy of article available here at Research Gate.

Abstract

Standard generational analysis focuses on birth-year cohorts and broad demographic patterns. This paper introduces Generation Intelligence (Gen I), a multidimensional framework that refocuses the analytical lens on the formative window of ages 8 to 12, the period of peak neuroplasticity, social identity formation, and communication technology imprinting. Across six living generations (Silent, Baby Boomer, Generation X, Millennial, Generation Z, and Generation Alpha), the framework applies five intelligence dimensions, Cognitive Intelligence (IQ), Emotional Intelligence (EQ), Social Intelligence (SQ), Spiritual Intelligence (SpQ), and AI Readiness (AQ),to produce a generational intelligence matrix. The framework synthesizes established developmental psychology, the Strauss-Howe generational cycle theory, and Pew Research Center longitudinal data with emerging research on digital media's cognitive effects and AI's developmental implications. Findings suggest that each generation's distinctive intelligence profile is predictable from its formative communication environment, and that the full intelligence spectrum, not any single generation's contribution alone, is required to address the complex challenges of the 21st century. Practical implications for education, organizational leadership, human development practice, and AI governance are discussed.

Keywords: generational intelligence, emotional intelligence, spiritual intelligence, AI readiness,  formative development, neuroplasticity, intergenerational leadership, Gen I.

Click on image to enlarge for easy reading

AI Brief: Aligning AI agents with developmentally appropriate CHC (select) cognitive domains—a thought provoking AI approach for school-age children and youth

 The source article is an arXiv open-access article available here.  I have no idea if it has undergone peer-review and is a pre-print or if it is working draft paper.  

Click on inserted images to enlarge for easier viewing.

Dr. Kevin McGrew with major assist from Google NotebookLM.  (Click here for brief explanation of how IQs Corner creates AI Alerts from article PDFs)

ChildAgentEval, a psychometrically grounded benchmark designed to evaluate how well artificial intelligence aligns with the cognitive development of children and youth. Inspired by the Wechsler Intelligence Scale for Children-IV four-factor CHC model (the study does not use actual WISC-IV items as per test security standards), this framework assesses multimodal AI agents across ten interactive subtests covering the CHC domains like working memory (Gwm), verbal abstraction and vocabulary (Gc), fluid-visual spatial reasoning (Gf/Gv), and processing speed (Gs).

The study reveals that standard prompting—simply asking an AI to "act like a child"—does not authentically replicate pediatric developmental cognition, as models often maintain adult-level reasoning. The authors introduced a skill-guided distillation strategy that applies data-driven filters to simulate age-appropriate cognitive constraints and limitations. While AI agents can successfully adapt their vocabulary, the research highlights significant challenges in mimicking human-like bottlenecks in perceptual processing and memory retention. 

ChildAgentEval is the first psychometrically grounded interactive benchmark designed to measure cognitive age alignment in multimodal large language model (MLLM) agents. The study addresses a critical gap in current AI development: while state-of-the-art agents excel at complex reasoning, they often do not scaffold learning within a child's Zone of Proximal Development, often using adult-level abstractions that exceed a young user's cognitive grasp. The authors advocate for a shift in AI development from maximizing raw capability to ensuring developmental appropriateness for young users (e.g., school-age children and youth)—a feature that is much needed if AI is to be used in with school-age children and youth. 

The ChildAgentEval Framework

Inspired by the Wechsler Intelligence Scale for Children (WISC-IV), the benchmark evaluates agents across ten interactive subtests mapped to four primary CHC cognitive factors:

• Crystallized Intelligence (Gc): Verbal abstraction and vocabulary.
• Fluid and Visual-Spatial Reasoning (Gf/Gv): Rule induction and spatial problem-solving.
• Working Memory (Gwm): Information retention and manipulation.
• Processing Speed (Gs): Quickness of visual scanning and timed execution.

Unlike static evaluations, ChildAgentEval utilizes a Playwright-driven browser environment where agents must perform physical actions like clicking and typing to solve tasks. The framework also implements clinical protocols such as reversal and discontinuation rules to ensure developmental validity.

Skill-Guided Distillation

A core contribution of the research is a data-driven skill distillation strategy that moves beyond simple "act like a child" prompts. By analyzing a multi-source corpus of real child and adolescent interactions (ages 6–17), the researchers developed cognitive profile vectors. These vectors are converted into executable constraints via five cognitive filter modules injected into the agent's prompt, memory, and reasoning layers:

1. Vocabulary Abstraction Filter: Limits academic concepts and controls syntactic complexity.
2. Working Memory Mask: Physically simulates shorter memory spans by injecting noise or restricting cross-page information.
3. Reasoning Budget Controller: Restricts the depth of multi-step logic.
4. Visual Reliance Module: Reproduces cognitive biases, such as being misled by physical arrangement illusions.
5. Social Perspective Filter: Restricts reasoning to age-appropriate viewpoints, such as first-person vs. institutional perspectives.

Key Findings and Experimental Results

The study evaluated several proprietary models (e.g., GPT-5.4, Gemini-3.1-Pro) and open-weight models (e.g., Qwen3.5-27B). The experiments revealed three major insights:

• Standard Prompting Fails: Merely asking an agent to "act younger" does not reliably change its underlying cognitive behavior; most models continue to maximize correctness regardless of the requested age.
• Skill Guidance Enables Alignment: In high-performing proprietary models, the distillation method successfully induced monotonic score trajectories, where performance expanded naturally as the target age increased—the results produced performances associated with developmental growth curves of cognitive abilities.
• Uneven Domain Alignment: While agents easily adapted their linguistic style (Gc), they struggled to authentically simulate human-like limits in working memory and perceptual reasoning. This "domain dissociation" suggests that MLLM architectures currently lack the structural developmental bottlenecks found in biological cognition.

Conclusion

The authors conclude that for sensitive applications like educational tutoring, technical correctness must be secondary to developmental appropriateness. ChildAgentEval establishes a new paradigm for AI safety and alignment, shifting the focus from maximizing raw capability to authentic cognitive simulation.

What is an “AI Brief” at IQs Corner?

I am currently working on expanding my skill set by incorporating AI tools. Although adapting to new technologies can be challenging, leveraging these resources offers significant benefits for professional growth.

AI Brief’s, at IQs Corner, are produced by requesting Google NotebookLM to generate a narrative summary of one or more PDF journal articles.  Google NotebookLM takes an uploaded article and with the prompt to “write a narrative summary of the article.”  While I find the first draft promising, I do fine the need to make corrections, add important links and missing information, and do more editing to enhance the briefs accuracy and informativeness. Future plans include using this AI tool to summarize multiple articles, find similarities and differences between the articles, and create comparative tables.

These incremental steps mark my transition toward utilizing AI to support one of my primary professional interests: producing informative blog and social media posts aimed at professionals such as school psychologists and special education teachers working with students who often are marginalized in educational contexts. The goal is to help bridge the gap between theory, technology, research, and practical application.

 

Feedback is encouraged and may be directed to iqmcgrew@gmail.com or via the social media platform (LinkedIn, Twitter/X, BlueSky) comment feature where this blog post was discovered. I’m hoping to add AI Briefs as a regular feature of IQs Corner Blog and associated social media platforms.

Research alert: Cognitive dark matter: Measuring what AI misses - #AI #intelligence #cognitive

 

Link to open access copy of article here

Abstract

We propose that the jagged intelligence landscape of modern AI systems arises from a missing training signal that we call “cognitive dark matter” (CDM): brain functions that meaningfully shape behavior yet are hard to infer from behavior alone. We identify key CDM domains—metacognition, cognitive flexibility, episodic memory, lifelong learning, abductive reasoning, social and common-sense reasoning, and emotional intelligence—and present evidence that current AI benchmarks and large-scale neuroscience datasets are both heavily skewed toward already-mastered capabilities, with CDM-loaded functions largely unmeasured. We then outline a research program centered on three complementary data types designed to surface CDM for model training: (i) latent variables from large-scale cognitive models, (ii) process-tracing data such as eye-tracking and think-aloud protocols, and (iii) paired neural–behavioral data. These data will enable AI training on cognitive process rather than behavioral outcome alone, producing models with more general, less jagged intelligence. As a dual benefit, the same data will advance our understanding of human intelligence itself.

Research alert: Toward Complementary Intelligence: Integrating Cognitive and Machine AI - #AI #machineAI #cognitiveAI #cognition #intelligence #schoolpsychologiy

In Current Directions in Psychological Science 

Unfortunately not open access.

Abstract

This article calls for complementary human-AI intelligence. Rather than redefining intelligence to fit machine capabilities, we argue for designing AI that complements and extends human cognition. We distinguish between cognitive AI, which is grounded in cognitive science to model human perception, learning, and decision-making, and machine AI, which achieves large-scale performance through data-driven optimization. Building on advances in machine learning alignment and human-AI complementarity, we propose an integrative framework that connects cognitive and machine AI across four routes: embedding integration, aligning human and machine representations; instruction encoding, using machine AI to translate goals into cognitive AI; training agents, using cognitive AI to guide and train machine AI through human-like data; and coevolving agents, enabling cognitive and machine AI to coadapt and improve together over time. These integration routes provide a foundation for complementary intelligence: systems that combine human interpretability with machine scalability and precision to enhance trust, adaptability, and human agency in complex sociotechnical environments.

Article Alert: AI tools for systematic literature reviews and meta-analyses in educational psychology: An overview and a practical guide - #AI #reviews #metaanalysis#EDPSY

 Click on images to enlarge for easy reading

This is an open access article that can be read or downloaded by clicking here.

A B S T R A C T

The rapid growth of research literature has made systematic reviews and meta-analyses increasingly time-consuming, limiting their utility in fast-evolving fields such as educational psychology. Artificial intelligence (AI) tools have enormous potential to streamline these processes, yet their adoption remains limited due to usability issues and a lack of systematic guidance. Out of 282 tools that we compiled from overviews that listed AI tools for research syntheses, we filtered a subset of 7 AI tools that met quality standards, such as transparency and accessibility. These tools were evaluated for their potential to support systematic reviews and meta-analyses in educational psychology. Our review highlights the tools' strengths, limitations, and ethical considerations for their responsible use by providing practical guidance and coding information. Educational relevance statement: This research identifies and evaluates AI tools that streamline systematic reviews and meta-analyses, addressing critical challenges in synthesizing educational psychology research. By making these processes more efficient, accessible, and accurate, the study empowers educators and researchers to derive timely insights into diverse learner needs. Practically, the findings guide the adoption of AI tools that reduce workload and cognitive bias, enabling more evidence-based and inclusive educational practices. This work supports the advancement of scientifically rigorous methods that enhance understanding of individual differences in learning, directly contributing to improved educational interventions and outcomes

IQ McGrew’s Recommended Reading: How Human Personality [and Intelligence] Will [May?] Change With the Use of Artificial Intelligence - #recrdg #personality #intelligence #AI #CHC #artificialintellignece #psychology #schoolpsychology #schoolpsychologists

Click on images to enlarge for easier reading

I seldom designate an article as a recommended reading. I typically make FYI posts about new research I finding interesting in my small corner of the larger sandbox of psychology…more as FYI alerts.  I break with my typical FYI research alert blogging behavior for this article by Dr. John D. Mayer.  I recommend reading Mayer’s thought provoking article—especially since it is open source and can be downloaded and read for free (click here to access).

Why?  Because it is a well-reasoned “thought piece” about the many unanswered questions regarding the potential positive and negative impact of AI on humans, in this case, human personality and cognition.  I’m relatively new to the fast-moving AI movement and, as an educational psychologist, I’m interested in how certain cognitive abilities (especially CHC cognitive abilities) may become “skilled” or “deskilled” with greater reliance on AI.  

Abstract

People change as they form new habits, encounter new situations, and mature. As people interact with artificial intelligence (AI), their personalities will change, including their emotional responses to AI, their cognition, and their self-understanding. The present theoretical integration draws together empirical studies of how personality changes in response to technological innovations, and to AI in particular. Research studies reviewed were selected according to their relevance and quality. Some key points include that (a) as AI becomes increasingly human-like, and humans represent themselves online, humans and bots become increasingly difficult to distinguish; (b) as people rely on AI as a coach to guide them in interpersonal interactions, they may become socially deskilled; and, (c) as they rely on AI for work tasks, they may become cognitively deskilled in key areas. These changes in personality will entail an overall shift in people’s self-concepts. Psychologists can track these changes by classifying people’s types of AI interactions and relating them to relevant personality attributes.

Effects of artificial #intelligence (#AI) on #educational functioning: A review and #metaanalysis — #EDPSY #schoolpsychologists #schoolpsychology #cognition

Link to journal

Abstract

 

Burgeoning integration of AI into educational settings could have profound implications for students’ performance. This systematic review and meta-analysis examined the effects of different types of AI and four levels of learning—cognition, knowledge utilization, meta-cognition, and psychological functioning—yielding 228 studies with 464 effect sizes that met criteria for inclusion. AI had large positive effects on cognition, r = 0.530, p < 0.001 [95%CI:0.447 to 0.613] and psychological functioning, r = 0.514, p < 0.001 [95%CI:0.246 to 0.720], moderate effects on knowledge utilization, r = 0.417, p < 0.001 [95%CI:0.305 to 0.747], and small effects on meta-cognition, r = 0.268, p = 0.21 [95%CI:-0.225 to 0.772]. Different types of AI had different effects on cognition, with generative AI demonstrating the largest effects, which were larger than other types of AI (e.g., intelligent tutoring, adaptive/personalized learning). However, different types of AI had comparable moderate effects in bolstering knowledge utilization and psychological functioning. AI had the largest effects on improving learning in arts and humanities. Analyses provided evidence for differential impact of AI on learning across countries with different economic advancement. The findings suggest that AI can be effective at improving learning under certain conditions and that the effectiveness varies with the type of AI.

IQs Corner pub alert: CHC theory of cognitive abilities used to define and evaluate AI - #AI #CHC #intelligence #schoolpsychology #schoolpsychologists #IQ #EDPSY

An exciting new paper from the Dan Hendryks et al. at the Center for AI Safety  The center is  a nonprofit with the mission “to reduce societal-scale risks from artificial intelligence.”  In this just released paper, they propose a modified CHC theory definition/framework for evaluating AI: 

• “AGI is an AI that can match or exceed the cognitive versatility and proficiency of a well-educated adult.”

Given my extensive research and publications regarding the Cattell-Horn-Carroll (CHC) theory of cognitive abilities, I was pleasantly surprised when Dan reached out for my comments and suggested revisions to the paper.  

I was extremely impressed as Dan and his group had been involved in a deep dive in the CHC literature and had developed, without my involvement, an ingenuous internet-based CHC set of “test” items that can be submitted to different AI agents (GPT-4, GPT-5, Grok) to assess their CHC broad ability domain performance (to evaluate the extent to which AI agents demonstrate the “cognitive versatility and proficiency of a well-educated adult”).  I had zero involvement in the conceptualization or development of the AI modified/adapted CHC assessment framework and resulting CHC AI metrics. 

I want to express my appreciation to Dan for including me among the list of over 24 authors.  I’m very excited to monitor future developments by Dan and his group, as well as to see the impact of the CHC theory model on AI.

Links to secure copies of the paper (in various formats and social media platforms) are listed at the bottom of this post.  

Note.  Click on all images to enlarge for easy reading

Abstract

The lack of a concrete definition for Artificial General Intelligence (AGI) obscures the gap between today's specialized AI and human-level cognition. This paper introduces a quantifiable framework to address this, defining AGI as matching the cognitive versatility and proficiency of a well-educated adult. To operationalize this, we ground our methodology in Cattell-Horn-Carroll theory, the most em-pirically validated model of human cognition. The framework dissects general intelligence into ten core cognitive domains—including reasoning, memory, and perception—and adapts established human psychometric batteries to evaluate AI systems. Application of this framework reveals a highly “jagged” cognitive profile in contemporary models. While proficient in knowledge-intensive domains, current AI systems have critical deficits in foundational cognitive machinery, particularly long-term memory storage. The resulting AGI scores (e.g., GPT-4 at 27%, GPT-5 at 58%) concretely quantify both rapid progress and the substantial gap remaining before AGI.

Modified CHC model for evaluating AI agents

Click on image to enlarge.

As mentioned in the abstract, the paper reports on the CHC AGI capabilities of GPT-4 and GPT-5 in the following figure.  Click on images to enlarge.

I was pleased to see (on page 14 of the PDF paper) the following “intelligence as processor” figure which is based on work by myself and Joel Schneider. The model in Figure 3 (below) is based on Kevin S. McGrew and W. Joel Schneider. CHC theory revised: A visual-graphic summary of Schneider and McGrew's 2018 CHC update chapter. MindHub / IAPsych working paper, 2018.  http://www.iapsych.com/mindhubpub4.pdf.  

The Schneider & McGrew (2018) heuristic CHC information processing model is below the Figure 3 figure.  Click on images to enlarge.

Dan Hendrycks and the Center AI Safety provide brief overviews describing this work on LinkedIn as well as Twitter/X (both that can be monitored for comments).

A PDF copy of the paper can be downloaded here.  A clickable web-based version of the paper can be accessed here.

Exciting stuff !!