An exciting new paper from the Dan Hendryks et al. at the Center for AI Safety The center is a nonprofit with the mission “to reduce societal-scale risks from artificial intelligence.” In this just released paper, they propose a modified CHC theory definition/framework for evaluating AI:
- “AGI is an AI that can match or exceed the cognitive versatility and proficiency of a well-educated adult.”
Given my extensive research and publications regarding the Cattell-Horn-Carroll (CHC) theory of cognitive abilities, I was pleasantly surprised when Dan reached out for my comments and suggested revisions to the paper.
I was extremely impressed as Dan and his group had been involved in a deep dive in the CHC literature and had developed, without my involvement, an ingenuous internet-based CHC set of “test” items that can be submitted to different AI agents (GPT-4, GPT-5, Grok) to assess their CHC broad ability domain performance (to evaluate the extent to which AI agents demonstrate the “cognitive versatility and proficiency of a well-educated adult”). I had zero involvement in the conceptualization or development of the AI modified/adapted CHC assessment framework and resulting CHC AI metrics.
I want to express my appreciation to Dan for including me among the list of over 24 authors. I’m very excited to monitor future developments by Dan and his group, as well as to see the impact of the CHC theory model on AI.
Links to secure copies of the paper (in various formats and social media platforms) are listed at the bottom of this post.
Note. Click on all images to enlarge for easy reading
Abstract
The lack of a concrete definition for Artificial General Intelligence (AGI) obscures the gap between today's specialized AI and human-level cognition. This paper introduces a quantifiable framework to address this, defining AGI as matching the cognitive versatility and proficiency of a well-educated adult. To operationalize this, we ground our methodology in Cattell-Horn-Carroll theory, the most em-pirically validated model of human cognition. The framework dissects general intelligence into ten core cognitive domains—including reasoning, memory, and perception—and adapts established human psychometric batteries to evaluate AI systems. Application of this framework reveals a highly “jagged” cognitive profile in contemporary models. While proficient in knowledge-intensive domains, current AI systems have critical deficits in foundational cognitive machinery, particularly long-term memory storage. The resulting AGI scores (e.g., GPT-4 at 27%, GPT-5 at 58%) concretely quantify both rapid progress and the substantial gap remaining before AGI.
Modified CHC model for evaluating AI agents
Click on image to enlarge.
As mentioned in the abstract, the paper reports on the CHC AGI capabilities of GPT-4 and GPT-5 in the following figure. Click on images to enlarge.
I was pleased to see (on page 14 of the PDF paper) the following “intelligence as processor” figure which is based on work by myself and Joel Schneider. The model in Figure 3 (below) is based on Kevin S. McGrew and W. Joel Schneider. CHC theory revised: A visual-graphic summary of Schneider and McGrew's 2018 CHC update chapter. MindHub / IAPsych working paper, 2018. http://www.iapsych.com/mindhubpub4.pdf.
The Schneider & McGrew (2018) heuristic CHC information processing model is below the Figure 3 figure. Click on images to enlarge.
Dan Hendrycks and the Center AI Safety provide brief overviews describing this work on LinkedIn as well as Twitter/X (both that can be monitored for comments).
A PDF copy of the paper can be downloaded here. A clickable web-based version of the paper can be accessed here.
Exciting stuff !!
