What is talent – and can science spot what we will be best at?

An excellent article by my friend and colleague Scott Barry Kaufman

http://gu.com/p/3h3b3/sbl

Content curated by IQs Corner and the Mind Hub

The Motivation and Academic Competence (MACM) Commitment Pathway to Learning Model: Crossing the Rubicon to Learning Action

[2-24-13.  Since this original  blog post appeared, I have received requests for printed copies.  To meet this request I have published this post as the first MindHub (TM) Pub. This publication can be downloaded here.]

 

There is only one unequivocal law of human behavior—the law of individual differences.  People are more different than they are alike. Probably no environment elicits individual differences sooner in life than formal education.
 
When asked by teachers or parents to help understand why a particular student is not achieving adequately, school psychologists have traditionally reached for an intelligence battery.  Although understanding a student’s general, broad and specific cognitive abilities contributes important information for determining general expectations and the need for special instructional serves, at best, measures of cognitive abilities account for only approximately 40% to 50% of a student’s predicted achievement.  Much is still unexplained.  Furthermore, attempts at modifying intelligence, or identifying evidence-based cognitive-aptitude-achievement interactions (ATI's) that can be implemented at the level of individual students, have not yet provided the magical link between cognitive ability testing and evidence-based instructional or cognitive modifiability recommendations.  It is clear that school psychologists must go “beyond IQ” to help teachers, parents, and students themselves, to maximize student learning.
    
But…if not IQ…then what?  The more appropriate question is “what should be added to cognitive ability assessment information to help school psychologists facilitate the achievement of all learners?”  To provide some answers to this question this paper was developed with three primary goals.  First, a conceptual framework is presented to help school psychologists better understand the salient non-cognitive individual difference student variables to consider when engaging in learning-related assessments and instructional planning.  Second, the primary domains of the model are defined.  Finally, how the various domains work within a commitment pathway model to learning (crossing the active learning rubicon) is briefly presented.

Beyond IQ:  What Models of School Learning Have Told Us

A number of comprehensive models of school learning have been advanced to describe and explain the school learning process (see McGrew, Johnson, Cosio, & Evans, 2004).  Walberg's (1981) theory of educational productivity is one of the few empirically tested theories of school learning.  Walberg's model is based on an extensive review and integration of over 3,000 studies (DiPerna, Volpe & Stephen, 2002; Wang, Haertel, and Walberg, 1997).  Walberg et al. reported that the following key variables are important for understanding school learning—student ability and prior achievement, motivation, age or developmental level, quantity of instruction, quality of instruction, classroom climate, home environment, peer group, and exposure to mass media outside of school (Walberg, Fraser & Welch, 1986).  The first three variables (ability, motivation, and age) reflect student individual difference characteristics.  The fourth and fifth variables reflect characteristics of instruction  (quantity and quality), and the final four variables (classroom climate, home environment, peer group, and exposure to media) represent aspects of the psychological environment (DiPerna et al., 2002).  Clearly student characteristics are important for school learning, but they only comprise a portion of the complete learning equation.

The Walberg research group (see Wang, Haertel, & Walberg, 1993) also concluded that psychological, instructional, and home environment characteristics (proximal variables) had a more significant impact on achievement than variables such as state-, district-, or school-level policy and demographics (distal variables).  More important for practicing school psychologists was the conclusion that student characteristics (i.e., social, behavioral, motivational, affective, cognitive, metacognitive) were the set of proximal variables that had the most significant impact on learner outcomes (DiPerna et al., 2002).

Beyond IQ:  The Need for a Non-Cognitive Learner Characteristic Taxonomy

If school psychologists are to focus on the most learning-relevant student characteristics (beyond cognitive abilities), what individual difference student characteristic domains should receive priority?  Even a partial list of potentially important non-cognitive domains mentioned in the school psychology literature is staggering.  Social-emotional learning.  Motivation.  Self-efficacy.  Engagement.  Study and homework skills.  Resilience.  Executive functions.  Engaged learning time.  Self-regulated learning strategies.  Social skills.   Social and emotional intelligence.  What are the similarities and differences between these different constructs?  Does each construct consist of a single dimension or is there a complex model of subdomain characteristics within each construct domain?  Where is a school psychologist to start?   It my opinion that the answer first lies in outlining a working taxonomy of important non-cognitive learning-related student characteristics.

I am an admitted taxonomist.  As stated in the context of human cognitive abilities, Joel Schneider and I stated “A useful classification system shapes how we view complex phenomena by illuminating consequential distinctions and obscuring trivial differences. A misspecified classification system orients us toward the irrelevant and distracts us from taking productive action. Imagine if we had to use astrological classification systems for personnel selection, college admissions, jury selection, or clinical diagnosis. The scale of inefficiency, inaccuracy, and injustice that would ensue boggles the mind.  Classification is serious business” (Schneider & McGrew, 2012, p. 99).
 
I believe that before defining and articulating instructional implications of important non-cognitive student characteristics, the broad domain(s) must first be circumscribed.  Furthermore, I believe that any working taxonomy must emerge from the extant empirical and theoretical literature, and not from the advocacy, policy, political arenas or narrow single trait programs of research.  Although a variety of models of school learning have been articulated, it is only recently that a model with sufficient breadth and depth, grounded in decades of educational and psychological research, has emerged with the potential to serve as a “bridging” mechanism between educational and psychological theory/research and educational practice.

Based on a large systematic program of educational research and research integration, Richard Snow and colleagues outlined a provisional and promising aptitude for learning taxonomy (Corno et al., 2002; Snow, Corno, & Jackson, 1996).  Richard Snow’s work unfortunately has flown under the radar screen of most of school psychology.  It is hoped that this brief paper rectifies this oversight by describing a Snow-inspired framework for understanding the most salient non-cognitive student characteristics that influence school learning.

A first attempt at outlining an adapted and updated Snow model, based on a comprehensive review and integration of approximately three decades of contemporary school learning research, was first described by McGrew et al. (2004).  This model was next revised as the Model of Academic Competence and Motivation (McGrew, 2007).   Figure 1 presents the  revision and update of the McGrew 2007 MACM model.

[click on image to enlarge]

The Model of Academic Competence and Motivation (MACM):  A Brief Overview

 The MACM model includes the three broad domains of orientations towards self (motivations), volitional controls (cognitive strategies and styles), and orientations towards others (social ability). [1]   As illustrated in Figure 1, the current focus is on the motivational and volitional domains of conation.  The term conative, as well as volition, may partially explain why Snow and colleagues work has not been widely infused into education and school psychology.  Conative and volition are not commonly used terms in education or psychology and, frankly, would result in puzzled looks from teachers and parents if used to describe characteristics of a student.  However, they are important and have been part of a long standing “ancient trilogy of human mental functioning that consists of cognition, affection and conation” (Corno, 1996, p.14, italics added).  This current paper seeks to amplify the importance of conative abilities as articulated by many giants in the field of intelligence theory and testing.

PDF files that contain detailed definitions of the MACM characteristics and theoretical foundations can be found here and here.

Conative abilities have long been recognized as the important brides made-trait to cognition when attempting to explain intelligent performance or behavior.  The APA Dictionary of Psychology (Vandenbos, 2007) defines conation as “the proactive (as opposed to habitual) part of motivation that connects knowledge, affect, drives, desires, and instincts to behavior.  Along with COGNITION and affect, conation is one of the three traditionally identified components of mind” (p. 210; caps in original).  Charles Spearman, who all psychologists associate with the birth of the psychometric study of intelligence, recognized the importance of conative abilities.  Spearman (1927) stated that “the process of cognition cannot possibly be treated apart from those of conation and affection, seeing that all these are but inseparable aspects in the instincts and behavior of a single individual, who himself, as the very name implies, is essential indivisible” (p. 2).  Alfred Binet, who is considered the father of the modern day intelligence test, also recognized the importance of “non-intellectual” factors in cognitive or intellectual performance.  According to Corno et al. (2002):

• Binet summed up his investigations in a famous description of intelligence: ‘the tendency to take and maintain a definite direction; the capacity to make adaptations for the purpose of attaining a desired end; and the power of auto-criticism’ (translation by Terman, 1916, p. 45).  All three of these phrases refer at least as much to conative processes and attitudes as to reasoning powers. Binet's concept of intelligence was much like Snow's concept of aptitudes (p. 5).

Sounding a similar chord, David Wechsler emphasized the importance of conative abilities, which he referred to as nonintellectual factors (e.g., persistence, curiosity, and motivation) (Zachary, 1990).  In Wechsler’s (1994) own words, “When our scales measure the nonintellectual as well as intellectual factors in intelligence, they will more nearly measure what in actual life corresponds to intelligent behavior” (Wechsler, 1944, p. 103).  More recently Richard Woodcock, first author of the WJ, WJ-R and WJ III, in his Cognitive Performance and Information Processing Models, includes the facilitator-inhibitor domain that includes both internal conative-like characteristics (e.g., health, attention and concentration, cognitive style), along with external variables (e.g., environmental distractions) that can “modify cognitive performance for better or for worse, often overriding the effects of strengths and weaknesses in the previously described cognitive abilities” (Woodcock, 1998, p. 146).

I humbly stand on the shoulders of Spearman, Binet, Wechsler, Woodcock and Snow  and recommend that school psychologists organize their thinking regarding essential student learning characteristics within a model of student competence and achievement that recognizes the importance of conative abilities.  To remove the terminology barrier to implementing this recommendation,  conative abilities have been renamed as motivations (orientations towards self) and cognitive strategies and styles (volitional controls; see Figure 1).  Being even more direct and simple, I have modified and extended the key question approach to understanding achievement motivation as presented by Wigfield and Eccles (2002).  The major domains represented in the MACM model (see Figure 1) can be reduced to five basic questions (borrowed and revised from Wigfield & Eccles, 2002) school psychologists should ask as they gather and integrate information regarding important non-cognitive school learning-related student information.

• Does the student think they can do the task?  This question focuses on understanding the student’s self-beliefs regarding their perceived locus of control, academic self-efficacy, academic self-concept, and ability conception.
• Does the student want to do the task and for what reasons?  When pondering this question, the goal is to understand the student’s motivational orientations such as degree of academic and intrinsic motivation, type of goal orientation, and the students’ goal setting abilities.  Additionally, understanding how a student values school learning and their global and situational academic domain-specific academic interests should be considered.
• What does the student need to do to succeed on the task?  High motivation and positive self-beliefs are necessary but not sufficient conditions for succeeding in educational environments.  A bridge must link abilities, self-beliefs and motivation with action-oriented behavior.  The bridge is the presence of motivational controls or self-regulated learning strategies (e.g., study skills, cognitive and learning strategies, engagement) that allow individuals to manage efforts to accomplish their goal.
• What are the student’s typical ways of responding to the task?  This question focuses on determining if a student has characteristic stable styles for approaching learning tasks, success or failure (e.g., self-worth protection; adaptive help-seeking) that either need to be enhanced or modified to insure increased positive achievement outcomes. 
• How does the student need to behave towards others to succeed on the task?  Traditionally U.S. schools have valued student characteristics such as citizenship, conformity to social rules and norms, cooperation, and positive social behavior.  The student who does not know how (or who lacks the appropriate skills) to behave appropriately and responsibly is at increased risk for academic failure and the possibility of not developing a sense of belonging or relatedness. 

The MACM Framework and the Commitment Pathway to Learning Model:  Crossing the Rubicon to Learning Action

There is no consensus explanatory model outlining how the various constructs included in Figure 1 interact within the MACM model or with other important learner characteristics (e.g., cognitive abilities) to produce positive achievement outcomes.  In their introduction to the Handbook of Competence and Motivation, a seminal attempt to corral the major theories and research regarding motivation, self-regulatory processes and competence, Elliot and Dweck (2005) summarize this state of affairs when they stated, with regard to the weaknesses in the achievement motivation literature:

• The literature lacks coherence and a clear set of structural parameters, and the literature is too narrowly focused and limited in scope.  In essence, what is commonly referred to as the “achievement motivation literature” represents a rather loose compendium of theoretical and empirical work focused on a colloquial understanding of the term “achievement” (p. 5). 

Further illustrating the impossible task of specifying a single consensus explanatory or causal MACM-achievement model is a recent series of reports from the Center on Education Policy (CEP)--Student Motivation:  School Reform's Missing Ingredient (Usher & Kober, 2012a).  No less than eight different expert or theoretical views of the dimensions of student motivation (which represents only the motivations component of the MACM model in Figure1) were the basis of the CEP’s series of six different policy briefs (Usher & Kober, 2012b).

Not only is the number of proposed explanatory models of achievement motivation a barrier to incorporating the MACM learner characteristics into school psychology practice, the complexity of some of the models is not practice friendly.  For example, the general expectancy-value model of achievement choices includes 11 separate model components (each with from 1 to 5 subcomponents) and over 12 different unidirectional or bidirectional arrows between components (Eccles, 2005).  A model focused just on evaluation anxiety during self-regulation includes 5 model components and 9 unidirectional arrows, while a proposed model of self-handicapping, which is just one self-protection style (i.e., a conative style in the MACM model—see Figure 1) is a figure with 11 different components and 13 different arrows (Rhodewalt & Vohs, 2005).  Finally, the MACM model includes two goal-related constructs (academic goal orientation and academic goal setting—see Motivational Orientations component in Figure 1), while a 1996 review of goal constructs in psychology listed 31 theories that have posited goal-like constructs and proposed a 6 domain, 24 subdomain taxonomy of human goals (Austin & Vancouver, 1996).
 
Against this backdrop, a simplified adaptation of Snow’s dynamic model of conation in the academic domain (Corno, 1993) is presented next.  It is assumed that the presentation of a simplified model is a first step towards helping school psychologists see the forrest-from-the-trees and thus, increases the chances of successful integration of the  MACM concepts in their assessment and instructional planning repertoire.  The MACM-based adaptation and extension of Snow’s model is presented in Figure 2.    

[click on slide to enlarge]

In simple terms, a three-stage process is at the heart of understanding the learner’s commitment pathway to learning and achievement.  Learners first address the questions of “can I do this task?” and “do I want to do this task and why?”  These questions reflect the learner contemplating or deliberating over their beliefs regarding what they can do, what they want to do or are being asked to do, and what intentions they form (positive or negative) regarding how to proceed.  Cleary et al. (2010) describe this as the forethought stage, or those processes that occur before the student commits to the learning task.  For example, a student with a strong interest in science and a mastery goal orientation (i.e., wanting to learn for the sake of learning and mastering new skills) would likely decide to deploy strong and sustained engagement and effort on a science project.  Conversely, a learner with a long history of academic failure may not feel they are capable (low self-efficacy) and may want to avoid failure as their primary goal, which would result in a different decision—a different degree of commitment to a science project and possible deployment of self-protection conative style behaviors.  The act of committing to a course of action for a task has metaphorically being called “crossing the Rubicon” (Corno, 1993; Corno et al., 2002).   Once committed to implementing a plan, the success of the student attaining their goals is turned over to their self-regulated learning strategies (volitional controls)—carrying out the plans and intentions.  The student has moved into the domain addressed by the question “what do I need to do to succeed on the task?”

Of course, this is on overly simple explanation of an obvious non-linear process where the results of plan implementation and self-regulation may require moving back to the contemplation and planning stage if the initials goals require modification (e.g., a student sets an unrealistic goal to perform perfectly on a math project).  Multiple recursive and dynamic iterations occur across the commitment to action pivot point (the Rubicon; see circle of arrows in Figure 1), with motivations modifying cognitive control and regulation strategies, and cognitive strategy feedback requiring goal adjustment and changes in plans, is often required.


Summary Comments

It is hoped that this brief description of the MACM model and the MACM Commitment to Learning Pathway Model (Crossing the Rubicon to Learning Action)stimulates thought, research, and further development.  Updates to this model will be posted at this blog and will typically be accessible by clicking on the MACM and Beyond IQ blog labels on the blog roll.

Reference Notes

Most all references cited in this post can be found at McGrew et al. (2004) and McGrew (2007).

-------------------------------------------------------------------------------------------------------------------

[1] A complete description and discussion of these three primary MACM domains is not possible here. The interested reader should review Corno et al. (2002), and McGrew et al. (2004) and McGrew (2007).  It is important to note that the MACM model is only a partial taxonomy of relevant school-related individual difference characteristics.  The model presented here only lists general categories under the two areas of Social Ability and does not include physical and psychomotor competences, affective or social-emotional characteristics, cognitive abilities, and overarching constructs such as personality, which Corno et al. (2002) include in the more comprehensive big picture taxonomy of aptitude related constructs.  The literature on social intelligence, social cognition, and social skills requires treatment in separate chapters or books.  Social ability is included in the MACM model to reflect an awareness of the importance of social ability and behavior constructs when discussing important non-cognitive characteristics important for school success.

Implications of 20 Years of CHC Cognitive-Achievement Research: Back-to-the-Future and Beyond CHC

[Click image to enlarge]

 

The key slides from my presentation at the first Richard Woodcock Institute on Cognitive Assessment are now posted at SlideShare.  I thought I had posted these before, but I can't seem to find them.  So here they are for the first (or second) time.  Below is the abstract for the paper that I also submitted--to be published eventually by the WMF Press.

Much has been learned about CHC CHC COG-->ACH relations during the past 20 years (McGrew & Wendling’s, 2010).  This paper built on this extant research by first clarifying the definitions of abilities, cognitive abilities, achievement abilities, and aptitudes.  Differences between domain-general and domain-specific CHC predictors of school achievement were defined.   The promise of Kafuman’s “intelligent” intelligence testing approach was illustrated with two approaches to CHC-based selective referral-focused assessment (SRFA).  Next, a number of new intelligent test design (ITD) principles were described and demonstrated via a series of exploratory data analyses that employed a variety of data analytic tools (multiple regression, SEM causal modeling, multidimensional scaling).  The ITD principles and analyses resulted in the proposal to construct developmentally-sensitive CHC-consistent scholastic aptitude clusters, measures that can play an important role in contemporary third method (pattern of strength and weakness) approaches to SLD identification. 

The need to move beyond simplistic conceptualizations of COG COG-->ACH relations and SLD identification models was argued and demonstrated via the presentation and discussion of CHC COG-->ACH causal SEM models.  Another example was the proposal to identify and quantify cognitive-aptitude-achievement trait complexes (CAATCs).  A revision in current PSW third-method SLD models was proposed that would integrate CAATCs.  Finally, the need to incorporate the degree of cognitive complexity of tests and composite scores within CHC domains in the design and organization of intelligence test batteries (to improve the prediction of school achievement) was proposed.  The various proposals presented in this paper represented a mixture of (a) a call to return to old ideas with new methods (Back-to-the-Future) or (b) the embracing of new ideas, concepts and methods that require psychologists to move beyond the confines of the dominant CHC taxonomy of human cognitive abilities (i.e., Beyond CHC).

AP101 Brief # 15: Beyond CHC: Cognitive-Aptitude-Achievement Trait Complex Analysis: Implications for SLD Assessment and Dx

This is the final post in a series of posts clarifying the nature of cognitive, aptitude, achievement ability constructs.  Readers should consult the preceding post (which contains links to all prior background posts) that defined cognitive abilities, aptitudes, achievement abilities, and CHC cognitive-aptitude-achievement trait complexes (CATTCs).  I apologize for not including the reference list.  These posts are snippets of a manuscript in preparation and I like to post to IQs Corner for feedback that I might incorporate in the final manuscript.  References are the last thing I do.

Beyond CHC:  CHC Cognitive-Aptitude-Achievement Trait Complex Analyses

I have previously argued that alternative non-factor analysis methodological (e.g., multidimensional scaling-MDS) and theoretical lenses need to be applied to validated CHC measures to better understand “both the content and processes underlying performance on diverse cognitive tasks” (McGrew, 2005, p. 172).  When MDS “faceted” methods have been applied to data sets previously analyzed by exploratory or confirmatory factor methods, “new insights into the characteristics of tests and constructs previously obscured by the strong statistical machinery of factor analysis emerge.” (Schneider & McGrew, 2012, p. 110).[1]   

Following the methods similar to that explained and demonstrated by Beauducel, Brocke and Liepmann (2001), Beauducel and Kersting (2002), SÜß and Beauducel (2005), Tucker-Drob and Salthouse (2009; this is an awesome example of MDS analyses side-by-side with factor analsysis of the same set of variables) and Wilhelm (2005), I subjected all WJ-R standardization subjects (McGrew, Werder & Woodcock, 1991) who had complete sets of scores (i.e., listwise deletion of missing data) for the WJ-R Broad Cognitive Ability-Extend (BCA-EXT), Reading Aptitude (RAPT), Math Aptitude (MAPT), Written Language Aptitude (WLAPT), Gf-Gc cognitive factors (Gf, Gc, Glr, Gsm, Gv, Ga, Gs), and Broad Reading (BRDG), Broad Math (BMATH), and Broad Written Language (BWLANG) achievement clusters to a Guttman Radex MDS analysis (n = 4,328 subjects from early school years to late adulthood).[2]  MDS procedures have more relaxed assumptions than linear statistical models and allow for the simultaneous analysis of variables that share common variables or tests—a situation that results in non-convergence problems due to excessive multicolinearity when using linear statistical models.  This feature made it possible to explore the degree of similarity of the WJ-R operationalized measures of the constructs of cognitive abilities, general intelligence (g), scholastic aptitudes, and academic achievement, in a single analysis.  That is, it was possible to explore the relations between and among the core elements of CHC-based cognitive-aptitude-achievement trait complexes (CAATC).  The results are presented in Figure 1. [Click on images to enlarge] 

Figure 1 (Click on image to enlarge)

WJ-R MDS Analysis:  Basic Interpretation

In Guttman Radex models, variables closest to the center of the 2-D plots are the most cognitively complex. Also, the variables are located along two continua or dimensions that often have substantive/theoretical interpretations.  The two dimensions in Figure 1 are labeled A<->B and C<->D.  The following is concluded from a review of Figure 1:

--The WJ-R g-measure (BCA-EXT) is almost directly at the center of the plot and is the most cognitive complex variable.  This makes theoretical sense given that it is a composite comprised of 14 tests from 7 of the CHC Gf-Gc cognitive domains.  Proximity to the center of MDS plots is sometimes considered evidence for g.

--Reading and Writing Aptitude (GRWAPT) and MAPT are also cognitively complex.  Both the GRWAPT[3] and MAPT clusters are comprised of four equally weighted tests of four different Gf-Gc abilities—and thus, the finding that they are also among the most cognitively complex WJ-R measures is not surprising.  The CHC Gf-Gc cognitive measures of Gf and Gc are much more cognitively complex that Gv, Glr, Ga and Gsm.[4]

--The A<->B dimension appears to reflect the ordering of variables as per stimulus content, a common finding in MDS analyses.  The cognitive variables on the left-hand side of the continuum midlines (Gv, Glr, Gf, Gs, MAPT) are comprised of measures with predominant visual-figural or numeric/quantitative characteristics.  The majority of the variables on the right-hand of the continuum midline (GRWAPT, Gc, Ga, Gsm, BRDG, BWLANG) are characterized as more auditory-linguistic, language, or verbal.  This visual-figural/numeric/quantitative-to-auditory-linguistic/language/verbal content dimension is very similar to the verbal, figural, and numeric content facets of the Berlin Model of Intelligence Structure (BIS; SÜß and Beauducel, 2005).[5] 

--The C<->D dimension appears to reflect the ordering of variables as per cognitive operations or processes, another common finding in MDS analyses.  The majority of the cognitive variables above the continua midline (Gv, Glr, Ga, Gc, Gsm, BCAEXT, GRWAPT) are comprised primarily of cognitive ability tasks the involve mental processes or operations.  Conversely, although not as consistent, three of the lowest variables below the continua midline are the achievement ability clusters (BRDG BWLANG; BMATH).  Thus, the C<->D dimension is interpreted as representing a cognitive operations/process-to-acquired knowledge/product dimension.

--In contrast to factor analysis, interpretation of MDS is more is more qualitative and subjective.  Variables that may share a common dimension are typically identified as lying on relatively straight lines or planes, in separate quadrants or partitions, or tight groupings (often represented by circles or ovals or connected as a shape via lines).  Inspecting the four quadrants created by the A<->B C<->D dimensions (see Figure 1) suggests the following.  The AC quadrant is interpreted to represent (excluding BCAEXT which is near the center) cognitive operations with visual-figural content (Gv; Glr).  The CB quadrant is interpreted as representing auditory-linguistic/language/verbal content based cognitive operations.  The BC quadrant only includes the three broad achievement clusters, and is thus an achievement or an acquired knowledge dimension.  Finally, the DA quadrant can be interpreted as cognitive operations that involve quantitative operations or numeric stimuli (e.g., Gf is highly correlated with math achievement; McGrew & Wendling, 2010; one-half of the Gs-P cluster is the Visual Matching test which requires the efficient perceptual processing of numeric stimuli—Glr-N).[6]  The interpretation of these four quadrants is very consistent with the BIS content-faceted content-by-operations model research.

--The theoretical interpretation of the two continua and four quadrants provides potentially important insights into the abilities measured by the WJ-R measures.  More importantly, the conclusions provide potentially important theoretical insights into the nature of human intelligence, insights that typically fail to emerge when using factor analysis methods (see Schneider & McGrew, 2012 and SÜß and Beauducel, 2005). In other MDS analyses I have completed, similar visual-figural/numeric/quantitative-to-auditory-linguistic/language/verbal and cognitive operations/process-to-acquired knowledge/product continua dimensions have emerged (McGrew, 2005; Schneider & McGrew, 2012).  When I have investigated a handful of 3-D MDS[7] models the same two dimensions emerge along with a third automatic-to-deliberate/controlled cognitive processing dimension which is consistent with the prominent dual-process models of cognition and neurocognitive functioning (Evans, 2008, 2011; Barrouillet, 2011; Reyna & Brainerd, 2011; Rico & Overton, 2011; Stanovich, West & Toplak, 2011) that are typically distinguished as Type I/II or System I/II  (see Kahneman’s, 2011, highly acclaimed Thinking, Fast and Slow).[8] 

--These higher-order cognitive processing dimensions, which are not present in the CHC taxonomy, suggest that intermediate strata (or dimensions that cut across broad CHC abilities) might be useful additions to the current three-stratum CHC model.  These higher-order dimensions may be capturing the essence of fundamental neurocognitive processes and argue for moving beyond CHC to integrate neurocognitive research to better understand intellectual performance.

WJ-R MDS Analysis:  Cognitive-Aptitude-Achievement Trait Complex (CAATC) Interpretation

Figure 2 is an extension of the results presented in Figure 1.  Two different CAATCs are suggested.  These were identified by starting first with the BMATH and BRDG/BWLANG achievement variables and next connecting these variables to their respective SAPTs (GRWAPT; MAPT).  Next, the closest cognitive Gf-Gc measures that were in the same general linear path were connected (the goal was to find the math and reading related variables that were closest to lying on a straight line).  Ovals encompassing the entire space comprising the two circle-line-circle traces where superimposed on the figure.  A dotted line that represented the approximate bisection of each of the cognitive-aptitude-achievement trait complex vectors was drawn.  Finally, an approximate correlation (r = .55; see Figure 2) between the two multidimensional CAATC was estimated via measurement of the angle between the CAATC vector dotted lines.[9]

Figure 2 (Click on image to enlarge)

As presented in Figure 2, Math and Reading-Writing CAATCs are suggested as a viable perspective from which to view the relations between cognitive abilities, aptitudes, and achievement abilities.  The primary conclusions, insights, and questions are drawn from Figure 1 and 2 are:

--It appears that the potential exists to empirically identify CAATCs via the use of CHC-grounded theory, the extant CHC COG->ACH relations research, and multidimensional scaling.  It also appears possible to estimate the correlation between different trait complexes (see math/reading-writing trait complex r = .55 in Figure 2).  I suggest these preliminary findings may help the field of cognitive-achievement assessment and research better approximate the multidimensional nature of human cognitive abilities, aptitudes, and achievement abilities.

--Although the WJ-R battery is not as comprehensive a measure of CHC abilities as the WJ III, the cognitive abilities within the respective math and reading/writing CAATCs are very consistent with the extant CHC COG->CHC relations research (McGrew & Wendling, 2010; click here for visual-graphic summary).  The reading-writing trait complex (see Figure 2) includes Ga-PC, Gc-LD/VL, and via the GRWAPT, Gs-P, and Gsm-MS, abilities that are listed as domain-general and domain-specific abilities in Figure 3.  In the case of math, the trait complex includes indicators of Gf-RG, Gv-MV, and via the MAPT, Gs-P (Visual Matching, which might also tap Gs-N) and Gc-LD/VL, abilities that are either domain-general or domain-specific for math in Figure 3.  Working memory (Gsm-WM) is not present (as suggested by Figure 3) as the WJ-R battery did not include a working memory cluster that could enter the analysis.

Figure 3 (Click image to enlarge)

--Also of interest are the three WJ-R cognitive factors (Gsm-MS, Glr-MA, Gs-P) that are excluded from the hyperspace representations of the proposed math and reading-writing CAATCs.  Although highly speculative, it may be possible that their separation from the designated trait complexes suggest, that if known to be related to reading-writing or math achievement, their independence from the narrower trait complexes may be an indication that they represent domain-general abilities.  Glr-MA and Gs-P are both listed as domain-general abilities in Figure 3.  Additional work is needed to determine if the independence (from identified CAATCs) of CHC measures known to be significantly related to achievement indicates domain general abilities.  Alternatively, it is very possible, given the previously demonstrated developmental nuances of CHC COG->ACH relations that the results presented in Figures 1 and 2, which used the entire age range of the WJ-R measures, may mask or distort findings in unknown ways.

--Those knowledgeable of the CHC COG->ACH relations research will obviously note the prior inclusion of certain Gv abilities (Vz, SR, MV) in Figure 3 as well as the inclusion of the WJ-R Gv-MV/CS cluster as part of the proposed math CAATC (Figure 2), despite the lack of consistently reported significant CHC Gv-ACH relations.  McGrew and Wendling (2010) recognized that some Gv abilities have clearly been linked to reading and math achievement (especially the later) in non CHC-organized research.  They speculated that the “Gv Mystery” may be due to certain Gv abilities being threshold abilities or that the cognitive batteries included in their review did not include Gv measures that measured complex Gv related Vz or MV processes.  Given this context, it may be an important finding (via the methods described above) that the WJ-R Gv measure is unexpectedly included in the math CAATC.  This may support the importance of Gv abilities in explaining math and concurrently indicate a problem with the operational Gv measures. 

--The long distance from the WJ-R Gv measure to the center of the diagram (see Figure 2) indicates that the WJ-R Gv measure, which included tests classified as indicators of CS and MV, is not cognitively complex.  This conclusion is consistent with Lohman’s seminal review of Gv abilities (Lohman, 1979) where he specifically mentions CS and MV as representing low level Gv processes and “such tests and their factors consistently fall near the periphery of scaling representations, or at the bottom of a hierarchical model” (Lohman, 1979, 126-127).  I advance the hypothesis that the math CAATC in Figure 2 suggests that Gv is a math-relevant domain, but more complex Gv tests (e.g., 3-D mental “mind’s eye” rotation; complex visual working memory), which would be closer to the center of the MDS hyperspace, need to be developed and included in cognitive batteries.  This suggestion is consistent with Wittmann’s concept of Brunswick Symmetry, which, in turn, is founded on the fundamental concept of symmetry which has been central to success in most all branches of science (Wittmann & SÜß, 1999).  The Brunswick Symmetry model argues that in order to maximize prediction or explanation between predictor and criterion variables, one should match the level of cognitive complexity of the variables in both the predictor and criterion space (Hunt, 2011; Wittmann & SÜß, 1999).  The WJ-R Gv-WJ-R BRMATH relation may represent a low (WJ-R Gv)-to-high (WJ-R BMATH) predictor-criterion complexity mismatch, thus dooming any possible significant relation. 

--Researchers and practitioners in the area of SLD should recognize that when third method POSW “aptitude-achievement” discrepancies are evaluated to determine “consistency”, the combination of domain-general and domain-specific abilities that comprise an aptitude for a specific achievement domain in many ways can be considered a mini-proxy for general intelligence (g).  In Figures 1 and 2 the BCA-EXT and MAPT and GRWAPT variables are in close proximity (which also represents high correlation) and are all near the center of the MDS Radex model.  The manifest correlations between the WJ-R BCA-EXT (in the WJ-R data used to generate the CAATCs in Figure 10) and RAPT, WLAPT, and MAPT clusters are .91, .89 and .91, respectively.  This reflects the reality of the CHC COG->ACH research as in both reading and math achievement, cognitive tests or clusters with high g-loadings (viz., measures of Gc and Gf), as well as shared domain-general abilities, are always in the pool of CHC measures associated with the academic deficit.

--However, the placement of GRWAPT and MAPT in the different content/operations quadrants in Figures 1 and 2 suggests that more differentiated CHC-designed achievement domain SAPT measures might be possible to develop.   The manifest correlations between MAPT and the two GRWAPT measures were .82 to .84, suggesting approximately 69 % shared variance.  GRWAPT and MAPT are strongly related SAPTs, yet there is still unique variance in each.  Furthermore, the WJ-R SAPT measures used in this analysis were equally weighted clusters and not the differentially weighted clusters as in the original WJ.  As presented previously, research suggests that optimal SAPT prediction requires developmentally shifting weights across age.  It is my opinion that the development of developmentally-sensitive CHC-designed SAPTs will result in lower correlations between RAPT and MAPT measures.

Beyond CHC Theory:  Cognitive-Aptitude-Achievement Trait Complexes and SLD Identification Models

The possibility of measuring, mapping and quantifying CAATCs raises intriguing possibilities for re-conceptualizing approaches to the identification of SLD.  Figure 4 presents the generic representation of the prevailing third-method SLD models as well as a formative proposal for a conceptual revision.  As noted previously, the prevailing POSW model (left half of Figure 4), although useful for communication and enhancing understanding of the conceptual approach, is simplistic.   Implementation of the model requires successive calculations of simple (and often multiple) discrepancies which fails to capture the multidimensional and multivariate nature of human cognitive, aptitude, and achievement abilities.  I believe that the CAATC representations in Figure 2, although still clearly imperfect and fallible representations of the non-linear nature of reality, are a better approximation of the complex nature of cognitive-aptitude trait complex relations.  The right-side of Figure 4 is an initial attempt to conceptualize SLD within a CAATC framework.  In this formative model, the bottom two components of the current third-method models (i.e., academic and cognitive weakness) have been combined into a single multidimensional CAATC domain.

Figure 4 (Click on image to enlarge)

CAATCs better operationalize the notion of consistency among the multiple cognitive, aptitude, and achievement elements of an important academic learning domain or domain of SLD.  As noted in the operational definition of a CAATC presented earlier, the emphasis is on a constellation or combination of elements that are related and are combined together in a functional fashion.  These characteristics imply a form of a centrally inward directed force that pulls elements together much like magnetism.  Cohesion appears the most appropriate term for this form of multiple element bonding.  Cohesion is defined, as per the Shorter English Oxford Dictionary (2002), as “the action or condition of sticking together or cohering; a tendency to remain united” (p. 444).  Element bonding and stickiness are also conveyed in the APA Dictionary of Psychology (VandenBos, 2007) definition of cohesion as “the unity or solidarity of a group, as indicated by the strength of the bonds that link group members to the group as a whole” (p. 192).  Thus, in the CAATC-based SLD proposal in Figure 4, the degree of cohesion within a CAATC (as designed by circular icon shape) is considered an integral and critical step to ascertaining if a strong cohesive CAATC, which represents a particular academic domain deficit, is present.  

The stronger the within-CAATC cohesion, the more confidence one could place in the identification of a CAATC as possibly indicative of a SLD.  This focus on quantifying the CAATC cohesion is seen as a necessary, but not sufficient, first step in attempting to identify SLD based on a multivariate POSW.  If the CAATC demonstrates very weak cohesion, the hypothesis of a possible SLD should receive less consideration.  If there is significant (yet to be defined) moderate to strong CAATC cohesion, then the comparison of the CAATC to the cognitive/academic strengths portion of the conceptual model is appropriate for SLD consideration.  To simplify, POSW-based SLD identification would be based first on the identification of a weakness in a cohesive specific CAATC which is then determined to be significantly discrepant from relative strengths in other cognitive and achievement domains.  

Of course, additional variations of this model require further exploration.  For example, should discrepant/discordant comparisons be made between other empirically identified and quantified CAATCs?  Would CAATC-to-CAATC comparisons between high empirical and theoretically correlated CAATCs (e.g., basic reading skills and basic writing skills), when contrasted to less empirically and theoretically correlated CAATC-to-CAATC domains (e.g., basic reading skills and math reasoning), be diagnostically important?  I have more questions than answers at this time.

      

Yes—this proposed framework is speculative and in the formative stages of conceptualization.  It is based on exploratory data analyses, theoretical considerations, and well reasoned logic.  It is not yet ready for applied practice.  Appropriate statistical metrics and methods for operationalizing the degree of domain cohesion are required.  I do not see this as an insurmountable hurdle as methods based on Euclidean distance measures (e.g., Mahalanobis and or Minkowski distance) which can quantify the cohesion between CAATC measures as well as the distance of all the trait complex elements from the centroid of a CAATC exist.  Or, statisticians much smarter than I can might apply centroid-based multivariate statistical measures to quantify and compare CAATC domain cohesion.  I urge those with such skills and interest to pursue the development of these metrics.  Also, the current limited exploratory results with the WJ-R data should be replicated and extended in more contemporary samples with a larger range of both CHC cognitive, aptitude, and achievement tests and clusters.  I would encourage split-sample CAATC model-development and cross-validation in the WJ III norm data.

The proposed CAATC framework, and integration into SLD models is, at this time, simply that—a proposal.  It is not ready for prime-time, in-the-field implementation.  It is presented here as a formative idea that will hopefully encourage others to explore.  Additional research and development, some of which I suggested above, will either prove this to be a promising methodology or an idea with limited validity or one with too many practical constraints that render it hard to implement.  Nevertheless, the results presented here suggest promise.  The results suggest possible incremental progress toward better defining SLD and learning complexes that are more consistent with nature—with the identification of CAATC taxon’s[10] that better approximate “nature carved at the joints” (Meehl, 1973, as quoted and explained by Greenspan, 2006, in the context of MR/ID diagnosis).  Such a development would be consistent with Reynolds and Lakin’s (1987) plea, 25 years ago, for disability identification methods that better represent dispositional taxon’s rather than classes or categories based on specific cutting scores which are grounded in “administrative conveniences with boundaries created out of political and economic considerations” (p. 342). 

---

[1] See SÜß and Beauducel (2005) and Tucker-Drob and Salthouse (2009) for excellent descriptions of these methods and illustrative results.

[2] The WJ-R battery was analyzed since it was the last version of the WJ series to include scholastic aptitude clusters.

[3] As noted in Figure 1, the Reading and Written Language Aptitude clusters, which were separate variables in the analysis, shared 3 of 4 common tests and nearly overlapped in the MDS plot.  Thus, for simplicity they were combined into the single GRWAPT variable in Figure 1.  This is also consistent the factor analysis of reading and writing achievement variables that typically produce a single Grw factor and not separate reading and writing factors.

[4] The primary narrow abilities measured by each of the cognitive Gf-Gc cluster are included in the label for each cluster.  Contrary to the WJ III, the Gf-Gc clusters were not all operationally constructed as broad Gf-Gc abilities (see McGrew, 1997; McGrew & Woodcock, 2001).  Only the WJ-R Gf and Gc clusters can be interpreted as measuring broad domains as per the requirement that broad measures must include indicators of different narrow abilities (e.g., Concept Formation-I and Analysis-Synthesis-RG).  The other five WJ-R Gf-Gc clusters are now understood to be valid indicators of narrow CHC abilities (Gsm-MS; Ga-PC; Glr-MA; Gv-MV/CS; Gs-P).

[5]  The BIS model is a heuristic framework, derived from both factor analysis and MDS facet analysis, for the classification of performance on different tasks and is not to be considered a trait-like structural model of intelligence as exemplified by the factor-based CHC theory.  Nevertheless, Guttman Radex MDS models often show strong parallels to hierarchical factor based models based on the same set of variables (Kyllonen, 1996; SÜß & Beauducel, 2005; Tucker-Drob & Salthouse, 2009).

[6] The MAPT cluster also includes the two Gf tests and Visual Matching.

[7] WJ III 3-D MDS model for norms subjects aged 9-13 is available at http://www.iqscorner.com/2008/10/wj-iii-guttman-radex-mds-analysis.html

[8] A similar dimension emerged as a plausible higher-order cognitive processing dimension in the previously mentioned Carroll type analysis of 50 WJ III test variables.

[9] Using trigonometry, the cosine of the intersection of the two trait complex vectors was converted to a correlation.  I thank Dr. Joel Schneider for helping fill the gap in my long-lost expertise in basic trigonometry via an excel spreadsheet that converted the measured angle to a correlation.

[10] The Shorter Oxford English Dictionary defines a taxon as “a taxonomic group of any ran, as species, family, class, etc; an organism contained in such a group” (p. 3193) and taxonomy as “classification, esp. in relation to its general laws or principles; the branch of science, or of a particular science or subject, that deals with classification; esp. the systematic classification of living organisms” (p. 3193; italics in original)