This
is the final post in a series of posts clarifying the nature of cognitive,
aptitude, achievement ability constructs.
Readers should consult the preceding
post (which contains links to all prior background posts) that defined cognitive abilities, aptitudes, achievement abilities, and CHC
cognitive-aptitude-achievement trait complexes (CATTCs). I apologize for not including the reference
list. These posts are snippets of a
manuscript in preparation and I like to post to IQs Corner for feedback that I
might incorporate in the final manuscript.
References are the last thing I do.
Beyond CHC: CHC
Cognitive-Aptitude-Achievement Trait Complex Analyses
I
have previously argued that alternative non-factor analysis methodological (e.g.,
multidimensional
scaling-MDS) and theoretical lenses need to be applied to validated CHC
measures to better understand “both the content
and processes underlying performance
on diverse cognitive tasks” (McGrew, 2005, p. 172). When MDS “faceted” methods have been applied
to data sets previously analyzed by exploratory or confirmatory factor methods,
“new insights into the characteristics of tests and constructs previously
obscured by the strong statistical machinery of factor analysis emerge.” (Schneider
& McGrew, 2012, p. 110).
Following
the methods similar to that explained and demonstrated by Beauducel, Brocke and
Liepmann (2001), Beauducel and Kersting (2002), SÜß and Beauducel (2005), Tucker-Drob and Salthouse (2009; this is an awesome example of MDS analyses side-by-side with factor analsysis of the same set of variables) and Wilhelm (2005), I subjected all WJ-R standardization
subjects (McGrew, Werder & Woodcock, 1991) who had complete sets of scores
(i.e., listwise deletion of missing data) for the WJ-R Broad Cognitive
Ability-Extend (BCA-EXT), Reading Aptitude (RAPT), Math Aptitude (MAPT),
Written Language Aptitude (WLAPT), Gf-Gc cognitive factors (Gf, Gc, Glr, Gsm,
Gv, Ga, Gs), and Broad Reading (BRDG), Broad Math (BMATH), and Broad Written
Language (BWLANG) achievement clusters to a Guttman
Radex MDS analysis (n = 4,328 subjects from early school
years to late adulthood).
MDS procedures have more relaxed
assumptions than linear statistical models and allow for the simultaneous analysis
of variables that share common variables or tests—a situation that results in
non-convergence problems due to excessive multicolinearity when using linear
statistical models. This feature made it
possible to explore the degree of similarity of the WJ-R operationalized
measures of the constructs of cognitive abilities, general intelligence (g), scholastic aptitudes, and academic
achievement, in a single analysis. That
is, it was possible to explore the relations between and among the core
elements of CHC-based cognitive-aptitude-achievement
trait complexes (CAATC). The
results are presented in Figure 1. [Click on images to enlarge]
Figure 1 (Click on image to enlarge)
WJ-R
MDS Analysis: Basic Interpretation
In
Guttman Radex models, variables closest to the center of the 2-D plots are the
most cognitively complex. Also, the variables are located along two continua or dimensions that often have substantive/theoretical interpretations. The two dimensions in Figure 1 are labeled A<->B and C<->D. The following is concluded from a review of Figure 1:
--The WJ-R g-measure (BCA-EXT) is almost directly at the center of the plot
and is the most cognitive complex variable.
This makes theoretical sense given that it is a composite comprised of
14 tests from 7 of the CHC Gf-Gc cognitive domains. Proximity to the center of MDS plots is sometimes
considered evidence for g.
--Reading and Writing Aptitude
(GRWAPT) and MAPT are also cognitively complex.
Both the GRWAPT
and MAPT clusters are comprised of four equally weighted tests of four
different Gf-Gc abilities—and thus, the finding that they are also among the
most cognitively complex WJ-R measures is not surprising. The CHC Gf-Gc cognitive measures of Gf and Gc
are much more cognitively complex that Gv, Glr, Ga and Gsm.
--The A<->B
dimension appears to reflect the ordering of variables as per stimulus
content, a common finding in MDS
analyses. The cognitive variables on the
left-hand side of the continuum midlines (Gv, Glr, Gf, Gs, MAPT) are comprised
of measures with predominant
visual-figural
or numeric/quantitative characteristics.
The majority of the variables on the right-hand of the continuum midline
(GRWAPT, Gc, Ga, Gsm, BRDG, BWLANG) are characterized as more
auditory-linguistic, language, or verbal. This
visual-figural/numeric/quantitative-to-auditory-linguistic/language/verbal
content dimension is very similar to the verbal, figural, and numeric content
facets of the
Berlin Model of Intelligence
Structure (BIS; SÜß and Beauducel, 2005).
--The C<->D
dimension appears to reflect the ordering of variables as per cognitive operations or processes,
another common finding in MDS analyses.
The majority of the cognitive variables above the continua midline (Gv,
Glr, Ga, Gc, Gsm, BCAEXT, GRWAPT) are comprised primarily of cognitive ability
tasks the involve mental processes or operations. Conversely, although not as consistent, three
of the lowest variables below the continua midline are the achievement ability
clusters (BRDG BWLANG; BMATH). Thus, the C<->D dimension is interpreted as representing a
cognitive operations/process-to-acquired knowledge/product dimension.
--In contrast to
factor analysis, interpretation of MDS is more is more qualitative and subjective. Variables that may share a common dimension
are typically identified as lying on relatively straight lines or planes, in
separate quadrants or partitions, or tight groupings (often represented by
circles or ovals or connected as a shape via lines). Inspecting the four quadrants created by the A<->B C<->D
dimensions (see Figure 1) suggests the following. The AC quadrant is interpreted to represent
(excluding BCAEXT which is near the center) cognitive operations with
visual-figural content (Gv; Glr). The CB
quadrant is interpreted as representing auditory-linguistic/language/verbal
content based cognitive operations. The BC
quadrant only includes the three broad achievement clusters, and is thus an achievement
or an acquired knowledge dimension.
Finally, the DA quadrant can be interpreted as cognitive operations that
involve quantitative operations or numeric stimuli (e.g., Gf is highly
correlated with math achievement; McGrew & Wendling, 2010; one-half of the
Gs-P cluster is the Visual Matching test which requires the efficient
perceptual processing of numeric stimuli—Glr-N).
The interpretation of these four quadrants is
very consistent with the BIS content-faceted
content-by-operations model research.
--The
theoretical interpretation of the two continua and four quadrants provides
potentially important insights into the abilities measured by the WJ-R
measures. More importantly, the
conclusions provide potentially important theoretical insights into the nature
of human intelligence, insights that typically fail to emerge when using factor
analysis methods (see Schneider & McGrew, 2012 and SÜß and Beauducel, 2005).
In other MDS analyses I have completed, similar
visual-figural/numeric/quantitative-to-auditory-linguistic/language/verbal
and
cognitive
operations/process-to-acquired knowledge/product continua dimensions have
emerged (McGrew, 2005; Schneider & McGrew, 2012). When I have investigated a handful of 3-D MDS
models the same two dimensions emerge along with a third
automatic-to-deliberate/controlled cognitive processing dimension which
is consistent with the prominent dual-process models of cognition and
neurocognitive functioning (Evans, 2008, 2011; Barrouillet, 2011; Reyna &
Brainerd, 2011; Rico & Overton, 2011; Stanovich, West & Toplak, 2011)
that are typically distinguished as Type I/II or System I/II (see Kahneman’s, 2011, highly acclaimed
Thinking, Fast and Slow).
--These
higher-order cognitive processing dimensions, which are not present in the CHC
taxonomy, suggest that intermediate strata (or dimensions that cut across broad
CHC abilities) might be useful additions to the current three-stratum CHC
model. These higher-order dimensions may
be capturing the essence of fundamental neurocognitive processes and argue for
moving beyond CHC to integrate
neurocognitive research to better understand intellectual performance.
WJ-R
MDS Analysis: Cognitive-Aptitude-Achievement
Trait Complex (CAATC) Interpretation
Figure
2 is an extension of the results presented in Figure 1. Two different CAATCs are suggested. These were identified by starting first with
the BMATH and BRDG/BWLANG achievement variables and next connecting these
variables to their respective SAPTs (GRWAPT; MAPT). Next, the closest cognitive Gf-Gc measures
that were in the same general linear path were connected (the goal was to find
the math and reading related variables that were closest to lying on a straight
line). Ovals encompassing the entire
space comprising the two circle-line-circle traces where superimposed on the
figure. A dotted line that represented
the approximate bisection of each of the cognitive-aptitude-achievement trait
complex vectors was drawn. Finally, an
approximate correlation (r = .55; see
Figure 2) between the two multidimensional CAATC was estimated via measurement
of the angle between the CAATC vector dotted lines.
Figure 2 (Click on image to enlarge)
As
presented in Figure 2, Math and Reading-Writing CAATCs are suggested as a
viable perspective from which to view the relations between cognitive
abilities, aptitudes, and achievement abilities. The primary conclusions, insights, and
questions are drawn from Figure 1 and 2 are:
--It appears that
the potential exists to empirically identify CAATCs via the use of CHC-grounded
theory, the extant CHC COG->ACH relations research, and multidimensional
scaling. It also appears possible to
estimate the correlation between different trait complexes (see
math/reading-writing trait complex r
= .55 in Figure 2). I suggest these preliminary
findings may help the field of cognitive-achievement assessment and research
better approximate the multidimensional nature of human cognitive abilities,
aptitudes, and achievement abilities.
--Although the WJ-R
battery is not as comprehensive a measure of CHC abilities as the WJ III, the
cognitive abilities within the respective math and reading/writing CAATCs are
very consistent with the extant CHC COG->CHC relations research
(McGrew & Wendling, 2010;
click here for visual-graphic summary). The
reading-writing trait complex (see Figure 2) includes Ga-PC, Gc-LD/VL, and via
the GRWAPT, Gs-P, and Gsm-MS, abilities that are listed as domain-general and
domain-specific abilities in Figure 3.
In the case of math, the trait complex includes indicators of Gf-RG,
Gv-MV, and via the MAPT, Gs-P (Visual Matching, which might also tap Gs-N) and
Gc-LD/VL, abilities that are either domain-general or domain-specific for math
in Figure 3. Working memory (Gsm-WM) is
not present (as suggested by Figure 3) as the WJ-R battery did not include a
working memory cluster that could enter the analysis.
Figure 3 (Click image to enlarge)
--Also of
interest are the three WJ-R cognitive factors (Gsm-MS, Glr-MA, Gs-P) that are excluded from the hyperspace
representations of the proposed math and reading-writing CAATCs. Although highly speculative, it may be
possible that their separation from the designated trait complexes suggest,
that if known to be related to reading-writing or math achievement, their
independence from the narrower trait complexes may be an indication that they
represent domain-general abilities.
Glr-MA and Gs-P are both listed as domain-general abilities in Figure 3. Additional work is needed to determine if the
independence (from identified CAATCs) of CHC measures known to be significantly
related to achievement indicates domain general abilities. Alternatively, it is very possible, given the
previously demonstrated developmental nuances of CHC COG->ACH relations that the
results presented in Figures 1 and 2, which used the entire age range of the WJ-R
measures, may mask or distort findings in unknown ways.
--Those
knowledgeable of the CHC COG->ACH relations research will obviously note the prior
inclusion of certain Gv abilities (Vz, SR, MV) in Figure 3 as well as the
inclusion of the WJ-R Gv-MV/CS cluster as part of the proposed math CAATC
(Figure 2), despite the lack of consistently reported significant CHC Gv-ACH
relations.
McGrew and Wendling (2010) recognized that some Gv abilities
have clearly been linked to reading and math achievement (especially the later)
in non CHC-organized research. They
speculated that the “Gv Mystery” may be due to certain Gv abilities being
threshold abilities or that the
cognitive batteries included in their review did not include Gv measures that
measured complex Gv related Vz or MV processes.
Given this context, it may be an important finding (via the methods
described above) that the WJ-R Gv measure is unexpectedly included in the math CAATC. This may support the importance of Gv
abilities in explaining math and concurrently indicate a problem with the
operational Gv measures.
--The long
distance from the WJ-R Gv measure to the center of the diagram (see Figure 2)
indicates that the WJ-R Gv measure, which included tests classified as
indicators of CS and MV, is not cognitively complex. This conclusion is consistent with Lohman’s
seminal review of Gv abilities (Lohman, 1979) where he specifically mentions CS
and MV as representing low level Gv processes and “such tests and their factors
consistently fall near the periphery of scaling representations, or at the
bottom of a hierarchical model” (Lohman, 1979, 126-127). I advance the hypothesis that the math CAATC
in Figure 2 suggests that Gv is a
math-relevant domain, but more complex Gv tests (e.g., 3-D mental “mind’s
eye” rotation; complex visual working memory), which would be closer to the
center of the MDS hyperspace, need to be developed and included in cognitive
batteries. This suggestion is consistent
with Wittmann’s concept of Brunswick Symmetry,
which, in turn, is founded on the fundamental concept of symmetry which has been central to success in most all branches of science
(Wittmann & SÜß, 1999). The
Brunswick Symmetry model argues that in order to maximize prediction or
explanation between predictor and criterion variables, one should match the
level of cognitive complexity of the variables in both the predictor and
criterion space (Hunt, 2011; Wittmann & SÜß, 1999). The WJ-R Gv-WJ-R BRMATH relation may represent
a low (WJ-R Gv)-to-high (WJ-R BMATH) predictor-criterion complexity mismatch, thus dooming any possible
significant relation.
--Researchers
and practitioners in the area of SLD should recognize that when third method
POSW “aptitude-achievement” discrepancies are evaluated to determine
“consistency”, the combination of domain-general and domain-specific abilities that
comprise an aptitude for a specific achievement domain in many ways can be considered a
mini-proxy for general intelligence (g). In Figures 1 and 2 the BCA-EXT and MAPT and
GRWAPT variables are in close proximity (which also represents high
correlation) and are all near the center of the MDS Radex model. The manifest correlations between the WJ-R
BCA-EXT (in the WJ-R data used to generate the CAATCs in Figure 10) and RAPT,
WLAPT, and MAPT clusters are .91, .89 and .91, respectively. This reflects the reality of the CHC COG->ACH
research as in both reading and math achievement, cognitive tests or clusters
with high g-loadings (viz., measures
of Gc and Gf), as well as shared domain-general abilities, are always in the
pool of CHC measures associated with the academic deficit.
--However, the
placement of GRWAPT and MAPT in the different content/operations quadrants in
Figures 1 and 2 suggests that more differentiated CHC-designed achievement
domain SAPT measures might be possible to develop. The manifest correlations between MAPT and
the two GRWAPT measures were .82 to .84, suggesting approximately 69 % shared
variance. GRWAPT and MAPT are strongly
related SAPTs, yet there is still unique variance in each. Furthermore, the WJ-R SAPT measures used in
this analysis were equally weighted clusters and not the differentially
weighted clusters as in the original WJ.
As
presented previously, research suggests that optimal SAPT
prediction requires developmentally shifting weights across age. It is my opinion that the development of developmentally-sensitive
CHC-designed SAPTs will result in lower correlations between RAPT and MAPT
measures.
Beyond CHC Theory:
Cognitive-Aptitude-Achievement Trait Complexes and SLD Identification
Models
The
possibility of measuring, mapping and quantifying CAATCs raises intriguing
possibilities for re-conceptualizing approaches to the identification of
SLD. Figure 4 presents the generic
representation of the prevailing third-method SLD models as well as a formative
proposal for a conceptual revision. As
noted previously, the prevailing POSW model (left half of Figure 4), although
useful for communication and enhancing understanding of the conceptual
approach, is simplistic. Implementation
of the model requires successive calculations of simple (and often multiple)
discrepancies which fails to capture the multidimensional and multivariate
nature of human cognitive, aptitude, and achievement abilities. I believe that the CAATC representations in
Figure 2, although still clearly imperfect and fallible representations of the
non-linear nature of reality, are a better approximation of the complex nature
of cognitive-aptitude trait complex relations.
The right-side of Figure 4 is an initial attempt to conceptualize SLD
within a CAATC framework. In this
formative model, the bottom two components of the current third-method models
(i.e., academic and cognitive weakness) have been combined into a single
multidimensional CAATC domain.
Figure 4 (Click on image to enlarge)
CAATCs
better operationalize the notion of consistency among the multiple cognitive,
aptitude, and achievement elements of an important academic learning domain or
domain of SLD. As noted in the
operational definition of a CAATC presented earlier, the emphasis is on a constellation or combination of
elements that are related and are combined together in a functional
fashion. These characteristics imply a
form of a centrally inward directed force that pulls elements together much
like magnetism. Cohesion appears the
most appropriate term for this form of multiple element bonding. Cohesion
is defined, as per the Shorter English Oxford Dictionary (2002), as “the action or condition of
sticking together or cohering; a tendency to remain united” (p. 444). Element bonding and stickiness are also
conveyed in the APA Dictionary of
Psychology (VandenBos, 2007) definition of cohesion as “the unity or
solidarity of a group, as indicated by the strength of the bonds that link
group members to the group as a whole” (p. 192). Thus, in the CAATC-based SLD proposal in
Figure 4, the degree of cohesion within
a CAATC (as designed by circular icon shape) is considered an integral and
critical step to ascertaining if a strong cohesive CAATC, which represents a
particular academic domain deficit, is present.
The stronger the within-CAATC cohesion, the more confidence one could
place in the identification of a CAATC as possibly indicative of a SLD. This focus on quantifying the CAATC cohesion
is seen as a necessary, but not sufficient, first step in attempting to
identify SLD based on a multivariate POSW.
If the CAATC demonstrates very weak cohesion, the hypothesis of a
possible SLD should receive less consideration.
If there is significant (yet to be defined) moderate to strong CAATC
cohesion, then the comparison of the CAATC to the cognitive/academic strengths
portion of the conceptual model is appropriate for SLD consideration. To simplify, POSW-based SLD identification
would be based first on the identification of a weakness in a cohesive specific CAATC which is then determined
to be significantly discrepant from relative strengths in other cognitive and
achievement domains.
Of course,
additional variations of this model require further exploration. For example, should discrepant/discordant
comparisons be made between other empirically identified and quantified CAATCs? Would CAATC-to-CAATC comparisons between high
empirical and theoretically correlated CAATCs (e.g., basic reading skills and
basic writing skills), when contrasted to less empirically and theoretically
correlated CAATC-to-CAATC domains (e.g., basic reading skills and math
reasoning), be diagnostically important?
I have more questions than answers at this time.
Yes—this proposed framework is speculative and in
the formative stages of conceptualization.
It is based on exploratory data analyses, theoretical considerations,
and well reasoned logic. It is not yet ready
for applied practice. Appropriate
statistical metrics and methods for operationalizing the degree of domain
cohesion are required. I do not see this
as an insurmountable hurdle as methods based on Euclidean distance measures (e.g., Mahalanobis and or Minkowski distance) which can quantify
the cohesion between CAATC measures as well as the distance of all the trait complex elements from the centroid of a CAATC exist. Or, statisticians much smarter than I can might
apply centroid-based multivariate statistical measures to quantify and compare
CAATC domain cohesion. I urge those with
such skills and interest to pursue the development of these metrics. Also, the current limited exploratory results
with the WJ-R data should be replicated and extended in more contemporary samples
with a larger range of both CHC cognitive, aptitude, and achievement tests and
clusters. I would encourage split-sample
CAATC model-development and cross-validation in the WJ III norm data.
The proposed CAATC framework, and integration into
SLD models is, at this time, simply that—a proposal. It is not ready for prime-time, in-the-field implementation. It is presented here as a formative idea that
will hopefully encourage others to explore. Additional research and development, some of
which I suggested above, will either prove this to be a promising methodology
or an idea with limited validity or one with too many practical constraints
that render it hard to implement.
Nevertheless, the results presented here suggest promise. The results suggest possible incremental
progress toward better defining SLD and learning complexes that are more
consistent with nature—with the identification of CAATC taxon’s that better approximate “nature carved
at the joints” (Meehl, 1973, as quoted and explained by Greenspan, 2006, in the
context of MR/ID diagnosis). Such a
development would be consistent with Reynolds and Lakin’s (1987) plea, 25 years
ago, for disability identification methods that better represent dispositional
taxon’s rather than classes or categories based on specific cutting
scores which are grounded in “administrative conveniences with boundaries
created out of political and economic considerations” (p. 342).
Posted with Blogsy
