Research Byte: Conjectures and refutations in #cognitive ability #structuralvalidity research [with #WISC-V]: Insights from Bayesian structural equation modeling - #schoolpsychology #IQ #intelligence #Wechslers #WISC-V

Conjectures and refutations in cognitive ability structural validity research [with #WISC-V]: Insights from Bayesian structural equation modeling

Click here to view Journal of Psychology of School Psychology source of publication - not open access.

Abstract

The use of Bayesian structural equation modeling (BSEM) provided additional insight into the WISC–V theoretical structure beyond that offered by traditional factor analytic approaches (e.g., exploratory factor analysis and maximum likelihood confirmatory factor analysis) through the specification of all cross loadings and correlated residual terms. The results indicated that a five-factor higher-order model with a correlated residual between the Visual-Spatial and Fluid Reasoning group factors provided a superior fit to the four bifactor model that has been preferred in prior research. There were no other statistically significant correlated residual terms or cross loadings in the measurement model. The results further suggest that the WISC–V ten subtest primary battery readily attains simple structure and its index level scores may be interpreted as suggested in the WISC–V's scoring and interpretive manual. Moreover, BSEM may help to advance IQ theory by providing contemporary intelligence researchers with a novel tool to explore complex interrelationships among cognitive abilities—relationships that traditional structural equation modeling methods may overlook. It can also help attenuate the replication crises in school psychology within the area of cognitive assessment structural validity research through systematic evaluation of complex structural relationships obviating the need for CFA based post hoc specification searches which can be prone to confirmation bias and capitalization on chance.

Applied #psychometrics 101: Strong programs of #constructvalidity—the #theory - #measurement framework with emphasis on #substantive & #structural validity - #WJIV #WJV #shoolpsychology #psychology

The validity of psychological tests “is an overall judgment of the degree to which empirical evidence and theoretical rationales support the adequacy and appropriateness of inferences and actions based on test scores or other modes of assessment” (Messick, 1995, p. 741).

The ability to draw valid inferences regarding theoretical constructs from observable or manifest measures (e.g., test or composite scores) is a function of the extent to which the underlying program of validity research attends to both the theoretical and measurement domains of the focal constructs (Bensen, 1998; Bensen & Hagtvet, 1996; Cronbach, 1971; Cronbach & Meehl, 1955; Loevinger, 1957; Messick, 1995; Nunnally, 1978). 

The theoretical—measurement domain framework that has driven the revisions of the WJ test batteries, particularly from the WJ-R to the forthcoming WJ V cognitive and achievement test batteries (Q1, 2025; COI disclosure: I am a coauthor of the current WJ IV and forthcoming WJ V), is represented in the figures below.  

The goal of this post is to provide visual-graphic (Gv) images that hopefully, if properly studied by the reader (and if I did a decent job), provide the basic concepts of what constitutes the substantive component (and to a more limited extent the structural component) of a strong program of construct validity—in particular, the theoretical-measurement domain mapping framework used in the WJ-R to the forthcoming WJ V. The external stage of construct validity is not highlighted in this current post.  The goal is for conceptual understanding…thus the absence of empirical data, etc.

For those who want written background information, the most succinct conceptual overview of a “strong program of construct validation” is Bensen (1998; click to download and read).  

Otherwise…sit back and enjoy the Gv presenation…where five images equal at least one or more chapter in a technical manual :). 

Be sure to click on each image to enlarge (and make readable)

This figure below was first published in a book on CHC theoretical (then known as Gf-Gc) interpretation of the Wechsler intelligence test batteries (Flanagan, McGrew, & Ortiz, 2000).

Yea…I know.  The following figure uses Gv as the sample cognitive ability construct domain and not Gf as in the prior figure.  I first crafted the figure below in 2005 and don’t have the time (nor attentional control focus bandwidth) to make a new version.  Consider the switch from the Gf domain (above) to Gv (below) as a test of your understanding of the material…and ability to generalize what you have learned.  And yes, I do see there is a spelling error (“on” for “one”)…but it is an old image file and I don’t have time to “clean it up” as noted above.  The primary new feature is the addition of the concept  of developing developmentally (difficulty) ordered sets of test items for the underling ability trait scales for manifest indicator tests C and D under the CHC theoretical narrow ability domain of spatial scanning, under the broad ability domain of Gv. This is where IRT (Rasch model) item scaling is involved.

The following figure is drawn from the WJ IV technical manual (McGrew, LaForte, Schrank, 2014) and illustrates the three-stage structural validity process used in the WJ IV.  The same process, with slightly different age groups and the addition of exploratory hierarchical psychometric network analysis (see exciting and ground-breaking work of Dr. Hudson Golino and colleagues) during stage 2A, will be presented in the WJ V technical manual (LaForte, Dailey & McGrew, Q1-2025).

Important CHC cross-battery factor analysis of 66 subtests across multiple batteries.

https://doi.org/10.1016/j.intell.2020.101433

Research byte: Phonemic awareness, working memory, RAN related but separate reading related constructs

Double click on images to ages to enlarge









Posted using BlogPress from Kevin McGrew's iPad
www.themindhub.com

Dissertation Dish: Structural Validity of the WJ III in a Referred Sample

Structural Validity of the Woodcock Johnson III Cognitive in a Referred Sample by Strickland, Tracy, Ph.D., Arizona State University, 2012 , 85 pages; AAT 3505630

Abstract (Summary)

The structural validity of the WJ-III Cognitive was investigated using the GIA-Extended Battery test scores of 529, six-to-thirteen-year-old students referred for a psychoeducational evaluation. The results of an exploratory factor analysis revealed 11 of the 14 tests loaded on their expected factors. For the factors Gc , Gf , Gs , and Gv , both tests associated with the factor loaded highly; for Gsm , Glr , and Ga , only one test associated with each factor loaded highly. Obtained congruence coefficients supported the similarity between the factors Gs , Gf , Gc , Glr , and Gv for the current referred sample and the normative factor structure. Gsm and Ga were not found to be similar. The WJ-III Cognitive structure established in the normative sample was not fully replicated in this referred sample. The Schmid-Leiman orthogonalization procedure identified a higher-order factor structure with a second-order, general ability factor, g , which accounted for approximately 38.4% of common variance and 23.1% of total variance among the seven, first-order factors. However, g accounted for more variance in both associated tests for only the orthogonal first-order factor Gf . In contrast, the Gc and Gs factors accounted for more variance than the general factor for both of their respective tests. The Gsm , Glr , Ga , and Gv factors accounted for more variance than g for one of the two tests associated with each factor. The outcome indicates Gc , Gf , Gs , and Gv were supported and thus are likely factors that can be utilized in assessment while Gsm , Glr , and Gr were not supported by this study. Additionally, results indicate that interpretation of the WJ-III scores should not ignore the global ability factor.




Posted using BlogPress from Kevin McGrew's iPad
www.themindhub.com