#ITC #webinar: Test #adaptation reporting #standards - May 8, 2025

 Additional information regarding the Interntional Test Commission (ITC) and registration info can be found here.

Click on image to enlarge.

#ITC #ATP #Guidelines for #technology - based psychological and educational #assessments

With the every-increasing trend towards computer and tablet-based psychological and educational tests, authors, publishers and users need to be familiar with the International Test Comission (ITC) and Association of Test Publishers recent (2022) Guidelines for Technology Assessment (click here to download free PDF copy).   Also see post earlier today regarding the related (and very important) Joint Test Standards for both test authors, publishers, and users (especially to evaluate tests).

The new #WAIS-V and forthcoming #WJ V tests—time for a reminder: The Joint Test #Standards, the publisher and you.

With the recent publication of a new WAIS-V and the forthcoming publication (projected Q1 2025) of the WJ V (COI - I’m a coauthor of the WJ V), intelligence assessment professionals are excited.  With these new revised test batteries, it is time to (again) remind users of the critical importance of the Standards for Educational and Psychological Testing (2014; aka the Joint [AERA, APA, NCME] Test Standards).  Click here for a FREE PDF copy to add to your “must read” list.  Yes…you can download and read for free…just in time for the holidays!!!!!!

It is often not understood that these standards, which are of critical importance to those who author and sell psychological and educational tests (i.e, publishers), also include important user standards…that is, assessment professionals who select, evaluate, administer and interpret such tests have certain standards they should ascribe to.

As stated in the Joint Test Standards, “validation is the joint responsibility of the test developer and the test user. The test developer is responsible for furnishing relevant evidence and a rationale in support of any test score interpretations for specified uses intended by the developer. The test user is ultimately responsible for evaluating the evidence in the particular setting in which the test is to be used. When a test user proposes an interpretation or use of test scores that differs from those supported by the test developer, the responsibility for providing validity evidence in support of that interpretation for the specified use is the responsibility of the user.   It should be noted that important contributions to the validity evidence may be made as other researchers report findings of investigations that are related to the meaning of test scores” (p.13; emphasis added).

The publication of a technical manual is the start..but not the end.  The publication of a tests technical manual provides the starting point or foundation (as per the Joint Test Standards) for an ongoing “never ending story” of additional validity evidence that accumulates post-publication through additional research studies.  A thorough and well written technical manual is a “must” at publication of a new test, adhering as close as reasonably possible to the relevant Joint Test Standards.

“It is commonly observed that the validation process never ends, as there is always additional information that can be gathered to more fully understand a test and the inferences that an be drawn from it” (p .21).

If the reader has access to the latest edition of Flanagan and McDonough’s Contemporary Intellectual Assessment (2018) book, the Montgomery, Torres, and Eiseman chapter on “Using the Joint Test Standards to Evaluate the Validity Evidence for Intelligence Tests”) is highly recommended as a concise summary of the ins-and-outs of the Joint Test Standards.

Psychological and educational assessment tools provide information that often result in crucial, and at times, life altering decisions for individuals, especial school age students.  All parties (authors, publisher, users) must take these Joint Test Standards seriously.  

Now Available: #ATP-#NCTA #Assessment Industry #Standards and Best Practices for #OnlineObservationofTests (“Standards and #BestPractices”)

The National College Testing Association (NCTA) and the Association of Test Publishers (ATP) represent two leading organizations dedicated to the advancement and study of testing and assessment. The ATP-NCTA Assessment Industry Standards and Best Practices for Online Observation of Tests (“Standards and Best Practices”) provides consensus-based requirements and considerations for the online observation of tests, with or without a proctor. It includes current thinking regarding fast moving technologies, such as artificial intelligence, biometrics, and advanced algorithms, and evolving regulations concerning privacy and accessibility.  Click here for more info.

The Standards and Best Practices were contributed to and reviewed by testing professionals globally. These guidelines are intended for use by test sponsors, such as certification bodies and online observation providers, to outline responsibilities and best practices that ensure privacy, test security, and data validity. The Standards and Best Practices were developed during rapid changes in the assessment industry, driven by the COVID-19 pandemic. As online monitoring evolved, the document was revised multiple times and will be updated periodically due to the fast-paced advancements in assessment technologies.

ATP and NCTA members can access this document for free as a membership benefit by logging into the members-only section of the website and navigating to "Publications Discounts," while non-members can purchase it for $19.95. Click here for more info.

What is an applied psychometrician?

I wear a number of hats within the broad filed of educational psychology.  One is that of an applied psychometrician.  Whenever anyone asks what I do, I receive strange looks when that title rolls out of my mouth.  I then always need to provide a general explanation.

I've decided to take a little time and generate a brief explanation.  I hope this helps.

The online American Psychological Association (APA) Dictionary of Psychology defines psychometrics as: n. the branch of psychology concerned with the quantification and measurement of mental attributes, behavior, performance, and the like, as well as with the design, analysis, and improvement of the tests, questionnaires, and other instruments used in such measurement. Also called psychometric psychology; psychometry.

The definition can be understood from the two components of the word. Psycho refers to “psyche” or the human mind. Metrics refers to “measurement.” Thus, in simple terms, psychometrics means psychological measurement--it is the math and science behind psychological testing.  Applied psychometrics is concerned with the application of psychological theory, techniques, statistical methods, and psychological measurement to applied psychological test development, evaluation, and test interpretation. This compares to more pure or theoretical psychometrics which focuses on developing new measurement theories, methods, statistical procedures, etc. An applied psychometrician uses the various theories, tools and techniques developed by more theoretical psychometricians in the actual development, evaluation, and interpretation of psychological tests. By way of analogy, applied psychometrics is to theoretical psychometrics, as applied research is to pure research.

The principles of psychometric testing are very broad in their potential application., and have been applied to such areas as intelligence, personality, interest, attitudes, neuropsychological functioning, and diagnostic measures (Irwing & Hughes, 2018). As noted recently by Irwing and Hughes (2018), psychometrics is broad as “It applies to many more fields than psychology, indeed biomedical science, education, economics, communications theory, marketing, sociology, politics, business, and epidemiology amongst other disciplines, not only employ psychometric testing, but have also made important contributions to the subject” (p. 3).

Although there are many publications of relevance to the topic of test development and psychometrics, the most useful and important single source is “the Standards for Educational and Psychological Testing” (aka., the Joint Test Standards; American Educational Research Association [AERA], American Psychological Association [APA], National Council on Measurement in Education [NCME], 2014). The Joint Test Standards outline standards and guidelines for test developers, publishers, and users (psychologists) of tests. Given that the principles and theories of psychometrics are generic (they cut across all subdisciplines of psychology that use psychological tests), and there is a standard professionally accepted set of standards (the Joint Test Standards), an expert in applied psychometrics has the skills and expertise to evaluate the fundamental, universal or core measurement integrity (i.e., quality of norms, reliability, validity, etc.) of various psychological tests and measures (e.g., surveys, IQ tests, neuropsychological tests, personality tests), although sub-disciplinary expertise and training would be required to engage in expert interpretation by sub-disciplines. For example, expertise in brain development, functioning and brain-behavior relations would be necessary to use neuropsychological tests to make clinical judgements regarding brain dysfunction, type of brain disorders, etc. However, the basic psychometric characteristics of most all psychological and educational tests (e.g., neuropsychological, IQ, achievement, personality, interest, etc.) assessment can be evaluated by professionals with expertise in applied psychometrics.

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (2014). Standards for Educational and Psychological Testing. Washington, DC: Author.

 Irwing, P. & Hughes, D. J. (2018). Test development. In P. Irwing, T. Booth, & D. J. Hughes (Eds.), The Wiley Handbook of Psychometric Testing: A Multidisciplinary Reference on Survey, Scale and Test Development (pp. 3-49. Hoboken, NJ: John Wiley & Sons

AAIDD "Death Penalty and Intellectual Disability" A Guide" is now available at AAIDD website!

The AAIDD ID and death penalty guide is now available at the AAIDD webpage!  Although the title is focused on the death penalty and ID (Atkins cases), having written two of the chapters (Flynn effect; Intellectual functioning), I believe all professionals in the field of psychological, assessment and ID should have this reference book--it presents recommendations for ID practice that go well beyond the official AAIDD classification book (the Green book) and are relevant to ID assessment practices in general.

Description of the guide at the AAIDD webpage.

• In the 2002 landmark decision Atkins v. Virginia 536 U.S. 304, the Supreme Court of the United States ruled that executing a person with intellectual disability is a violation of the Eighth Amendment of the U.S. Constitution, which prohibits “cruel and unusual punishment,” but left states to determine their own criteria for intellectual disability. AAIDD has always advocated against the death penalty for people with intellectual disability and has long provided amicus curiae briefs in Supreme Court cases. Thus, in this comprehensive new book published by AAIDD, notable authors in the field of intellectual disability discuss all aspects of the issues, with a particular focus on foundational considerations, assessment factors and issues, and professional concerns in Atkins assessments.

Professional ethics and standards for assessment practices in Atkins MR/ID cases

Psychological assessment is serious business. This is particularly true in the life-or-death nature of Atkins MR/ID death penalty cases. The two primary relevant codes of ethics and standards are briefly summarized below. This summary does not include similar codes/standards that have been promulgated and are relevant to psychology specialties (e.g, forensic psychology; neuropsychology; school psychology; etc.). Professionals are responsible for knowing and following specialty specific codes and guidelines.

This is a summary of key assessment related principles and standards and is not intended to serve as a comprehensive single source to guide professional behavior.

Psychological testing and test score interpretation in general, and IQ testing in the current context of diagnosing MR/ID in an Atkins setting in particular, are serious professional activities with enormous potential positive and negative consequences for the individual being tested (AERA, APA, NCME, 1999). Aside from the guidelines specified in 11th edition of Intellectual Disability: Definition, Classification, and System of Supports (AAIDD, 2010), detailed professional codes of ethics and standards for developing psychological tests, selecting psychological tests, testing and test score interpretation have been promulgated by the relevant professional governing bodies. As set forth in the codes and standards (a select key set) described below, test users have a number of responsibilities to ensure that inferences drawn from test scores are valid and are based on existing scientific evidence.

The “gold standard” set of standards for the development, use and interpretation of psychological and education tests is the American Educational Research Association (AERA), American Psychological Association (APA), and National Council on Measurement (NCME), Standards for Educational and Psychological Testing (1999: a revision of these standards is in progress; typically called the Joint Test Standards). As prescribed in the Joint Test Standards, test users have a number of responsibilities to ensure that inferences drawn from test scores are valid. These include (emphasis via italics added):

Standard 1.4: If a test is used in a way that has not been validated, it is incumbent on the user to justify the new use, collecting new evidence if necessary.

Standard 11.15: Test users should be alert to potential misinterpretations of test scores and to possible unintended consequences of test use; users should take steps to minimize or avoid foreseeable misinterpretations and unintended negative consequences.

The American Psychological Association (APA) Ethical Principles of Psychologists and Code of Conduct (2010 Amendments) also sets forth important ethical principles governing the administration and interpretation of psychological assessment instruments. Principles relevant to assessment and diagnosis include (emphasis via italics added):

9.01 Bases for Assessments:

(a) Psychologists base the opinions contained in their recommendations, reports, and diagnostic or evaluative statements, including forensic testimony, on information and techniques sufficient to substantiate their findings.

9.02 Use of Assessments:

(a) Psychologists administer, adapt, score, interpret, or use assessment techniques, interviews, tests, or instruments in a manner and for purposes that are appropriate in light of the research on or evidence of the usefulness and proper application of the techniques.

(b) Psychologists use assessment instruments whose validity and reliability have been established for use with members of the population tested. When such validity or reliability has not been established, psychologists describe the strengths and limitations of test results and interpretation.

9.06 Interpreting Assessment Results:

When interpreting assessment results, including automated interpretations, psychologists take into account the purpose of the assessment as well as the various test factors, test-taking abilities, and other characteristics of the person being assessed, such as situational, personal, linguistic, and cultural differences, that might affect psychologists' judgments or reduce the accuracy of their interpretations. They indicate any significant limitations of their interpretations.

9.08 Obsolete Tests and Outdated Test Results:

Psychologists do not base their assessment or intervention decisions or recommendations on data or test results that are outdated for the current purpose.

(b) Psychologists do not base such decisions or recommendations on tests and measures that are obsolete and not useful for the current purpose.

Collectively, the Joint Test Standards and APA code of ethics prescribe that assessment professionals (a) use contemporary instruments and procedures, (b) base their test interpretations on scientific evidence supporting the reliability and validity of their interpretations, (c) be aware of the strengths and limitations of the assessment procedures they use, (d) document any limitations in the instruments used and scores provided that bear on their interpretation of results, (e) only interpret scores for purposes for which they have been validated, (f) be aware of, and take necessary steps to minimize, unintended negative consequences of their testing and test interpretation for individuals, particularly in high stakes settings, (g) and follow the assessment related principles and standards articulated in the Joint Test Standards and APA code of ethics.

Inherent in these principles and standards is the understanding that psychologists, when faced with discrepant test data, attempt to explain and hypothesize possible reasons for the divergent results based on scientific evidence and accepted professional standards. The Joint Test Standards in particular make it clear that psychologists must be aware of the basic psychometric characteristics, strengths and limitations of the assessment tools they use and apply that knowledge in their subsequent interpretation and presentation of conclusions in written or oral communication.



Posted using BlogPress from Kevin McGrew's iPad
www.themindhub.com

Book nook: Reference Manual on Scientific Evidence

Thanks to Kevin Foley for bringing the following National Academy of Sciences publication to my attention. It is possible to download PDF copies of the entire book or select chapters by going to this link and registering and following the instructions. Below is a copy of the table of contents

Reference Manual on Scientific Evidence: Third Edition
Committee on Science, Technology, and Law





- iPost using BlogPress from Kevin McGrew's iPad
intelligence intelligence testing Atkins cases ICDP blog psychology school psychology neuropsychology forensic psychology criminal psychology criminal justice death penalty capital punishment ABA IQ tests IQ scores psychometrics adaptive behavior AAIDD mental retardation intellectual disability scientific evidence Daubert standard expert testimony law and science

Generated by: Tag Generator

Gresham and Reschly (2011) on "Standard of Practice and Flynn Effect Testimony in Death Penalty Cases"

Double click on image to enlarge. This article will be added to the FE Archive Project soon....it is long overdue for an update.




- iPost using BlogPress from my Kevin McGrew's iPad

intelligence intelligence testing Atkins cases ICDP blog psychology school psychology neuropsychology forensic psychology criminal psychology criminal justice death penalty capital punishment ABA IQ tests IQ scores adaptive behavior AAIDD mental retardation intellectual disability Flynn effect psychological assessment standards expert witnesses

Generated by: Tag Generator

Dr. Detterman's intelligence bytes: Testing standards

Another in the Dr. Detterman's Intelligence Bytes series


Testing has become a highly professional activity and carefully specified standards for tests and testing have been developed. The most widely used standard was developed jointly by the American Educational Research Association, the American Psychological Association, and the National Council on Measurement in Education. These standards provide guidelines for the construction of tests, specify who should use them, what they should be used for, and how results should be interpreted. Within professions that use tests, test use is frequently regulated by professional associations. Improper use can be sanctioned by the association or licensing boards.

In addition to test standards, the use of tests is controlled by an increasing body of legal decisions specifying how tests may be used in hiring, educational decisions, and other applications including determining who should get the death penalty in the U.S.. There are also “fairness in testing” laws designed to protect the test taker. Increasingly, tests are used to make important decisions about people: if they will get into kindergarten, where they go to college, if they will be hired, etc. It is incumbent on those who use tests to appreciate the importance a test can have to an individual and to fully understand what a test can and cannot do. It is not difficult to give a test but correct interpretation of the meaning of the score requires substantial knowledge about tests.

Doug Detterman Dr. Dettermans intelligence bytes intelligence individual differences psychology school psychology cognitive psychology neuropsychology ISIR forensic psychology IQ IQ differences



- iPost using BlogPress from my Kevin McGrew's iPad