ETS Research Report Series
type: channel
description: The ETS Research Report Series features peer-reviewed articles based on measurement and education research conducted by the scientists at ETS. Our reports have been available to teachers, policy makers, and the general public since our first volume was published in 1948. Topics range from English language learning and mathematics skills to teacher preparedness and workforce development to assessments and artificial intelligence. Articles are published in three sections: reports, memoranda, and notes. Issues prior to 2025 are available on the Wiley Online Library. Our staff includes Executive Editor Daniel F. McCaffrey, Lord Chair in Measurement and Statistics, and more than a dozen associate editors. Our full backlist is available at the Wiley Online Library.
publisher: ETS
language: en
publication name: ETS Research Report Series
issn: 2330-8516
items: Preparing K–12 Students With AI Literacy: Proposed Framework, Progression, and Task Design Principles
items: An Evaluation of Item Fit Based on Generalized Residual Item Response Functions
items: TOEFL iBT® Technical Manual
items: Pathways to Graduate School: 5. A Data Overview of U.S. Prospective Graduate Students by Black Students and Gender
items: Pathways to Graduate School: 4. A Data Overview of U.S. Prospective Graduate Students by Pell Grant Eligibility and Gender
items: Pathways to Graduate School: 3. A Data Overview of U.S. Prospective Graduate Students by Parental Education and Gender
items: Pathways to Graduate School: 2. Data Overview of U.S. Prospective Graduate Students by Hispanic Subgroup and Gender
items: Pathways to Graduate School: 1. A Data Overview of U.S. Women Prospective Graduate Students—Profiles of Nine Racial Groups
items: Navigating the Digital Horizon: A Proposed Framework and Strategies for Assessing Digital Literacy
items: Communication as a Future Ready Skill: A Proposed Framework and Strategies for Assessment
items: Improving the Quality of Numeracy Skills: Progressing from Level 2 to Level 4 on the PIAAC Cycle 2 Proficiency: An ETS Return on Investment Study: Phase 1
items: Literacy Skills at and Around Level 2 of the PIAAC Cycle 2 Proficiency Scale An ETS Return on Investment Study: Phase 1
items: Level Up: Raising the Skills of Adults in the United States and Other Countries
items: Responsible AI for Measurement and Learning: Principles and Practices
items: Aligning Scores of Language Proficiency Tests: A Score Concordance Study Between IELTS Academic and TOEFL iBT®
items: Testing Academic Language Proficiency: Comparing the TOEFL iBT® Test and the Duolingo English Test

Preparing K–12 Students With AI Literacy: Proposed Framework, Progression, and Task Design Principles
type: item
description: This paper presents a conceptual framework for AI literacy, a hypothesized learning progression, and assessment design principles for advancing AI literacy among K–12 learners. Recognizing the importance of technical competencies alongside ethical awareness, the framework integrates foundational knowledge, societal implications, and practical applications of AI. Key competencies include ethical decision-making, AI-powered collaboration, and critical evaluation of AI outputs. Developed through an evidence-centered design (ECD) process involving a review of existing literature and frameworks, the proposed AI literacy framework and progression maps a hypothesized trajectory of students’ skill development, providing a structured pathway for improvement with behavior indicators connected to core AI literacy subskills. In this way, the framework and progression may offer educators a roadmap to apply scaffolded and differentiated teaching strategies that actively foster learners’ skill acquisition. To further support connections between assessment and instruction, we introduce three design principles for task design: ensuring relevance to learners, minimizing barriers to resource access, and providing opportunities for skill advancement. These design principles may guide the creation of activities that evaluate and enhance students’ AI literacy. By aligning scaffolded assessments and learning activities with the progression, this framework bridges instruction, assessment, and students’ skill development. It ultimately may be used to support students in developing skills to critically and ethically engage with AI technologies, preparing them to navigate the digital landscape by fostering inclusive instruction that deepens students’ understanding of AI concepts. Chakraburty, S., Ober, T. M., & Liu, L. (2025). Preparing K–12 students with AI literacy: Proposed framework, progression, and task design principles (Research Report No. RR-25-14). ETS. https://doi.org/10.64634/46jn…
subject: Reports
  topic: section
subject: AI literacy
  topic: keywords
subject: K-12 education
  topic: keywords
subject: skills assessment
  topic: keywords
subject: skills progresion
  topic: keywords
subject: task design
  topic: keywords
creator: Srijita Chakraburty
creator: Teresa Ober
creator: Lei Liu
rights: Copyright (c) 2025 Educational Testing Service https://creativecommons.org/licenses/by-nc-nd/4.0
date: 2025-11-21
publication date: 2025-11-21
doi: 10.64634/46jn1p41

An Evaluation of Item Fit Based on Generalized Residual Item Response Functions
type: item
description: Evaluation of item ft for item response theory (IRT) models often involves a comparison of the observed and expected item response functions (IRFs). Several statistics have been suggested for evaluating item ft based on the discrepancy between IRFs, but the asymptotic distributions of the statistics under the null hypothesis are often not well established. Haberman et al. developed a method for evaluating the ft of IRFs based on generalized residuals. These residuals are functions of the latent proficiency variable in the IRT model and follow the standard normal distribution asymptotically. We develop a method to summarize these generalized residuals into a single summary statistic for each item and evaluate its asymptotic distribution. Kondratek suggested a similar Wald-type statistic, but without accounting for the uncertainty in the estimation of the item parameters. Our method combines the work of Haberman and Kondratek, resulting in a single ft statistic per item while accounting for estimation error. A series of simulations was carried out to investigate the performance of our statistic and compare it to several popular item ft statistics. Our method resulted in similar Type I errors as Kondratek’s statistic, with slightly better results in the case of small samples. Furthermore, the recovery was consistent across different levels of item difficulty, and power of the new item ft statistic was relatively low, except for problematic individual items, but this result was found with two competing statistics as well. Suggested citation: Liao, X., van Rijn, P., & Sinharay, S. (2025). An evaluation of item fit based on generalized residual item response functions (Research Report No. RR-25-13). ETS. https://doi.org/10.64634/b68vz316
subject: Reports
  topic: section
subject: item fit
  topic: keywords
subject: generalized residuals
  topic: keywords
subject: item response functions
  topic: keywords
subject: single summary statistic
  topic: keywords
creator: Xiangyi Liao
creator: Peter van Rijn
creator: Sandip Sinharay
rights: Copyright (c) 2025 Educational Testing Service https://creativecommons.org/licenses/by-nc-nd/4.0
date: 2025-11-19
publication date: 2025-11-19
doi: 10.64634/b68vz316

TOEFL iBT® Technical Manual
type: item
description: This technical manual describes the purpose and intended uses of the TOEFL iBT test, its target test-taker population, and relevant language use domains. The test design and scoring procedures are presented first, followed by a research agenda intended to support the interpretation and use of test scores. Given the updates to the test starting January 2026, this technical manual is intended to serve as an overview and rationale for the test design as well as a reference point for informing investigations of validity evidence to support the intended test uses over time. Designed as a living document, this manual will be updated as the test's design, administration, scoring, and evidence of measurement quality (including reliability, validity, and fairness) evolve, along with its intended uses. Suggested citation: Manna, V. F., Li, S., Papageorgiou, S., & Gu, L. (2025). TOEFL iBT® technical manual (TOEFL Research Report No. RR-106). ETS. https://doi.org/10.64634/eje8f497
subject: Reports
  topic: section
subject: English language proficiency
  topic: keywords
subject: test tasks
  topic: keywords
subject: test design
  topic: keywords
subject: TOEFL iBT®
  topic: keywords
subject: validity
  topic: keywords
subject: fairness
  topic: keywords
creator: Venessa Manna
creator: Shuhong Li
creator: Spiros Papageorgiou
creator: Lixiong Gu
rights: Copyright (c) 2025 Educational Testing Service https://creativecommons.org/licenses/by-nc-nd/4.0
date: 2025-10-09
publication date: 2025-10-09
doi: 10.64634/eje8f497

Pathways to Graduate School: 5. A Data Overview of U.S. Prospective Graduate Students by Black Students and Gender
type: item
description: One understudied segment of the domestic talent pool is the Black or African American individuals who are U.S. citizens and aspire to earn a graduate degree. Gender differences within this talent pool are specifically understudied. The 101,039 individuals who are U.S. citizens, who provided gender and race/ethnicity information, and who had scores on all three GRE® measures are the subjects for this descriptive study, one of a series of five such reports. GRE data from July 1, 2016, to June 30, 2021, supplemented with U.S. Department of Education and U.S. Census data, are analyzed. These individuals, referred to as prospective graduate students (PGS), are examined through six core questions: (a) Who were the PGS? (b) Where did they reside? (c) What were their education and work experiences? (d) What were their undergraduate experiences? (e) What were their plans for graduate study? and (f) What were their emerging graduate school choice sets? Key findings include the following: (a) 34% were 22 years of age or younger; (b) 37% were currently enrolled in college; (c) 75% attended an institution in their home state, with women doing so slightly more than men; (d) 46% were first-generation college students; (e) nearly 60% were Pell Grant–eligible; (f) as undergraduates, women favored life sciences, while men leaned toward physical sciences and engineering; (g) women were slightly more interested in master’s degrees; and (h) 76% included at least one in-state institution in their graduate school choice sets. The report concludes with recommendations for future research and practical applications, particularly in the graduate school application process. Suggested citation: Millett, C. M. (2025). Pathways to graduate school: 5. A data overview of U.S. prospective graduate students by Black students and gender (Research Report No. GRE-25-05). ETS.
subject: Reports
  topic: section
subject: Black
  topic: keywords
subject: African American
  topic: keywords
subject: graduate school applications
  topic: keywords
subject: test measures
  topic: keywords
subject: GRE®
  topic: keywords
subject: prospective graduate students
  topic: keywords
subject: demographics
  topic: keywords
subject: work experience
  topic: keywords
subject: undergraduate experience
  topic: keywords
subject: graduate enrollment
  topic: keywords
subject: graduate program
  topic: keywords
creator: Catherine Millett
rights: Copyright (c) 2025 Educational Testing Service https://creativecommons.org/licenses/by-nc-nd/4.0
date: 2025-09-30
publication date: 2025-09-30
doi: 10.64634/5jw0j752

Pathways to Graduate School: 4. A Data Overview of U.S. Prospective Graduate Students by Pell Grant Eligibility and Gender
type: item
description: Today, the Federal Pell Grant program is ubiquitous in conversations about how to pay for undergraduate education. Since 1982, it has supported more than 212 million students, making it a cornerstone of higher education access in the United States. However, what happens next for Pell Grant recipients who earn bachelor’s degrees—especially when pursuing graduate study—remains largely unexplored. This knowledge gap leaves questions unanswered about how gender and financial need shape the path beyond college. The 673,573 individuals who are U.S. citizens, who provided gender and their undergraduate Pell Grant eligibility information, and who had scores on all three GRE® General Test measures are the subjects for this descriptive study, one of a series of five such reports. GRE General Test data from July 1, 2016, to June 30, 2021, supplemented with U.S. Department of Education and U.S. Census data, are analyzed. These individuals, referred to as prospective graduate students (PGS), are examined through six core questions: (a) Who were the PGS? (b) Where did they reside? (c) What were their education and work experiences? (d) What were their undergraduate experiences? (e) What were their plans for graduate study? and (f) What were their emerging graduate school choice sets? Key findings include the following: (a) 50% of Pell-eligible PGS were White, and 75% of non-Pell-eligible PGS were White; (b) fewer Pell-eligible PGS were enrolled in college; (c) more Pell-eligible PGS were enrolled at in-state institutions; (d) substantially more Pell-eligible PGS were first-generation college students compared to their non-Pell-eligible peers; (e) more men than women in both Pell eligibility profiles majored in the physical sciences and engineering than in the life sciences; (f) more than 90% of PGS of both Pell eligibility profiles earned a 3.0 or higher grade point average in their undergraduate majors; and (g) most PGS were considering at least one graduate school in their stat…
subject: Reports
  topic: section
subject: Pell Grant
  topic: keywords
subject: federal grants
  topic: keywords
subject: graduate school applications
  topic: keywords
subject: test measures
  topic: keywords
subject: GRE®
  topic: keywords
subject: prospective graduate students
  topic: keywords
subject: demographics, work experience
  topic: keywords
subject: undergraduate experience
  topic: keywords
subject: graduate enrollment
  topic: keywords
subject: graduate program
  topic: keywords
creator: Catherine Millett
rights: Copyright (c) 2025 Educational Testing Service https://creativecommons.org/licenses/by-nc-nd/4.0
date: 2025-09-30
publication date: 2025-09-30
doi: 10.64634/8c704s43

Pathways to Graduate School: 3. A Data Overview of U.S. Prospective Graduate Students by Parental Education and Gender
type: item
description: Having parents with higher educational attainment is associated with many positive academic outcomes for students (e.g., college enrollment, degree attainment). This study aims to raise awareness and promote understanding of the characteristics, aspirations, and graduate degree intentions of students who are considering graduate school, particularly through the lens of parental education and gender. Three distinct groups are profiled: no parent bachelor of arts (BA), one parent BA, and one parent BA+. The 980,903 individuals who are U.S. citizens, provided gender and parent educational attainment information, and had scores on all three GRE® General Test measures are the subjects for this descriptive study, one of a series of five such reports. GRE General Test data from July 1, 2016, to June 30, 2021, supplemented with U.S. Department of Education and U.S. Census data, are analyzed. These individuals, referred to as prospective graduate students (PGS), are examined through six core questions: (a) Who were the PGS? (b) Where did they reside? (c) What were their education and work experiences? (d) What were their undergraduate experiences? (e) What were their plans for graduate study? and (f) What were their emerging graduate school choice sets? Key findings include the following: (a) The percentage of White PGS increased as parental education increased; (b) more than 50% of PGS from all three parental education groups live in 10 states; (c) Federal Pell Grant eligibility decreased as parental education increased; (d) approximately half of PGS in each parental education group majored in a science, technology, engineering, or mathematics field, with notable gender differences; (e) more than 90% of PGS across all parental education groups reported a B or higher undergraduate major grade point average; (f) a master’s degree was the most common degree objective among all three parental education profiles; (g) most PGS were considering attending at least one graduate scho…
subject: Reports
  topic: section
subject: parent education level
  topic: keywords
subject: graduate school applications
  topic: keywords
subject: test measures
  topic: keywords
subject: GRE®
  topic: keywords
subject: prospective graduate students
  topic: keywords
subject: demographics
  topic: keywords
subject: work experience
  topic: keywords
subject: undergraduate experience
  topic: keywords
subject: graduate enrollment
  topic: keywords
subject: graduate program
  topic: keywords
creator: Catherine Millett
rights: Copyright (c) 2025 Educational Testing Service https://creativecommons.org/licenses/by-nc-nd/4.0
date: 2025-09-30
publication date: 2025-09-30
doi: 10.64634/zdwwc433

Pathways to Graduate School: 2. Data Overview of U.S. Prospective Graduate Students by Hispanic Subgroup and Gender
type: item
description: While discussions around increasing diversity often focus on attracting a broad range of students, the unique needs and challenges of specific groups—such as Hispanic graduate school aspirants—are frequently overlooked. This oversight is problematic given the growing demographic significance of Hispanic individuals in the United States and their underrepresentation in the highest levels of academia. Hispanic graduate students represent a small but vital part of America’s future intellectual capital. Even less frequently explored is the rich heterogeneity of Hispanic graduate school aspirations, with a particular focus on Hispanic subgroup differences and gender. The 120,445 individuals who are U.S. citizens, who provided gender and race/ethnicity information, and had scores on all three GRE® General Test measures are the subjects for this descriptive study, one of a series of five such reports. GRE General Test data from July 1, 2016, to June 30, 2021, supplemented with U.S. Department of Education and U.S. Census data, are analyzed. These individuals, referred to as prospective graduate students (PGS), are examined through six core questions: (a) Who were the PGS? (b) Where did they reside? (c) What were their education and work experiences? (d) What were their undergraduate experiences? (e) What were their plans for graduate study? and (f) What were their emerging graduate school choice sets? Key findings include the following: (a) For Mexican, Puerto Rican, and other Hispanic PGS, roughly 68% are 25 years of age or younger, with many being under 22 years of age; (b) the three Hispanic PGS groups share six states where they reside in high numbers, but each subgroup has at least one unique state where they are present in significant numbers; (c) across the three groups, more than two-thirds attended a public or private baccalaureate institution in their states of residence; (d) parental education varied across three groups; (e) in all three Hispanic subgroups, more…
subject: Reports
  topic: section
subject: Hispanic students
  topic: keywords
subject: graduate school applications
  topic: keywords
subject: test measures
  topic: keywords
subject: GRE®
  topic: keywords
subject: prospective graduate students
  topic: keywords
subject: demographics
  topic: keywords
subject: work experience
  topic: keywords
subject: undergraduate experience
  topic: keywords
subject: graduate enrollment
  topic: keywords
subject: graduate program
  topic: keywords
creator: Catherine Millett
rights: Copyright (c) 2025 Educational Testing Service https://creativecommons.org/licenses/by-nc-nd/4.0
date: 2025-09-30
publication date: 2025-09-30
doi: 10.64634/0dr4j239

Pathways to Graduate School: 1. A Data Overview of U.S. Women Prospective Graduate Students—Profiles of Nine Racial Groups
type: item
description: Understanding the diverse pathways women take toward graduate education is crucial but often overlooked. When these journeys are presented, they are typically compared to men’s experiences, which can obscure the unique challenges and opportunities women face. This descriptive research study, one of five reports, aims to provide a comprehensive profile of U.S. women who may pursue graduate study, disaggregated by race/ethnicity. Nine distinct profiles of women, based on race/ethnicity, are presented. The 698,298 women who provided gender and race/ethnicity information and had scores for all three GRE® test measures are the subjects. GRE General Test data from July 1, 2016, to June 30, 2021, supplemented with U.S. Department of Education and U.S. Census data, are analyzed. These women, referred to as prospective graduate students (PGS), are examined through six core questions: (a) Who were the women PGS? (b) Where did they reside? (c) What were their education and work experiences? (d) What were their undergraduate experiences? (e) What were their plans for graduate study? and (f) What were their emerging graduate school choice sets? Key findings include the following: (a) The majority of women PGS across the nine profiles lived in 10 states, however, the composition of these 10 states varied by group; (b) two out of five women PGS were enrolled in college, with considerable variation across the nine racial profiles; (c) across the profiles, there was a 40 percentage point difference in having no parent with a bachelor’s degree, ranging from 68% of Mexican women to 28% of White women; (d) 33% of women PGS reported being eligible for a Federal Pell Grant, ranging from 24% of White women to 60% of Black women; (e) women across the nine profiles were comparable in majoring in the physical sciences or engineering as undergraduates but differed substantially in majoring in life sciences; and (f) 32% of all women PGS considered a regional comprehensive university in their s…
subject: Reports
  topic: section
subject: women
  topic: keywords
subject: gragraduate school applications
  topic: keywords
subject: test measures
  topic: keywords
subject: GRE®
  topic: keywords
subject: prospective graduate students
  topic: keywords
subject: demographics
  topic: keywords
subject: work experience
  topic: keywords
subject: undergraduate experience
  topic: keywords
subject: graduate enrollment
  topic: keywords
subject: graduate program
  topic: keywords
creator: Catherine Millett
rights: Copyright (c) 2025 Educational Testing Service https://creativecommons.org/licenses/by-nc-nd/4.0
date: 2025-08-29
publication date: 2025-08-29
doi: 10.64634/0n2pxr04

Navigating the Digital Horizon: A Proposed Framework and Strategies for Assessing Digital Literacy
type: item
description: Digital literacy skills are essential for success in today’s social, academic, and professional environments. This paper introduces a digital literacy framework, outlining key subskills and skill indicators for high school students, grounded in prior research and K–12 educational standards. We examine the implementation of digital literacy assessments in K–12 education and offer recommendations for their integration into educational systems. Our goal is to support the thoughtful development, assessment, and feedback processes necessary to foster students’ digital literacy skills. Suggested citation: Liu, L., Courey, K. A., Kinsey, D., Ober, T. M., & Johnson, D. G. (2025). Navigating the digital horizon: A proposed framework and strategies for assessing digital literacy (Research Report No. RR-25-06). ETS.
subject: Reports
  topic: section
subject: digital literacy
  topic: keywords
subject: skills
  topic: keywords
subject: competencies
  topic: keywords
subject: assessment
  topic: keywords
subject: competency-based education
  topic: keywords
subject: Portrait of a Graduate
  topic: keywords
creator: Lei Lei Liu
creator: Karyssa Courey
creator: Devon Kinsey
creator: Teresa Ober
creator: Darin Johnson
rights: Copyright (c) 2025 Educational Testing Service https://creativecommons.org/licenses/by-nc-nd/4.0
date: 2025-07-31
publication date: 2025-07-31
doi: 10.64634/9jatfh97

Communication as a Future Ready Skill: A Proposed Framework and Strategies for Assessment
type: item
description: Effective communication skills are essential for success in both academic and professional contexts. This concept paper presents a novel framework designed to operationally define and support the development of assessments of communication skills with a specific emphasis on K–12 settings. Through discussions on important considerations for the conceptualization, development, implementation, and use of assessments of communication skills, this paper aims to equip educators and interest holders with insights to foster effective assessment practices of communication skills. Ultimately, we hope this framework may support the identification, development, and assessment of communication skills among school-age learners as we strive to empower students to navigate and excel in the future. Suggested citation: Ober, T. M., Johnson, D. G., Liu, L., Kinsey, D., & Courey, K. A. (2025). Communication as a future ready skill: A proposed framework and strategies for assessments (Research Report No. RR-25-05). ETS.
subject: Reports
  topic: section
subject: communication
  topic: keywords
subject: durable skills
  topic: keywords
subject: assessment
  topic: keywords
subject: competency-based education
  topic: keywords
subject: K-12
  topic: keywords
subject: framework
  topic: keywords
creator: Teresa Ober
creator: Darin Johnson
creator: Lei Liu
creator: Devon Kinsey
creator: Karyssa Courey
rights: Copyright (c) 2025 Educational Testing Service https://creativecommons.org/licenses/by-nc-nd/4.0
date: 2025-07-08
publication date: 2025-07-08
doi: 10.64634/2h5c6j22

Improving the Quality of Numeracy Skills: Progressing from Level 2 to Level 4 on the PIAAC Cycle 2 Proficiency: An ETS Return on Investment Study: Phase 1
type: item
description: This research memorandum highlights the collaborative efforts of the numeracy panel for the ETS Return on Investment (ROI) Study, Phase 1, comprising David (Dave) Tout, Kees Hoogland, and Javier Díez-Palomar, in developing their research memorandum, Improving the Quality of Numeracy Skills: Progressing from Level 2 to Level 4 on the PIAAC Cycle 2 Proficiency Scale. This memorandum captures the expert opinions of the panel sessions. Suggested citation: Tout, D., Hoogland, K., & Díez-Palomar, J. (2025). Improving the quality of numeracy skills: Progressing from Level 2 to Level 4 on the PIAAC Cycle 2 proficiency scale: An ETS return on investment study, Phase 1 (Research Memorandum No. RM-25-02). ETS.
subject: Memorandum
  topic: section
creator: Dave Tout
creator: Kees Hoogland
creator: Javier Javier Diez-Palomar
rights: Copyright (c) 2025 Educational Testing Service https://creativecommons.org/licenses/by-nc-nd/4.0
date: 2025-06-04
publication date: 2025-06-04
starting page: 93
ending page: 93
doi: 10.64634/cehf7n96

Literacy Skills at and Around Level 2 of the PIAAC Cycle 2 Proficiency Scale An ETS Return on Investment Study: Phase 1
type: item
description: This research memorandum highlights the collaborative efforts of the literacy panel for the ETS Return on Investment (ROI) Study, Phase 1, comprising Jean-François Rouet, M. Anne Britt & Tobias Richter, in developing their research memorandum, Literacy Skills at and Around Level 2 of the PIAAC Cycle 2 Proficiency Scale. This memorandum captures the expert opinions from the panel sessions. Suggested citation: Rouet, J.-F., Britt, A., & Richter, T. (2025). Literacy skills at and around Level 2 of the PIAAC Cycle 2 proficiency scale: An ETS return on investment study, Phase 1 (Research Memorandum No. RM-25-01). ETS.
subject: Memorandum
  topic: section
creator: Jean-François Rouet
creator: Anne Britt
creator: Tobias Richter
rights: Copyright (c) 2025 ETS https://creativecommons.org/licenses/by-nc-nd/4.0
date: 2025-06-04
publication date: 2025-06-04
starting page: 41
ending page: 41
doi: 10.64634/tgtdn740

Level Up: Raising the Skills of Adults in the United States and Other Countries
type: item
description: In an era of globally competitive and technology-driven societies, we are seeing a growing interest in developing a better understanding of the types of skills adults need to succeed both in the workplace and in everyday life. One large-scale comparative survey designed to inform that understanding is the OECD’s Program for the International Assessment of Adult Competencies (PIAAC), a survey of adults ages 16-65 conducted in over 30 countries. PIAAC was designed for, and with, participating countries and represents a wide range of knowledge and skills that are required across a variety of adult contexts. The PIAAC data make it clear that adults with lower-level literacy and numeracy skills experience less favorable social, educational and labor market outcomes when compared with their more highly skilled cohorts. While some of those adults possess key foundational reading and numeracy skills, they would benefit from enhancing their skills so that they can more readily navigate, critically analyze, and problem solve in today’s data-intensive, complex digital environments. This report describes a project in which two groups of experts – one in the domain of literacy and the other in numeracy – conducted analyses of the PIAAC data with the goal of defining the knowledge and skills associated with various levels of proficiency in those domains. The key purpose of this work is to form a foundation for the development of strategic interventions to improve adult literacy and numeracy skills. The argument made herein is that the insights gained from these expert analyses into the skills needed to transition to higher levels of literacy and numeracy can be leveraged to create a coherent learning and assessment system that could significantly enhance the literacy and numeracy skills of various adult populations. Suggested citation: Kirsch, I., Lennon, M. L., Sands, A., with Rouet, J.-F., Britt, A., Richter, T., Tout, D., Hoogland, K., and Diez-Palomar, J. (2025). Level up: Ra…
subject: Reports
  topic: section
subject: literacy
  topic: keywords
subject: numeracy
  topic: keywords
subject: technology skills
  topic: keywords
subject: workplace skills
  topic: keywords
subject: Program for the International Assessment of Adult Competencies (PIAAC)
  topic: keywords
subject: skills for adults
  topic: keywords
subject: large-scale assessments
  topic: keywords
creator: Irwin Kirsch
creator: Mary Louise Lennon
creator: Anita Sands
creator: Jean-François Rouet
creator: Anne Britt
creator: Tobias Richter
creator: Dave Tout
creator: Kees Hoogland
creator: Javier Diez-Palomar
rights: Copyright (c) 2025 Educational Testing Service https://creativecommons.org/licenses/by-nc-nd/4.0
date: 2025-06-03
publication date: 2025-06-03
starting page: 54
ending page: 54
doi: 10.64634/t8byp813

Responsible AI for Measurement and Learning: Principles and Practices
type: item
description: The rapid proliferation of artificial intelligence (AI) in educational measurement presents both transformative opportunities and complex ethical challenges. This paper articulates foundational principles for the responsible integration of AI in measurement and learning, drawing on established guidelines set forth by leading organizations such as NIST, OECD, UNESCO, the U.S. Department of Education, and others. We propose a principled framework encompassing fairness and bias mitigation, privacy and security, transparency, explainability, accountability, educational impact and integrity, and continuous improvement. Through the synthesis of current research, best practices, and cross-sector standards, we highlight practical measures to ensure that AI-driven assessment systems are equitable, valid, and reliable. Special emphasis is placed on the significance of representative data, ongoing bias analysis, secure-by-design development, and stakeholder involvement throughout the AI lifecycle. This approach is designed to foster trust, uphold educational values, and safeguard individual rights. By emphasizing ethical and sustainable practices, we advocate for a vision of AI as a driver of human development—supporting learners, educators, and society at large in the pursuit of educational and economic mobility. The principles and recommendations outlined here offer guidance not only for our organization but serve as a resource for the broader educational and measurement community, charting a course for responsible AI innovation that advances both the science and the practice of measurement in support of lifelong learning. Suggested citation: Johnson, M. S. (2025). Responsible AI for measurement and learning: Principles and practices (Research Report No. RR-25-03). ETS.
subject: Reports
  topic: section
subject: artificial intelligence
  topic: keywords
subject: AI
  topic: keywords
subject: fairness
  topic: keywords
subject: best practices
  topic: keywords
subject: bias
  topic: keywords
subject: machine-based learning
  topic: keywords
subject: assessments
  topic: keywords
subject: human-in-the-loop
  topic: keywords
subject: lifelong learning
  topic: keywords
subject: human development
  topic: keywords
subject: testing
  topic: keywords
creator: Matthew S. Johnson,
rights: Copyright (c) 2025 ETS https://creativecommons.org/licenses/by-nc-nd/4.0
date: 2025-06-03
publication date: 2025-06-03
starting page: 62
ending page: 62
doi: 10.64634/4ftg8n64

Aligning Scores of Language Proficiency Tests: A Score Concordance Study Between IELTS Academic and TOEFL iBT®
type: item
description: Scores of different language tests intended for similar purposes (e.g., admission to higher education) are used to determine candidates’ language proficiency and readiness for a chosen domain. To be fair to all students irrespective of the test they took, score requirements should be comparable. Score concordance tables provide an empirical basis for such comparability when good practice principles are met. We report on a score concordance project whose ambitious goal was to adhere to the good practice principles laid out in Knoch and Fan (2024). The providers of the two most widely-used English language proficiency tests for academic admissions purposes, IELTS Academic and TOEFL iBT®, collaborated to complete this project. Research teams representing both tests recruited 969 test takers who took the tests in a counterbalanced order. The study participants represented the major first language groups of both test-taking populations. Every effort was made to keep the interval between taking the two tests short to minimize any effect of changes in the test-takers’ language proficiency level. There were no self-reported score data, a major limitation of existing score concordance studies, as all score reports were verified by the test providers using their official score verification service. Equipercentile equating was conducted by a third party, independent from the test providers. We discuss the challenges in meeting several good practice principles and present the implications for building trustworthy score concordance tables to help stakeholders make informed decisions about language test acceptance. Suggested citation: Ikeda, N., Clark, T., Papageorgiou, S., Gu., L., Ohta, R., Blackhurst, A., & Bruce, E. (2025). Aligning scores of language proficiency tests: A score concordance study between IELTS Academic and TOEFL iBT® (TOEFL Research Report No. RR-105). ETS.
subject: Reports
  topic: section
subject: English language proficiency
  topic: keywords
subject: IELTS Academic
  topic: keywords
subject: score concordance
  topic: keywords
subject: score requirements
  topic: keywords
subject: test equating
  topic: keywords
subject: TOEFL iBT®
  topic: keywords
creator: Naoki Ikeda
creator: Tony Clark
creator: Spiros Papageorgiou
creator: Lixiong Gu
creator: Renka Ohta
creator: Andrew Blackhurst
creator: Emma Bruce
rights: Copyright (c) 2025 ETS https://creativecommons.org/licenses/by-nc-nd/4.0
date: 2025-06-03
publication date: 2025-06-03
starting page: 1
ending page: 28
doi: 10.64634/tn27a897

Testing Academic Language Proficiency: Comparing the TOEFL iBT® Test and the Duolingo English Test
type: item
description: This report provides an in-depth comparison of TOEFL iBT® and the Duolingo English Test (DET) in terms of the degree to which both tests assess academic language proficiency in listening, reading, writing, and speaking. The analysis is based on publicly available documentation on both tests, including sample test questions available on the test websites. For each skill area, the construct as defined by the test developer is discussed, followed by a review of the test content and the cognitive processes involved in completing items related to that skill. The results are evaluated against three propositions from the TOEFL® validity argument (Chapelle et al., 2008). The analysis suggests that although both tests provide evidence of general language proficiency, the input to test takers and the test items for TOEFL iBT are more academic in nature, both in terms of the content and the cognitive demands of the task. Some of the aspects of TOEFL iBT that make the test relatively more academic include extensive reading and listening passages and integrating information from multiple texts in speaking and writing. The report also highlights some of the ways in which a commitment to automated item generation and scoring limits the ability of a test like DET to fully represent the construct of academic language proficiency. Suggested citation: Cushing, S. T. (2025). Testing academic language proficiency: Comparing the TOEFL iBT® test with the Duolingo English Test (TOEFL Research Report No. RR-104). ETS.
subject: Reports
  topic: section
subject: academic language proficiency
  topic: keywords
subject: admissions tests
  topic: keywords
subject: TOEFL iBT
  topic: keywords
subject: Duolingo English Test
  topic: keywords
subject: construct validity
  topic: keywords
creator: Sara Cushing
rights: Copyright (c) 2025 ETS https://creativecommons.org/licenses/by-nc-nd/4.0
date: 2025-06-02
publication date: 2025-06-02
starting page: 49
ending page: 49
doi: 10.64634/0437kw56