Appendix A

Biographical Sketches of Panel Members and Staff

KAREN J. MITCHELL (Chair) recently retired as senior director of the Medical College Admission Test (MCAT) at the Association of American Medical Colleges (AAMC). At AAMC, Mitchell oversaw test development and scoring, test administration and reporting, test preparation services, testing research, and outreach and communication. She directed the redesign and 2015 launch of the current version of the MCAT exam and directed its continued administration during COVID-19. Previously, at the RAND Corporation and U.S. Army Research Institute for the Behavioral and Social Sciences, she directed research on the validity and use of large-scale accountability and selection tests. As a senior program officer at the National Research Council, she directed research on the role of licensure tests in improving teacher quality, codirected the congressionally mandated evaluation of NAEP, and worked on the evaluation of the Voluntary National Tests. She has a Ph.D. in educational research methodology from Cornell University.

ISAAC I. BEJAR recently retired from the Educational Testing Service, where he held the position of principal research scientist. For most of his career, his research focused on the application of technology and cognitive science to the scoring of constructed responses, adaptive testing, and automated item generation (AIG). With funding from the Department of Defense, he is currently carrying out research on AIG in the context of the Armed Services Vocational Aptitude Battery. He has authored multiple journal articles, chapters, and books on these topics, and holds several patents

related to human and automated scoring. He has a Ph.D. in psychology from the University of Minnesota.

SEAN PATRICK (JACK) BUCKLEY is head of assessment and learning sciences at Roblox. Previously, he was president and chief scientist at Imbellus, a game-based assessment technology startup. He also previously served as senior vice president at the American Institutes for Research (AIR), where he led the research and evaluation area, and he still serves as an AIR fellow on several projects, including chairing the NAEP Validity Studies Panel. Before joining AIR, he helped lead the redesign of the Scholastic Aptitude Test (SAT) at the College Board, where he served as senior vice president of research. Earlier, he served as commissioner of the U.S. Department of Education’s NCES, where he was responsible for the measurement of all aspects of U.S. education, including conducting the National Assessment of Educational Progress and coordinating participation in international assessments. He has a Ph.D. in political science from SUNY Stony Brook.

STUART W. ELLIOTT (Study Director) is a scholar in the Division of Behavioral and Social Sciences and Education of the National Academies of Sciences, Engineering, and Medicine. Previously at the National Academies, as the director of the Board on Testing and Assessment, he led numerous studies on educational tests and indicators, assessment of science and 21st-century skills, applications of information technology, and occupational preparation and certification. He recently spent 3 years at the Organisation for Economic Co-operation and Development (OECD) working with their test of adult skills, resulting in a 2017 report, Computers and the Future of Skill Demand. He has a B.A. in economics from Columbia University and a Ph.D. in economics from the Massachusetts Institute of Technology, and he received postdoctoral training in cognitive psychology at Carnegie Mellon University.

BRIAN GONG is senior associate and cofounder of the Center for Assessment, a nonprofit organization that provides technical assistance around assessment and accountability primarily to states, districts, and related organizations, including the Council of Chief State School Officers and the U.S. Department of Education. He provides wide-ranging knowledge of innovations in large-scale assessment—including computer-assisted testing, online test administration, automated scoring, performance assessment, innovative item types—as well as of the challenges of making such innovations operationally feasible and legally defensible, including accommodations and supports for the full range of students with disabilities. He has served on many committees, including the committee responsible for the most recent edition of the Standards for Educational and Psychological

Testing, the Validation Committee for the Common Core State Standards, the group that produced Initiative on the Future of NAEP for the National Center for Education Statistics, and the NAEP Quality Assurance Technical Panel. He has a Ph.D. in education from Stanford.

ANDREW D. HO is Charles William Eliot professor of education at the Harvard Graduate School of Education. He is a psychometrician whose research aims to improve the design, use, and interpretation of test scores in educational policy and practice. He is director of the National Council on Measurement in Education and a trustee for the Carnegie Foundation for the Advancement of Teaching. He served two terms as a member of the National Assessment Governing Board, where he chaired the Committee on Standards, Design, and Methodology. He also chaired the research committee for the vice provost for advances in learning at Harvard University, which governed research on “massive” open online courses. He is an elected member of the National Academy of Education. He has a Ph.D. in educational psychology from Stanford.

JUDITH KOENIG (Senior Program Officer) is on the staff of the Committee on National Statistics of the National Academies of Science, Engineering, and Medicine, where she directs measurement-related studies designed to inform education policy. Her work has included studies on the National Assessment of Educational Progress; teacher licensure and advanced-level certification; inclusion of special-needs students and English-language learners in assessment programs; setting standards for the National Assessment of Adult Literacy; assessing 21st-century skills; and using value-added methods for evaluating schools and teachers. Previously, she worked at the Association of American Medical Colleges and as a special education teacher and diagnostician. She has a B.A. in elementary and special education from Michigan State University, an M.A. in psychology from George Mason University, and a Ph.D. in educational measurement, statistics, and evaluation from the University of Maryland.

STEPHEN LAZER is president and CEO of Questar Assessment, Incorporated. Previously, he served held various positions at Educational Testing Service (ETS), where he served as NAEP social studies coordinator, NAEP development director, NAEP executive director, and cognizant corporate officer for NAEP. At ETS he also served as vice president of assessment development from and as senior vice president for student and teacher assessment. He has written extensively on NAEP and has served as a resource to previous studies of the program. He has a B.A. degree in political science and English from McGill University and an M.A. degree in political science from Princeton.

SUSAN MARIE LOTTRIDGE is senior director of automated scoring at Cambium Assessment, Inc. (CAI). In this role, she leads CAI’s machine learning and scoring team on the research, development, and operation of CAI’s automated scoring software. This software includes automated essay scoring, short-answer scoring, automated speech scoring, and an engine that detects disturbing content in student responses. In this and previous work, she has contributed to the design, research, and use of multiple automated scoring engines including equation scoring, essay scoring, short answer scoring, alert detection, and dialogue systems. She has a Ph.D. in assessment and measurement from James Madison University.

RICHARD M. LUECHT is professor of educational research methodology at the University of North Carolina at Greensboro, where he teaches graduate courses in applied statistics and advanced measurement topics, including advanced item response theory modeling and estimation, computer-based testing systems and methods, and test score equating. His research and expertise include the integration of statistical, computer science, and cognitive science technologies in assessment, advanced psychometric modeling and estimation, and the application of industrial engineering design principles to the design and construction of test (i.e., the “assessment engineering” framework). He has designed numerous algorithms and software programs for automated test assembly and devised a comprehensive computerized adaptive multistage testing framework used by several large-scale testing programs. He is also a technical consultant and advisor for many state departments of education and large-scale testing organizations. He has a Ph.D. in educational psychology, statistics, and measurement from the University of Wisconsin-Milwaukee.

ROCHELLE S. MICHEL is senior proposal writer and the state team lead on the bids and proposals team at Curriculum Associates. She has held a number of positions in the areas of educational testing, assessment, educational research, and nonprofit program management, including executive director of admission programs at Educational Records Bureau; director of research in the Academic to Career Research Center at Educational Testing Service (ETS); research director at Curriculum Associates; and psychometric manager at ETS. She is a member of the American Psychological Association, the National Council on Measurement in Education (NCME), and the Northeastern Educational Research Association, and she currently serves on the editorial boards of NCME Applications of Educational Measurement and Assessment, a book series, and NCME’s journal, Educational Measurement: Issues and Practice. She is a past president of the Northeastern Educational Research Association. She has a Ph.D. in psychometrics from Fordham University.

SCOTT NORTON is deputy executive director of programs at the Council of Chief State School Officers (CCSSO). In this role, he oversees the programmatic work of the council, including student expectations, student transitions, teacher workforce, and school leadership/school improvement. He previously held the position of strategic initiative director for standards, assessment, and accountability at CCSSO. Prior to joining CCSSO, he worked as the assistant superintendent of the Office of Standards, Assessments, and Accountability at the Louisiana Department of Education, and as a public school teacher in Louisiana. He is continuing long-time service as a member of the NAEP Validity Studies Panel. He has a Ph.D. in educational administration and supervision from Louisiana State University.

JOHN WHITMER is senior fellow in data science with the Federation of American Scientists. In this position, he works on activities to implement data science into operational programs, expand data access and usability, and support the creation of an ongoing data science fellowship program at the Institute of Education Sciences at the U.S. Department of Education. Previously, he led teams of data scientists, research scientists, and machine learning engineers in large educational assessment companies (ACTNext), edTech providers (Blackboard), and educational institutions (California State University and California Community Colleges). An educational researcher by training, he approaches these projects with a commitment to improving the lives of underserved and marginalized students through advanced computational methods. He has an Ed.D. in educational leadership from the University of California-Davis.

This page intentionally left blank.