Applied Innovations with Jake Thompson

The work of the Achievement & Assessment Institute (AAI) and its centers encompass a vast range of services, solutions, and partnerships. In our short interview series, Applied Innovations, we get to know some of the talented members of our team and their projects, across our many centers and various areas of impact.

In this edition, we meet Jake Thompson, Senior Psychometrician with AAI's Accessible, Teaching, Learning, and Assessment Systems (ATLAS).

Can You Tell Us a Little About Yourself?

Outside of work, I’m a die-hard fan of the Kansas City Chiefs and KU basketball, an avid creator and consumer of baked goods, and a Taylor Swift enthusiast. I can usually be found building Lego sets, winning Harry Potter trivia, or hanging out with my wife and our dog, Larry.

What Attracted You To Your Field?

There are two aspects of psychometrics that get me excited every morning (after coffee). First is the puzzle of trying to measure the unmeasurable. Unlike physical characteristics such as height or color, we can’t directly measure a student’s math knowledge. Building statistical models to infer a student’s knowledge from their performance on different academic tasks is a difficult problem with many possible solutions. This leads to the second aspect, which is finding ways to communicate a student’s knowledge, skills, and understanding in a way that’s useful for students, teachers, and parents. Too often, users of educational assessments don’t know what to do with the scores they receive. I enjoy trying to solve the challenging problem of reporting a student’s achievement in an accurate and useful manner.

What Populations Do You or Your Center Support?

My work supports teachers, parents, and students. My research primarily focuses on the application of diagnostic classification models, which are used to provide assessment results that are more fine-grained and actionable to support student learning. Additionally, these models can be used by researchers to evaluate, for example, the effect of an instructional intervention. Within ATLAS, these models are currently being used to support students with significant cognitive disabilities and applied researchers in the wider education community.

In addition to these groups, ATLAS has other projects that support, general education students, and educators.

What Are Your or Your Center’s Key Projects?

I support two main projects. The first is the Dynamic Learning Maps (DLM) Alternate Assessment System. The DLM assessments are operational K-12 assessments in English language arts, mathematics, and science for students who have the most significant cognitive disabilities. The DLM assessments are also the first large-scale educational assessments to use diagnostic classification models for reporting student results. Using diagnostic models allows us to provide more detailed feedback about which knowledge and skills students have learned, and which areas could use more instruction.

Second, I am currently the Principal Investigator of a grant funded by the Institute of Education Sciences focused on developing a software package to estimate diagnostic classification models. Despite the many benefits of diagnostic models, they have seen little use in operational assessment or applied research, due in large part to the lack of user-friendly software. In this project, I’m creating a user-friendly program, complete with documentation, case studies, and online support resources to facilitate the use of these models in the larger research community.

In addition, ATLAS has several other contracts and grants including the Kansas Assessment Program, the 5E-Model Professional Development in Science Education for Special Educators (5E-SESE), the Shared Writing Instructional Model (SWIM), and the Special Educator Technology-Based Training of Trainers (SETTT).

What Are Some Future Opportunities or Upcoming Initiatives That Excite You?

Diagnostic models are a huge opportunity that more people should take advantage of. In educational assessment, there have been increased calls to provide more actionable scores, while also reducing the amount of time students must spend testing. Diagnostic models can accomplish both goals. Rather than provide a single score to describe performance, diagnostic models can identify specific skills that have been mastered and which may need more instruction. This level of feedback is incredibly helpful for teachers, parents, and students. Additionally, because these models place students into discrete classes instead of attempting to place them along a continuum of ability, we can achieve reliable results with far fewer items. Being on the forefront of this opportunity and working to make these models more accessible to a wider range of practitioners is incredibly exciting!

What About Working at a Mission-Driven Organization Like AAI Appeals to You?

I enjoy working at an organization that conducts research to move the field forward and applies research to practice. Too often, researchers publish their findings and simply hope that practitioners will apply the findings. At AAI, we get to do both. When I work on a research study, I then get to see the resulting lessons and recommendations get applied to other assessment and projects. Most organizations don’t offer the ability to directly improve students’ experiences.

*Editors Note: Jake Thompson is also an incredible user of the programming language R. He's built an R package that contains lyrics and audio statistics for Taylor Swift’s discography. See the project here.