Optimising IQB Assessment Instruments and Analytical Methods

Validity of Competence Assessments 🔗

The IQB develops measurement tools to assess conceptually grounded competences and key outcomes of educational processes. The research examines the extent to which these tools meet the requirements necessary to fulfil their intended purpose. This primarily concerns the validity and dimensionality of the methods: Do they measure what they are intended to measure, and can different facets of the constructs be distinguished?

Current research projects in this area:

• NEPS-Cohort Succession at the Lower Secondary Level (NEPS-SC8)

Optimisation of Assessment Instruments and Administration Conditions 🔗

Since 2021, the IQB has been transitioning the institute’s assessment instruments (IQB Trends in Student Achievement, VERA) and related test development processes to computer- or technology-based testing.As part of this transition, the IQB examines how working on test items across different devices (tablets, laptops) may affect results and explores the integration of adaptive and formative testing features.

The IQB also investigates variations in computer-based test administration conditions. Digital tests offer many ways to adapt assessments – for example, through accessibility features for students with special educational needs or by including motivating elements such as multimedia-enhanced feedback or gamification. Furthermore, additional process data, such as response times, can help distinguish effortful responding from rapid guessing. Pilot studies conducted by the IQB examine which adaptations make sense in practice and can be implemented technically – for example, when tests are conducted in schools over networks with limited capacity, which can restrict the amount of data that can be transmitted online. The research aims to design online tests that motivate all students and encourage them to sustain their effort throughout.It also investigates the fairness and validity of test design, particularly regarding potential bias affecting students with special educational needs or those whose first language is not German.

Evaluation and Refinement of Analytical Methods 🔗

The IQB continually reviews and refines both established and emerging quantitative methods in educational research. A key focus is on the applicability of analytical methods, both for addressing the IQB’s core tasks and for conducting secondary analyses. Educational research is increasingly drawing on data-driven approaches that have produced promising results in other disciplines for identifying complex patterns of association. For that reason, the IQB also investigates the potential of machine-learning methods.

The IQB’s datasets are valuable for machine-learning analyses, as they make it possible to examine relationships between hundreds of covariates and students’ achievement. Before applying these methods to the IQB’s core tasks or secondary analyses, their configurations must be thoroughly tested for this specific use case. Through these projects, the IQB aims to test and evaluate the suitability of machine-learning methods for analysing large-scale student achievement data. The projects assess the strengths and limitations of these methods and derive recommendations for their use across different types of research questions. To this end, results from analyses using machine-learning methods are compared with those from traditional analytical methods in terms of accuracy, efficiency, and interpretability.