IQB Pilot Study 2024

The aim of the Pilot Study 2024 was to administer the newly developed test items to a larger number of students in order to ensure that they were appropriately challenging and generally easy for students to understand. The findings from the pilot study informed the refinement of the test items before their use in further assessments and educational monitoring.

Assessing the quality of the test items also required collecting information on learning conditions, both inside and outside school. For this reason, students, teachers, and headteachers were asked to complete a questionnaire.

A total of around 6,500 fourth-grade students from approximately 315 mainstream and special schools across eight German federal states participated in the pilot study. Testing took place from May to July 2024. At each school, the assessment was usually carried out on two consecutive mornings within this time frame.

The study is conducted as part of the KMK’s overall strategy for educational monitoring and is based on decisions adopted by the education ministers of the German federal states. Overall responsibility for scientific oversight and data protection compliance rests with the Institute for Educational Quality Improvement (IQB) at the Humboldt-Universität zu Berlin under the academic leadership of Prof. Dr Petra Stanat. The IQB developed and pre-tested the assessment instruments used in the study. IEA Hamburg was responsible for organising and conducting the data collection. IEA Hamburg also trained external test administrators, who then carried out the assessments in the schools.

In German, the assessment included test items covering the competence domains of Reading and Listening Comprehension, Orthography, and Language and Language Use. In Mathematics, test items addressed all five content-related competence domains (core ideas) defined in the educational standards. The assessment encompassed various item formats, including closed-format items (such as multiple-choice items) and constructed response items that required a short answer.

Because the study was designed to pilot a wide range of item formats, students received different sets of test items. As a result, each student completed only a small proportion of all the test items being trialled. A random process assigned each student a set of test items to complete. Students with special educational needs in the areas of learning, language, and emotional and social development were given test items adjusted in difficulty and a shorter questionnaire.

At most schools, students completed all test items on a tablet. In some cases, students worked on the test items on paper or on a different digital device, such as a PC or laptop. This allows for an assessment of how the device or medium (tablet, PC, or paper) affects the completion of the test items.

At no point do teachers, other educational staff, or legal guardians have access to the students’ test responses or questionnaire data. All data from the tests and questionnaires are collected and processed in pseudonymised form, meaning that no school or student names are recorded. All reports on the study’s findings will be based on aggregated data. Therefore, it will not be possible to link the results to individual students.

Compliance with data regulations is a crucial aspect of conducting the study. All materials and processes used in the context of the study have been assessed with regard to data protection regulations and authorised by the respective federal state’s ministry of education. Please note our explanations on data protection in the IQB Trends in Student Achievement.

Image sources listed in order of appearance on the page, from left to right and top to bottom; attribution in accordance with the guidelines of the copyright holders/image databases.

• Copyright 2025 Katerina Limpitsouni via undraw.co