Information on tests / test items
What kind of items were used in the tests?
The test items used in national assessments studies are often shorter and take less time to answer than the exercises used during lessons. The 2012 study used a variety of formats: a) closed response formats (e.g. multiple choice questions that require students to pick from a list of answers), b) half-open response formats (items that require students to give a brief answer), and c) open response format (e.g. items that require students to write freely or produce their own diagrams and pictures). Items with a closed response format are marked electronically, while the two remaining response formats have to be analysed by trained coders using detailed coding instructions.
Did the students all answer the same items?
The study used several different versions of test booklets. One set contained only mathematics items, another contained only items on biology, chemistry and physics, and the third contained items for both mathematics and science. The study used a large number of test booklets containing different combinations of items taken from the pool. Students were assigned their booklets at random.
Who developed the items and what was done to make sure that ninth-grade students could solve them?
The test items were developed by a team of experienced teachers drawn from all 16 states in Germany. They were led and advised by the IQB’s academic partners from the relevant fields of subject didactics. The partners provided the teachers with intensive training to prepare them for the process of developing the items. Drafts of the items were subjected to a linguistic review and an assessment of their subject-didactic and psychometric quality, and then revised on the basis of the feedback. Prior to being included in the national assessment study, the items were trialled and further improved in extensive academic studies that used large numbers of students from all over Germany. The national assessment studies only include test items that make it through the various preliminary studies.
How was the test run?
Students sat the test at school in the morning. It lasted about 3.5 hours (including breaks) in total and was broken down into three stages:
Test booklet 1 (items on mathematics or biology, chemistry and physics), part 1: 60 minutes
BREAK: 15 minutes
Test booklet 1 (items on mathematics or biology, chemistry and physics), part 2: 60 minutes
BREAK: 15 minutes
Test booklet 2 (indicators of reading ability and basic cognitive skills) and student questionnaire: approx. 45 minutes
What happens with the study results?
Teachers, other educational staff, and parents and guardians will not, at any stage, have access to the responses students gave in their tests or on the questionnaire. The test administrators collected the booklets and questionnaires at the end of the sessions and sent them directly to the IEA Data Processing and Research Center (DPC). Once the data had been collected, they were entered into the system at the DPC, processed, pseudonymised and forwarded to IQB project management for analysis. All reports on the results of the study will use aggregated data, so it will be impossible to attribute results to individuals. Complying with data protection regulations is an extremely important condition of carrying out the study. All documents and processes used in the study were checked to ensure that they complied with the regulations, and were approved by the relevant state ministry.
Do students receive a grade on their tests?
The national assessment studies aim to measure student competencies as accurately as possible at the level of the education system in each state. The tests are not suited to evaluating the performance of individual students, so their test results are not graded.