National Assessment Study 2008/2009
Who participated in the study and when was it carried out?
The National Assessment Study on Languages at Secondary Level I was based on two separate rounds of data collection that were carried out at different times and organised separately. The results were published in a joint report in 2010. The tests for German and English as the first foreign language were conducted between May and July 2009. Nearly 40,000 ninth-grade students participated throughout the country. They came from 1,500 schools that covered the full spectrum of mainstream schooling in Germany’s general education system (lower secondary schools, intermediate secondary schools, upper secondary schools, and schools with multiple educational tracks, including comprehensive schools). Each school was selected at random and then lots were drawn to select one to two classes that would participate in the study.
In the case of French as the first foreign language, data for the national assessment study was collected earlier, in 2008, in connection with a study for the assessment of educational standards in French. The sample for French included nearly 6,000 students from schools with multiple educational tracks, intermediate secondary schools, upper secondary schools and integrated comprehensive schools in the states of Baden-Württemberg, Berlin, Hesse, North Rhine-Westphalia, Rhineland Palatinate and Saarland.
Which content areas were tested?
The German test covered three content areas: listening, spelling, and reading. Some of the students were given items on all three content areas, while others were tested on just two out of the three. For English and French as the first foreign language, the students were all tested on reading comprehension and listening comprehension.
Apart from information on student competencies, what other data did the study collect?
The competency tests were supplemented by a questionnaire that asked students to provide information about themselves and their lessons. The teachers of the participating students also answered a questionnaire. The information was used to obtain an overview of the teaching and learning conditions in Germany’s schools and to investigate the extent to which all students receive optimal support under certain conditions.
Who is responsible for the study and who conducted it?
The study was commissioned by the ministers of education and cultural affairs in each state. Overall academic and data-protection responsibility for the study were with the Institute for Educational Quality Improvement (IQB) at Humboldt-Universität zu Berlin. This was where the test instruments and questionnaires used in the study were developed, trialled and optimised. The IQB tasked the IEA Data Processing and Research Center (DPC) in Hamburg with organising and carrying out the study. The tests themselves were conducted by external administrators who had received special training for the task from the DPC.
Who analysed the study?
The DPC recorded the data electronically, coded the students’ responses and handled the first data processing steps in line with IQB requirements. Scaling the competency scores and analysing the data happened at the IQB. The results of the data analysis were summarised in a report that was presented to the public in summer 2010 (Köller, Knigge & Tesch, 2010). Since then, a number of in-depth analyses of the data have been carried out and published.
What items were used in the tests?
The test items are based on the descriptions of the national educational standards drawn up by the Standing Conference and aim to measure the extent to which Germany’s individual states have achieved these targets. The test items used in student assessment studies were often shorter and took less time to answer than exercises used during lessons. The National Assessment Study 2009 included multiple choice items, where students had to choose from a list of possible answers, and items that required short answers of just a few words or sentences.
Who developed the items and what was done to make sure that 9th-grade students could solve them?
The items were developed by a team of experienced teachers drawn from all 16 states in Germany. They received special training on the process of developing the items and collaborated closely with the IQB and its academic cooperation partners from subject didactics. Prior to being used in tests, the items went through a series of pilot and standardisation studies in which they were trialled with large numbers of students throughout Germany. The national assessment study itself only included test items that were considered suitable from a subject-didactic perspective and that had made it through the various preliminary studies.
How was the test run?
The data for the national assessment study on languages were collected on a single day. The test lasted just under four hours, which included time for instructions and breaks. The test items were divided into two 60-minute sections that were separated by a long break. With the German and English tests, most students were given German items in one of the sections and English items in the other. The order in which the subjects were tested varied. A smaller number of students were given German items in both sections. After the second section (and a second break), students were given the questionnaire to answer. Depending on how quickly they worked, they spent a maximum of 60 minutes on this. The data collection for French was organised in the same way, although both sections focused exclusively on French skills.
A summary of the National Assessment Study 2009 is available for download here (in German).
