Technology-based competency diagnostics
Computer-aided tests are becoming increasingly common in diagnostics used in educational psychology. This is partly due to administrative and economic concerns (e.g. savings on test materials and test administrators) and partly due to considerations related to content, in particular the potential to expand and optimise the test procedures. Two main research strategies exist in the field of technology-based competency diagnostics. The first seeks to transfer existing measurement instruments to new media and determine the extent to which their equivalence across different test media can be guaranteed. The second aims to use videos or interactive tasks to expand the content of competency measurements or make them more efficient.
The IQB’s research into technology-based competency diagnostics reflects the two-part structure of the field. Firstly, we conducted a series of studies into the comparability of measurement instruments operationalised with different test media: smartphones, laptops, and paper and pens. The results, which correspond to most of the findings reported in the literature, indicate that computer-aided tests are just as capable of measuring performance as paper-based test are. However, depending on the specific test, method factors should be taken into account to allow for the effects of the particular media used.
The IQB’s work in this field also aims to develop innovative test formats and review them in terms of their psychometric suitability. For instance, initial studies on measuring comprehension skills in science using short videos followed by a series of comprehension questions have shown that this so-called visual comprehension skill does not equate to an independent construct, but rather to a scientific comprehension skill that can be tested using the traditional paper-and-pen method.
Our goal for the future is to try and further exploit the opportunities presented by modern test media. In doing so, we will pay particular attention to competencies where the assumption is that they will elude measurement by traditional test media but can be unlocked by innovative approaches.