Emerging with Honour from a Dilemma Inherent in the Validation of Educational Achievement Measures.

A dilemma arises in the attempt to establish procedures for valid assessment of academic learning. Measures of learning with high pedagogical validity often have poor psychometric properties. Conversely, tests that are well-constructed by psychometric standards may succeed in sorting and ordering students or schools, but have little pedagogical relevance. There are several approaches for dealing with apparent trade-off. One way is to revise notions of achievement, moving away from test-based criteria toward performance-based criteria. Experience in native- and second-language learning is cited as instructive, because of the multidimensional emphasis on the goals of communication in realistic settings. Valid assessment depends on situation and mode of communication. A systematic, cumulative record of performance is essential to this approach. The writing folder, a dossier kept by the student of diverse samples of his or her writing, is a good example of such a record. Another approach connects the monitoring function of assessment to the learning function at the classroom level. It uses comprehensive item pools administered by item sampling as part of a survey of teaching and learning specifically designed to reflect school learning and suggest improvements. As an example, the Second International Mathematics Study is discussed. (LPG)