The Future of Education

Edition 12

Accepted Abstracts

PISA – Models and the Reality

Anabela Serrão, Instituto Universitário de Lisboa - ISCTE-IUL Centro de Investigação e Estudos de Sociologia - CIES-IUL (Portugal)

Carlos Pinto-Ferreira, Instituto Superior Técnico - IST-UL Institute for Systems and Robotics - ISR-Lisbon (Portugal)

Abstract

The Programme for International Student Assessment – PISA – is the most ambitious endeavour of large-scale education systems evaluation ever implemented. The Organization for Economic Co-operation and Development – OECD – launched this exercise for the first time in 2000, and in the 2012 edition 65 education systems were assessed. According to OECD, the programme “[…] is a triennial international survey which aims to evaluate education systems worldwide by testing skills and knowledge of 15-year-old students.” And, “[…] tests are designed to assess to what extent students at the end of compulsory education, can apply their knowledge to real-life situations and be equipped for full participation in society.”

Albeit being a prestigious programme, entrenched in sound theoretical grounds, and notwithstanding all the efforts made by PISA experts to mitigate shortcomings, the PISA is not exempt from criticisms of various kinds. When analysing the quotes mentioned above, and taking into consideration the applied methodologies, several questions can be raised and some concerns should be pointed out.

The first question arising in the process of evaluation is that any measurement always affects, direct or indirectly, the system itself, disturbing its inner workings. This fact is particularly relevant when social systems are at stake.

A second difficulty results when students from very different countries in what regards culture, tradition, and beliefs are subjected to the same test.  Although all items are always carefully analysed by panels of experts in order to detect cultural bias or offending interpretations, there is no complete guarantee that the final set of items is adequate to evaluate all students.

Another question regarding the fairness of PISA results is the fact that a paper-and-pencil (or computer) test, limited to three disciplinary domains, cannot encompass the possibly rich, diverse, and unsuspected knowledge and skills of 15-year-old students.

There are also technical criticisms regarding the adopted approaches and methodologies, from the utilization of the Rasch model to negative remarks about the way data are collected and questions are coded.   

Some of what could be considered advantages of PISA – the skill based instead of a curriculum based approach, the assessment of 15-year-old students instead of a particular school year pupils, and the definition of a large set of indicators, as is the case of ESCS – have been also severely criticised.

Finally, some of the criticisms reside, not in the PISA methods and characteristics themselves but on an excessive focus on country rankings, primarily promoted by media, and consequently followed by political leaders.

The main objective of this research is to reframe difficulties and artefacts together with virtuous results of PISA, putting in perspective praises and criticisms to foster a better understanding of this important programme.

Back to the list

Reserved area

Pixel - Via Luigi Lanzi 12 - 50134 Firenze (FI) - VAT IT 05118710481
    Copyright © 2022 - All rights reserved

Privacy Policy

Webmaster: Pinzani.it