On the Reliability of the Findings of PISA Tests

Knowledge is a hidden variable, and we therefore require a test in order to rank subjects according to their level of knowledge. A test is a battery of questions of varying levels of difficulty. The test results constitute an ordinal variable, since one cannot measure knowledge quantitatively, as one would height or weight. 

A test can merely rank subjects according to their level of knowledge. It is common practice to rank the success of education systems in various countries according to the average score achieved by students who take a certain international test. An example ofsuch is the PISA test, on which Israel is ranked 29th out of the 33 OECDcountries. Averaging is a valid procedure for a quantitative variable, but not for an ordinal variable, the items of which can only be ranked. Since an ordinal variable can be ranked but not averaged, some of the rankings based on averages are unreliable, because one could have devised an alternative test with questions of a different degree of difficulty that would have altered the ranking of the mean scores.