Blame it on the algorithm?

Algorithms have been mentioned a lot in recent days. We hear that it was an ‘algorithm’ that was responsible for the ‘downgrading’ of A level results this year.

In fact, there is nothing wrong with an algorithm. Of itself it is neutral. Neither good nor bad.

Let’s take a closer look. Firstly, what is an algorithm?

The Oxford Dictionary (online) defines an algorithm as:

‘a set of rules that must be followed when solving a particular problem’.

It is a simple definition. Notice that there is no mention of a computer, although it is true to say that many of the more complex algorithms would involve the use of computers.

At its most basic level we could say that an example of an algorithm is a recipe. In a recipe we would typically have a list of ingredients followed by a method with a set of instructions. So, the algorithm might look like this.

Making a cake

1. Place 125g of butter and 125g of sugar in a mixing bowl
2. Mix together until light and fluffy
3. Add 2 eggs and mix
4. Add 125g of self-raising flour and mix thoroughly
5. Put mixture into a greased cake tin
6. Bake at 180 degrees for 20 minutes

This algorithm works fine if what we want is a cake. However, if we had really wanted to make say, cheese scones or chocolate brownies then it would not be a helpful algorithm at all. We would need to change it significantly or to use a different algorithm.

Now let’s move onto another algorithm: this time a more mathematical one. In this situation we have a small shop with five employees who earn the same amount (say £8.60) per hour but who work different numbers of hours each. In one week, we need an algorithm for calculating the total amount spent on wages. So, the algorithm would be something like:

1. Multiply the number of hours worked by employee 1 by £8.72. This will give us the wages earned by employee 1 in this week
2. Repeat for all other employees
3. Add the wages of all employees to calculate the total wages bill

This is still a fairly simple example. We could calculate this ‘by hand’ or we might decide to use a spreadsheet or computer program. See below.

Again, this algorithm works well for calculating the total wages bill but it would be of no use if we wanted to know for example the total electricity bill for the week.

So the issue is not really if algorithms are good or bad (they are neither) but if they are fit for purpose.

This brings me back to the ill-fated algorithm used to ‘standardise’ A level results this year. It is important to remember that some form of standardisation is used for exam results every year. This seems reasonable enough as it is a way of trying to ensure fairness from one year to another. In previous years students would only know their final grades on results day and most would be generally unaware of ‘algorithms’.

The difference this year, of course, is that there were no actual exams and so the plan was to use teacher assessments and predictions based on the previous work of the students.

If these teacher-assessed grades had been left unchanged then almost certainly we would have had different headlines around the A level results. We would have heard that the results were excellent and standards had improved or that grades had been inflated and therefore worth less than in previous years.

As stated in Ofqual’s interim report, the standardisation can draw on the following evidence:

“1. Historical outcomes for each centre  

2. Prior attainment (Key Stage 2 or GCSE) of this year’s students and those in previous years within each centre

3. Expected national grade distribution for the subject given the prior attainment of the national entry.”

There is more to it than that, but these are the key areas.

On one level this kind of adjustment does have some merit: it ensures that there is some consistency in terms of average grade distribution with previous years. The problem is that it is not sophisticated enough to look after individual students: for example, an A* student in a school that has never had any A* grades may be seriously disadvantaged. It can also be a problem for whole schools: an example would be a school which has made rapid progress would still be judged on the last three years. In the last few days, we have seen numerous examples of individuals and schools who feel that the system has been unfair.

So, whilst there is nothing wrong with an algorithm as such, it seems like the algorithm used here left a lot to be desired.

Maybe it’s a bit like using a cake recipe to make cheese scones?