Background

One of my main pieces of content here on Ballot Box Scotland, and indeed a unique selling point of the project, is my coverage of Council By-Elections. Using the wealth of information available as a result of machine counts, I go into depths that other polling and election accounts don’t have the time or expertise for. As time has gone on, I’ve perfected my process and added various bells and whistles, and it seemed like it might now be useful to put together an explanation for the whole deal.

The Voting System

Council elections in Scotland are conducted via the Single Transferable Vote (STV). This is a form of (partially) proportional representation, first introduced for elections in 2007 as a result of a deal the Liberal Democrats did with Labour to form a coalition Scottish Executive in 2003.

As first introduced, every Council in Scotland was divided into wards electing 3 or 4 councillors. The Islands Act has since allowed wards containing inhabited islands to have just one or two councillors, whilst another Electoral Reform Act has expanded the range for mainland wards to between 2 and 5 councillors. Some councils with inhabited islands saw changes to their wards under these new rules in time for the 2022 elections, whereas for the rest of the country changes won’t come until at least 2027.

To elect those councillors, voters simply mark the candidates 1, 2, 3… etc in order of preference. Candidates have to achieve a “quota” of votes cast in order to be elected, and lower-ranked candidates will be eliminated and their votes transferred until all seats are filled. Candidates who are elected with more votes than they need will also have a portion of those votes transferred. A more detailed explanation of the maths is available on this dedicated page.

By-elections work on exactly the same basis as a full election, but just for fewer councillors. This is almost always just one councillor. Rarely, there will be a by-election for more than one councillor in a ward. There were two such by-elections in 2015, and one in 2019, which are thus far the only examples of this happening. A single councillor election is functionally equivalent to the Alternative Vote.

There are two things that are important to bear in mind about STV. The first is that your later preferences have no effect on your earlier preferences. A false claim is sometimes made that you should only mark one candidate or one party, or you lower their chances. This isn’t true, as your vote will only transfer and count for someone else if that candidate wins or drops out. The other is that as this is a transferable voting system, it isn’t possible to “split the vote”. If anyone is telling you otherwise, they either don’t understand the system or are deliberately misleading you.

How Council By-Election Previews Work

My preview posts include a huge amount of data about the ward(s) facing a by-election. This is conveyed via maps and charts. Charts are provided in an interactive Google Chart form, though for the 2022 elections the same data is available to download in a spreadsheet format for each council area on their individual pages. For a preview piece, the data is effectively lifted directly from the relevant page.

Ward and Polling District Map

These maps show the outline of the ward in question. They are intended to be relatively clean and simple, using data on buildings, roads, parks, and bodies of water to show the key areas in the ward and how they are connected, without the clutter of a full geographic map. 

A small inset map shows the location of the by-election ward relative to the whole council, tinted to the colour of the party that led in first preferences. The main map shows the ward divided into polling districts, with each district tinted to the colour of the party that won the most first preference votes there.

Candidates, Councillors and Key Stats
Councillors and Key Stats

4 Councillors, in order elected:
🔴Labour: Malcolm Cunning
🟡SNP: Paul McCabe
🟡SNP: Margaret Morgan
🔴Labour: Catherine Vallis
Change vs 2017: +1 Labour, -1 Conservative
Turnout: 41.2%
Electorate: 22308
Valid
: 8994 (97.8%)
Spoiled: 200 (2.2%)
Quota: 1799

Candidates

🔵Conservative: Euan Blockley
🔴Labour: Malcolm Cunning
Alba: Angela Jones
🟡SNP: Paul McCabe
🟠Lib Dem: Joe McCauley
🟡SNP: Margaret Morgan
Independent: James Toner
🔴Labour: Catherine Vallis
🟢Green: Keith Warwick

These boxes highlight key details you’d expect to know for an election contest. That includes a full list of the candidates who contested the election and the councillors elected, in order of election. It also highlights any changes versus the previous full election in 2022, plus turnout figures and the “quota” that was necessary to win a seat.

First Preferences

First preferences are nice and easy. These charts simply show the total number of first preference votes at the previous election. For parties that stood more than one candidate, there is both a bar that counts their total (using the party’s primary colour with a black edge) and individual bars per candidate (using primary, then secondary, then tertiary colours with no edge). 

Transfers (single winner recalculation)

Charts for transfers are much more complicated, though the same basic format of having an interactive and an expandable image version suitable for social media applies.

As STV at a full election results in multiple councillors, we can’t directly compare results in the traditional “gain-hold-loss” manner people are used to with First Past the Post. A party that won 20% and elected a councillor probably won’t win a by-election if that councillor resigns and there’s another party that won 40%, so though they’ve lost the seat in real terms, it’s not a loss (or gain for the other party) in terms of the electoral maths.

Additionally, because votes transfer, there’s no guarantee the party that had the most first preference votes would win a by-election for a single seat, but we can’t tell that using the transfer flows from a three or four councillor election. However, because these elections are machine counted, there is a detailed record of every ballot and the transfers on them. This can be used to re-calculate the previous election for one (or two) seats with near-perfect accuracy. In the example ward used for this piece, we can see that although the SNP had a narrow lead in first preferences, Labour would overturn it after transfers.

That’s not the same as saying that’s definitely how that election would have turned out with just one councillor, as voter behaviour might have been slightly different. But it is an absolutely spot-on measure of how the election would have been according to the votes actually cast. Note that both lower ranked candidates from parties that stood multiples and successfully elected Independents are eliminated at the start of this count process to best approximate by-election conditions. 

Two-Candidate Preferred

This is simply the final stage of the above re-calculation, showing only the final two candidates, giving further clarity on who would have won a single-seat election, by how much – and how many votes were exhausted and could, at a genuine by-election, help shift things slightly.

Transfers (full election)

This is another transfers chart, but it instead shows the results as they actually were for the full number of seats available. It’s relevant to seeing how each councillor was elected, but it doesn’t directly tell us too much about by-election possibilities, and is simply provided for information.

First Preferences by Polling District

This chart shows the estimated vote share for each party (or Independent candidate) per polling district. This information is available thanks to those machine counts, which generally count each ballot box individually, allowing them to be paired up with polling districts. As postal votes are not broken down by district, a simple estimated apportionment to districts is made, whereby if e.g. 10% of a party’s in-person vote came from a given district, 10% of its postal votes are assigned to it too.

Note however that boxes with relatively small numbers of voters (<200) are required to be merged with another box to help preserve the secrecy of the ballot. This can effectively mean that the data for two or more districts has to merge entirely as well, which may disguise different voting patterns across those districts. This is especially true in by-elections themselves, due to much lower turnout.

Second Preferences by First Preference Party

This chart shows the breakdown of second preferences based on first preference party. So for example, the first column shows SNP first preferences, of which 1,065 (35.45%) went to the Greens, whereas 667 (22.2%) voters didn’t mark a preference beyond SNP candidates. This is data we don’t get just from the actual election transfers, because as soon as the first candidate drops out, we can’t directly see how many second preferences they had from other parties.

These are useful for giving a relative indication of party and candidate favourability between parties. They also clearly burst the assumption made by so many that voters move neatly between parties according to “obvious” patterns, for example that the SNP and Greens form a unified pro-Independence bloc and the Conservatives, Labour and Lib Dems a pro-Union bloc, with votes moving within but not between blocs. Similarly, voters don’t all move obviously along the left-right spectrum.

Prediction Call

From the re-calculation, I report the total number of votes the top two candidates had at the last competitive round of the count, as well as the number of votes that had “exhausted” and didn’t transfer. I then combine that outcome with how current polling is looking, the results of any other elections since or recently, and just a pinch of gut feel to make a call as to how I think it will go. There are four calls that I can make:

  • Tossup – I’m expecting the election to be close or unpredictable, and do not feel comfortable calling it for any one of the parties identified as possible winners.
  • Leans – I’m expecting the election to be close, but reckon that one party has a particular edge. I will not be surprised if another party identified as competitive wins.
  • Likely – I’m expecting the party in question to be the clear frontrunners. I will be surprised if another party identified as competitive wins, but not entirely.
  • Win – I’m expecting the party in question to win the by-election hands down. I will be extremely surprised if another party wins, and will have to pre-heat the oven for the crow I’ll need to eat.

By-Election Results

For the most part, coverage of by-election results are basically the same format as the preview. I provide charts of first preferences and transfers. However, as by-elections are allowed to be hand counted instead of machine counted, my level of analysis beyond that will vary.

If it has been another machine count, and the data has been made available in good time, I’ll usually do all the same analysis as for a preview. If it’s a hand count, you’ll only get the first preferences and transfers, as that’s the only information available to anyone.