How Google predicted Brexit when the polls got it wrong
The 2015 general election was the last time we got to see the pollsters in action. Despite the contest becoming the most polled in British history, all predictions utterly failed to call a Conservative majority, instead opting for a minor Tory win or hung parliament.
Fast-forward to 2016 and, alongside all the mindless chatter about the EU referendum, Google pushed out two key pieces of data that proved (for the most part) much more prescient than the predictions of pollsters with powerful paycheques.
So, could Google take the mantle from polling agencies and use its real, representative and valuable data to help determine the outcome of the next major national political event?
How did the polls get it so wrong?
It takes some real gumption to stick your neck out and call a referendum result. Especially on a topic as divisive as EU membership. Then again, that’s how pollsters earn a living.
No exit polling was permitted for the EU referendum – as is customary with general elections – so the pollsters made their predictions earlier.
Market research company ComRes signalled its belief that the Remain campaign had shifted up a gear, with their final poll calling the outcome as 54% to 46% in favour of Remain. Polling agency YouGov predicted the same result, but by a smaller margin, stating a 51% to 49% preference for Remain. The Telegraph aggregated six of the last polls released before the referendum, and called the result in favour of Remain with a split of 51% to 49%.
Some of the lesser-known polling players – TNS and Opinium Research for example – did find in favour of the Leave campaign, but many commentators just shrugged their shoulders and said it was too close to call.
“My apologies if that is not precise enough for you. If you need a more exact forecast, I suggest you toss a coin or ask an astrologer.” Peter Kellner, former YouGov President.
Google search traffic and the referendum campaign
In among all the excitable warbling, Google analysed national search behaviour to see what insights could be gleaned. On the day of the referendum vote (23 June 2016), Google became an understated voice of reason. The search giant released an interactive heat-map of the UK that drew together search traffic from 31 May to 7 June in order to predict a winner.
Without ever claiming it as any kind of official poll (instead, only touting it as an overview of popular search terms), Google’s map effectively predicted that the Leave campaign would comfortably win the referendum.
The data showed that the majority of searches were for terms relating to with the Leave campaign. Although open to interpretation, the publication of this map and subsequent referendum outcome indicates that there was clear evidence of the result well ahead of the announcement on Friday morning.
Google Trends and the Brexit vote fallout?
And, so it was that Google’s prediction was borne out. The referendum result (51.9% – 48.1% in favour of Leave) exposed numerous divisions within the electorate:
- Geographical: The UK will be divided from Europe, and quite possibly, from Scotland and Northern Ireland.
- Age-related: Younger voters predominantly voted remain, whilst older people opted for Leave.
- Socio-economic background: Regions that have received low levels of government funding in recent years were more likely to vote Leave.
So, maybe it’s no wonder that there wasn’t a politician or pollster savvy enough to untangle this unholy mess.
But, even as the Westminster bubble grappled to steady the ship against turbulent waters, Google Trends gave us a second little nugget of insightfulness (or, as some might call it, ‘rubbed more salt into the wound’).
Early risers woke up to the following face-palm moment on Friday 24 June around 06:00 GMT, shortly after the overall result was confirmed:
*Courtesy of Twitter
People searched ‘What is the EU?’ three times more often after the referendum result than before it.
Remain commentators claim this trend provides evidence that the Leave voters did not make a fully informed decision.
Both the Remain and Leave camps simplified complex political arguments during the campaign. Google’s data suggests that neither camp successfully educated the public on the basics of EU membership, or the potential consequences of leaving.
Unsurprisingly, a slew of somewhat despondent (hopefully, tongue-in-cheek) search terms also rose close to the top, including “Getting an Irish passport”, “How to emigrate” and “Buy gold”.
Is all what it seems?
But, again, the reality is not quite what the statistics would make you think. Google accesses a massive range of data covering the entire nation, and media agencies can make statistics fit just about any headline. A few clever bods with serious SEO skills dug deeper into the data and found all was not quite what it first seemed.
Google Trends data doesn’t simply record the number of searches for a term. It analyses the proportion of total search traffic dedicated to a specific keyword set at a given time, within a prescribed geographical area. The spike appears smaller when put into context. “What is the EU?” was only really motoring between 06:30 and 10:30. “Weather forecast” then took over as the most searched term (us Brits really know how to party).
*Graph courtesy of Google Trends
Google’s stated hike of 250% in search traffic for this term was also somewhat misleading. In the month prior to the referendum vote, people searched “What is the EU?” 261 times each day on average. Based on that figure, this ‘spike’ only really constituted the queries of fewer than 1,000 people.
So, could Google Trends predict political outcomes more accurately than the polls?
The only thing we can be sure about at this moment in time is that the polls were inaccurate. This wasn’t the first time they’ve failed to properly grasp the voting intentions of the electorate in recent political history.
Politicians, and the general public, are clamouring for a more accurate alternative to the pollsters.
Does Google have what it takes to provide what so many high-paid pollsters couldn’t? Here’s the case for and against:
- Google’s data collection is almost universal. Aside from a few self-flagellators that still use Bing and/or Yahoo!, Google collects data from pretty much everyone. This is in stark contrast to current polling methods. These usually focus on a batch containing just a couple of thousand people who aren’t necessarily representative of wider trends.
- Google collects information with additional depth. Governments can spend years deciding on the wording for referendum questions, and pollsters follow the same logic. On Google, people use their own language. Rather than a binary ‘yes’ or ‘no’, Google Trends lets us dig deeper into people’s true feelings and intentions.
- Google shields itself from political bias. The Telegraph will promote certain pollsters while The Guardian will utilise others. Google doesn’t discriminate against voters of any stripe. It compiles queries that reflect a wider cross-section of the public, with less political bias than some of the traditional pollsters.
- Google can’t always contextualise search queries. Statements like “What would happen if the UK voted for Brexit?” could be construed in different ways for both the Leave and Remain camp. The final information source chosen may give some evidence of intention. However this would mean a whole extra (and more complicated) parameter to consider.
- Google can’t quantify how searches volumes correlate with votes. Even if the intentions of the people that searched for certain terms could be deciphered, there’s no way that Google could know the proportion of these people who actually turned out to vote, or were even eligible to do so.
- Google can’t take account of people who don’t use its service. Not everybody felt the need to ask questions of Google in order to make a decision about which way to vote. As such, Google cannot include this portion of the electorate in its analysis.
Google undoubtedly beat the pollsters when it came to predicting the outcome in advance. And, while the “What is the EU?” story might not have been everything it first seemed. It was based on concrete numbers relating to a near-universal sample – a concept alien in traditional polling. Google could bring so much more to our political process. It could help predict voter preferences and highlight important campaign issues.
Traditional polling methods have been left floundering. There’s certainly no reason that search behaviour shouldn’t be used as an indicator of voter preference at future elections. And, with a little fine-tuning, Google could one day be putting pollsters out of a job.