Are the Repeated failure of Polls to Predict Canadian Elections due to a form of Observation Bias?

In recent years, polls have failed to predict the outcomes of numerous federal and provincial elections. In last week’s Ontario election, poll aggregator Eric Grénier predicted a Liberal minority government with the possibility of a Progressive Conservative minority government. These predictions were based upon aggregating the best-available information but proved to be quite different than the eventual election result. The failures of polls to predict the election of a Liberal majority government in Ontario, follow larger predictive failures in the last Alberta and British Columbia elections and a general failure to predict the 2011 federal election. Jonathan McQuarrie wrote about these difficulties for and sees this as part of a larger Canadian crisis of confidence in social scientific evidence.

The difficulties of predicting Canadian elections stand in stark contrast to the United States, where polling has been highly successful at predicting presidential, senate, congressional and gubernatorial elections in recent years. This raises the fundamental question as to why American pollsters are effective in predicting elections but Canadian ones are not. Grénier suggests that the failure lies with the methodology of Canadian pollsters, either in which voters they reach or the ways in which they predict which voters will turnout. Political polling is a similar exercise to social scientific research and they see its difficulties as being solvable with better research methodology.

However, the problem is not in the calibre of polling but rather that polls shape voter behaviour to a greater degree in Canada than in the United States. The social scientific ideal is to be able to observe behaviour without the observation shaping that behaviour. We can later make conclusions and try to shape future developments based on this knowledge. This is particularly the case for historians, as our study of the past cannot change human actions that have already occurred. Sociology and anthropology both have considerable literature about the how to deal with situations in which observing human behaviour changes the behaviour being observed. Much of that is relevant to the failure of polls to predict Canadian elections.

Canadian polling is implicitly compared to the United States. However, we have fundamentally different political systems, particularly as the US operates on a two-party system. Only Saskatchewan provincial elections are similar, and these have the best polling track-record in recent years. Meanwhile, there are four-party systems in Quebec and Alberta provincial elections and Quebec federal elections. The remainder of provinces have three-party systems. A key difference between two and three party systems is that there is a noticeable portion of the Canadian electorate that is comfortable voting for more than one party and makes a decision in the final week based upon electability. These are most frequently centre and left-of-centre voters who support either the NDP or Liberals but the switch their voting intentions in the last day or two of a campaign out of fear of a Conservative governments. To a lesser extent there are voters willing to switch between other combinations of parties. These are engaged voters who prefer one party and inform pollsters of such, but vote to avoid a disliked government. For them, polls serve not to predict behaviour but to shape it. Looking at polling errors in recent elections, I expect such engaged last-minute swing voters are between 5 and 10% of the Canadian electorate. I also expect that they are disproportionately young and urban. The ways in which polling shapes their voting explain many of the discrepancies between pre-election polls and voting results.

The last-week volatility of Canadian voters has been pronounced since at least the 1988 federal election. In 1993, the final pre-election polls predicted a Progressive Conservative minority government. In the 2004 federal election, fear of a Conservative minority government led about 5% of Canadian voters to switch from the New Democratic Party to the Liberal Party on the weekend before Election Day. The reverse happened in 2011, when the NDP gained support in Quebec in the middle of the election campaign and overtook the Liberals as the second-most supported party nationally with two weeks remaining. At that point, many Liberal supporters switched to either the NDP or Conservatives to avoid a government of the other party, so that between 10 and 15% of Canadian voters switched support during a one-month campaign. Polls changed perceptions of electability in that campaign and the volatile voter behaviour cannot be understood without their influence. Another example is that 2012 Alberta provincial election when polling indicated that the Wildrose Party was headed to a majority government. Those polls inspired a successful viral campaign, along with many centre voters switching their allegiance to the Progressive Conservatives in the last days of the campaign.

American political polling appears more social scientific because it generally tracks the voter behaviour of more a more polarized country and does not influence it. Similar methods in Canada have less predictive power because our three and four-party systems mean that they influence some voters to switch allegiances. In this way they play a more important function in Canadian elections than the American equivalents. They cannot be understood as anything more than a reflection of voter intentions on the day being surveyed but should continue to strive for the best standards at those times.

The more volatile Canadian electorate also speaks to some other factors that distinguish Canadian elections. Compared to the United States, parties have considerably less organization and member involvement. As its effects would be less predictable on election outcomes, Canadian parties also do not attempt to organize Get Out the Vote efforts which compare to the American ones. Meanwhile, volatile Canadian voters mean that elections are often contingent on polling. Elections have a large impact on policy and people’s lives so it can be nice to think of them as the product of larger structures. People disagree about the relative merits of the 2¢/kWh subsidy to Northern Ontario heavy industry or the creation of an Ontario Pension Plan that have been promised by Kathleen Wynne. However, one cannot dispute that these will have a large influence on the future of Ontario and the lives of Ontarians. Policy played a large role the last campaign and will be its lasting impact, but polling mattered and its distinctly Canadian importance should be valued.


Leave a comment

Filed under Uncategorized

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s