Uncertainty and polls

It was Christmas of 2006. My family was gathered in my Aunt’s family room after the usual Christmas dinner. The subject of conversation turned to politics which as usual, got me all excited. Being a partisan Democrat I was still elated by the results of the 2006 Congressional elections. After talking about the Democratic Takeover a friend of the family asked “So are you excited about seeing Hilary and Rudy Giuliani go at each other in 2008?” I hesitated and then expressed some reservations about Giuliani’s chances in a Republican Primary; his stances on social issues, a possible inability to resonate culturally with the republican base and his relatively small experience campaigning on a national stage were all things I cited. My Uncle seemed surprised by my analysis and mentioned that the polls he had read seemed to indicate that a Rudy/Hilary match up was likely, if not inevitable. Although my instincts seem rather prescient in hindsight its easy to see why my Uncle, like many others, had those expectations.

In an age where Americans are getting the information they want easier, quicker and more conveniently than ever the result of a Presidential election certainly goes against the grain. Its hard to even remember when this presidential campaign started, Democratic front-runner Hilary Clinton officially announced her bid on January 22nd 2007 (http://www.cnn.com/2007/POLITICS/01/20/clinton.announcement/index.html), McCain on the March 1st edition of the David Letterman show (http://www.cnn.com/2007/POLITICS/02/28/mccain.running/index.html), and Barck Obama on February 10th, 2007 (http://news.bbc.co.uk/2/hi/americas/6349081.stm). The speculation, of course, started much earlier. I should mention that in the last national Newsweek Poll released on October 24th 2008 already contains some data on the 2012 Republican Primary with Mitt Romney leading the pack with 35% of registered Republicans, Mike Huckabee with 26% and Sarah Palin with 10% (http://www.newsweek.com/id/165570). Are we now destined for an Obama/Romney matchup in 2012?
Hardly. First of all, Obama hasn’t been elected to his first term yet. Secondly, the people conducting this poll for Newsweek presented respondents with a series of options. We don’t know if the candidates who were presented as options will even be running, yet alone if they’ll be politically viable. Furthermore, figures can suddenly appear on the national stage; just ask Barack Obama. Its easy to imagine someone like David Petraus having a significant effect on this poll were he listed as an option. And what about the voters of Iowa and New Hampshire? Someone like Tim Pawlenty may be able to connect with Iowa caucus goers more efficiently than Mitt Romney, much like Mike Huckabee connected with Iowa Republicans last January. Barack Obama certainly looked like the frontrunner of the Democratic contest after winning the Iowa caucus.

I think its important to note that although polls can be useful in terms of identifying trends, they’re hardly a flawless predictor of future events. That being said, I still find myself spending hours of my personal time reading polls. There is a dizzying array of polls on this election. I follow eight national tracking polls, one tracking poll that covers Pennsylvania and several state polls. I check three websites daily that assemble, weight and average statewide and national polls. I read crosstabs, compare the newest polls with other polls, check sample sizes, party identification, the record of the group conducting the poll and the pollsters own analysis of their work. I’ve began to wonder though, as interesting as the daily “horse race” is - are the polls I’m reading really giving me a greater ability to predict the results of this election? In an effort to quell that concern, I looked back at the accuracy of polls on past elections and found that process an insightful one.

One historical trend I found is of great relevance to the 2008 election. Although Gallup’s final pre-election polls aren’t always accurate at predicting the popular vote totals there have only been three occasions since 1980 where whoever was ahead in the pre-election poll three weeks prior to the election did not win the popular vote. The first of those occasions occurred in 1980 when Carter led Reagan by three points the third week of October. Reagan ended up winning the popular vote in that election by nearly 10 points. Its important to note though that the only presidential debate in 1980 occurred on October 28th and was largely regarded as a turning point in that campaign. The other incidents happened in 2000 and 2004 where Gallup had both sets of candidates tied in mid-October, Gore won the popular vote by .5% and Bush by 2.4% respectively. I should mention that Barack Obama was head by 7 in a Gallup poll taken three week prior to this election. In every election aside from 1980 with a clear frontrunner three weeks prior to the election (84, 88, 92, 96) Gallup accurately predicted the winner of the popular vote. In an issue of Public Opinion Quarterly Dr. Michael W. Traugott concluded that “In the end, the final estimates of the pre-election polls, the bread and butter of the polling industry, were very good at suggesting it would be a close race, with Bush the likely winner. In historical perspective, the overall performance was above average for the period since 1956.” (http://poq.oxfordjournals.org/cgi/content/full/69/5/642)

One way analysts try to predict the results of a presidential election is by assembling polls of individual states in an effort to determine a likely winner of the electoral college. The website http://www.electoral-vote.com correctly predicted the winner of the Electoral College on their last prediction on November 3rd. In 2004 Zogby International infamously predicted that John Kerry would win the Electoral College by over 300 electoral votes (http://www.freerepublic.com/focus/f-news/1297588/posts). In 2000 ABC News, The Associated Press, CNN, Fox News, NBC News, The Orvetti Report, President Elect, the President Elect computer, Portrait of America, and the Washington Times all correctly predicted the winner of the Electoral College although none of these organizations predicted the actual electoral vote total. The website FiveThirtyEight.com has become popular in this election cycle partly due to their system of analysis out performing other analyst’s predictions of several contests in the 2008 Democratic Primary. The site is run by Nate Silver who compiles polling data using methodology similar to baseball sabermetrics. Silver currently predicts Barck Obama to win the Electoral College with over 300 votes.

The Historic nature of Barack Obama’s candidacy has some analysts wondering if Barack Obama might underperform expectations set by polls due to racial motivations of voters who are afraid to disclose their true preference. This phenomenon is commonly referred to as the “Bradley Effect” named after California gubernatorial candidate Tom Bradley who underperformed pre-election polls in 1982. Interestingly enough Bradley’s performance may have had less to do with race and more to do with polls of the contest consistently being within the margin of error and a larger push for elderly Californians to vote absentee leading Republican leaning voters to be under-sampled in polls (http://www.realclearpolitics.com/articles/2008/10/the_bradley_effect_selective_m.html). A new analysis done by Harvard Post-Doctoral Fellow Daniel J. Hopkins using new data from 133 gubernatorial and Senate elections from 1989 to 2006 found that “before 1996, the median gap (underperformance) for black candidates was 3.1 percentage points, while for subsequent years it was -0.3 percentage points.” Hopkins also argued that part of the “Bradley” or “Wilder” effect (referring to Virginian Douglas Wilder the first African American governor in America) may be related to individual candidates status’ as a front runner which may also lead to favorable polling inaccuracies. After doing a survey of the results 31 contests from the 2008 Democratic Primary FiveThirtyEight.com’s Nate Silver concluded that Barack Obama overperformed expectations set by polls in 19 States including states with higher percentages of white voters like Iowa, Wisconsin and Oregon (http://www.fivethirtyeight.com/2008/08/persistent-myth-of-bradley-effect.html). A Presidential Election is certainly different than a Democratic Presidential Primary though, so its hard to tell what effect racially motivated voters will have on the final results.

Another difficulty in presidential polling is determining which voters will actually show up to the polls. Gallup has used whats called a “Likely Voter Model” with mixed results, most infamously in 2000 where on October 24th Gallup had Gore ahead of Bush by 1 point and three days later they had Bush ahead by 13 points. This cycle Gallup has a “Traditional Likely Voter Model” meant to mirror the 2004 electorate and an “Expanded Likely Voter Model” that estimates higher turnout by young Americans and minorities. Under this “Traditional” model a voter can be registered and say they’re determined to vote and still be excluded from the model if they didn’t vote in 2004. There has also been some recent concern that pollsters are undersampling young Democratic leaning voters because of a prevalence of cell phone only households among those under 30. Pollster Ann Selzer estimates that as many as 50% of voters under 30 rely on cell phone service (Selzer & Co).
Polling has been especially erratic this election cycle, a current survey of national polls has Obama’s margin of victory as low as 3 points (Rasmussen Reports) and as high as 14 points (Pew Research Center). There is a lot of anecdotal evidence to add to all of this uncertainty. On August 5th the New York Times reported that reported that “since 2005, Democrats have gained some 214,000 voters in the 26 states with available registration data while Republican registration has dropped by more than 1.4 million”. More than 540,000 votes have already been cast in Georgia and black voters have made up a disproportionately high percentage of those votes, accounting for 37 percent of Georgia's early votes when 29 percent of the state's 5.6 million registered voters are African American (Associated Press), this may be some indication of unprecedented turnout among African Americans. Oregon’s Vote By Mail system provides an interesting study for voter turnout. The amount of ballots cast prior to October 29th 2008 represent a 19% decline in turnout compared to the ballots cast before October 29th 2004. Interestingly enough is to compare the difference between 2008 and 2004 in specific counties. In traditionally Democratic counties like Lane, Benton and Multnomah County the decline in turnout is between 12 and 16%. In Republican leaning counties like Klamath, Douglas and Linn County the decline in turnout varies from 16% to a staggering 41.5% (http://www.sos.state.or.us/elections/). This will certainly have some impact on races in Oregon, but again, if this trend is some indicator of a national trend in GOP enthusiasm and manifests itself in other states it could lead to some very interesting results on November 4th. The reason why I bring up all this anecdotal evidence is to illustrate the point that there are important trends that may not manifest themselves in polling. If certain groups are undersampled or oversampled then pollsters “Likely Voter Models” will be useless.

That being said I do feel that following polls, in addition to being a fun hobby, is a valuable practice. This is especially true when compared to the alternative of not following polls. Knowing that a certain candidate is ahead or behind can provide a meaningful context in which to view important events in a presidential campaign. If a candidate does something drastic like running a negative ad or naming an unknown Governor of Alaska as his Vice Presidential Candidate knowing that they’re behind in the polls may give you some insight into their decision making process. If you can’t decide between making a donation to a local political campaign or a national one it might be helpful to know candidates viability. Polls are great at providing some foresight as to who will decide the future of our communities and ultimately our country. By many traditional metrics it looks likely that Barack Obama will be the first African American president of the United States but things are far from certain. Unprecedented events occur on a regular basis and that’s what keeps me interested in politics (and checking dozens of polls everyday).