Annals of Youtube Science 2(3): L’Arab Barometer

Dans la lignée des billets précédents sur les enquêtes d’opinion (les sondages), et notamment des billets :

… je reprends la série de vidéos informatives sur la mesure des attitudes à partir de données d’enquête en population générale.

En voici une sur l’Arab Barometer, mise en ligne par l’association marocaine TAFRA (que j’ai eu la chance de découvrir, grâce à un ancien étudiant et ami, lors de mon dernier séjour dans le pays, et dont le site Web est extrêmement riche : memos, vidéos, rapports…) :

Je n’ai jamais utilisé les données Arab Barometer, ni Afrobarometer ou AmericasBarometer d’ailleurs, mais je connais un peu les données WVS (World Values Survey), et la vidéo ci-dessus donne furieusement envie d’aller voir les questions, les réponses et ce qui est prévu pour les vagues à venir.

Annals of Youtube Science 2(2): Les échantillons non probabilistes

Ce blog peine à sortir de sa torpeur éditoriale, et il s’est déjà écoulé six mois entre l’épisode 2(1), qui parlait lui aussi de sondages, et celui-ci. Plus d’activité à la rentrée, promis.

Le Pew Research Center (chaîne Youtube), déjà évoqué dans l’épisode 1(14) sur les échantillons aléatoires, vient de publier cette vidéo sur les échantillons non probabilistes, c’est-à-dire la vaste majorité des échantillons :

Les enquêtes dites « opt-in », menées sur des échantillons de volontaire via des plateformes Web, sont un exemple d’échantillon non probabiliste ; les enquêtes « par quotas » en sont un autre, très différent, mais dans les deux cas, les calculs des marges d’erreur, les « erreurs standard« , cessent d’être paramétriques – les erreurs standard cessent précisément d’être « standard »1.

Ce que montre la vidéo ci-dessus, c’est qu’il est tout à fait possible de s’engager dans un travail de collecte de données sur une base non probabiliste, puis de mettre en place la recherche méthodologique nécessaire pour chercher les erreurs non paramétriques à y apporter pour pouvoir en généraliser les résultats2.

C’est la seconde étape qui fait souvent défaut dans les sondages par quotas publiés par les instituts privés, ou dans leur utilisation pour assembler certaines enquêtes scientifiques comme l’European Values Survey (EVS). J’en ai déjà parlé à multiples reprises.

D’une certaine manière, la vogue actuelle pour les « big data » et la « data science » contourne quasi-complètement cet obstacle, en faisant reposer l’analyse sur des échantillons sciemment non représentatifs (les utilisateurs de Twitter, par exemple), et en travaillant sur « la totalité des données disponibles », bien qu’empiriquement, cette « totalité » reste dans la plupart des cas un… échantillon des données (Twitter, toujours).

  1. En économétrie ou en statistiques appliquées, on peut faire toute une carrière sur la correction de ce problème. Dans le reste des sciences sociales, on peut faire toute une carrière en ignorant – plus ou moins volontairement – ce problème. []
  2. L’inférence visera généralement la population générale, souvent adulte, et en science politique, encore plus souvent restreinte au corps électoral. On ne manquera toutefois pas de faire commencer le communiqué de presse par « Les Français ». []

Annals of Youtube Science 2(1): Sondages et taux de non-réponse

Il est temps de reprendre la série de billets Annals of Youtube Science, dont le « volume 1 », publié entre janvier et septembre de l’an dernier, compte 16 épisodes. Cette année, je vais faire plus « light« , mais continuer quand même : c’est un bon moyen pour moi de faire le tour des canaux Youtube sur lesquels on trouve régulièrement des conférences intéressantes. J’en profite pour indiquer que je viens de rétro-publier l’épisode 1(12), qui était resté coincé dans le purgatoire des billets au format « brouillon ».

Roger Tourangeau, de l’entreprise américaine Westat, a donné fin 2017 une conférence passionnante au Santa Fe Institute (canal Youtube) sur les paradoxes d’un phénomène bien connu dès que l’on s’intéresse aux enquêtes de population — les taux de non-réponse ont explosé, mais pas les erreurs moyennes sur les quantités dont on connaît la valeur réelle, comme les résultats électoraux :

La slide qui résume le paradoxe :

Le problème est le suivant : sur les 25 dernières années, les taux de réponse aux enquêtes en population générale ont chuté de manière dramatique, vers des taux à un chiffre. On pourrait se dire que c’est le bon moment pour en revenir – car c’était la norme, en gros, avant la Seconde guerre mondiale – à des méthodes non probabilistes, de type « échantillonnage par quotas » — mais ce serait ignorer le paradoxe soulevé par Merkle et Edelman : le taux de réponse ne détermine pas l’erreur moyenne de mesure.

Tourangeau donne tout ce qu’il faut pour comprendre les fondements empiriques et théoriques du paradoxe—paradoxe peu connu (ou peu enseigné, en tout cas) dans les pays européens, où les instituts de sondage ne font, de toute manière, que très peu d’échantillonnage réellement aléatoire : Tourangeau cite, à titre d’exemple, la kyrielle d’enquêtes britanniques s’étant complètement trompées sur l’élection générale de 2015.

Annals of Youtube Science 1(14): Survey Methods 101

Le Pew Research Center (chaîne Youtube), qui fournit d’excellentes données socio-démographiques sur la population américaine, a lancé une série de vidéos sur les méthodes d’enquête, dont voici le premier épisode sur l’échantillonnage aléatoire :

Cette vidéo tranche avec les erreurs et les approximations que l’on trouve malheureusement dans plusieurs autres descriptions « grand public » des méthodes d’enquête, ce à quoi il faudrait faire quelque chose, mais le temps manque.

Les autres vidéos de la série « Methods 101 » du Pew Research Center sont identifiables via leur blog Fact Tank, également présent sur Twitter.

Et pour un autre exemple de très bonne vidéo explicative sur les difficultés des enquêtes en population générale, voir « How Pew Research Center Conducted Its 2017 Survey of Muslim Americans », au sujet de cette enquête.

Les législatives britanniques 2017 en quelques mots

Quelques mots sur les législatives d’outre-Manche. Pour ce qui concerne les élections législatives françaises de 2017, qui commencent aujourd’hui, j’ai prévu de publier un billet le 13 juin et un autre le 21 juin.

L’élection générale britannique avait lieu jeudi dernier, et l’on pouvait suivre l’événement en direct depuis la London School of Economics (sauf mention contraire, les images de ce billet proviennent du stream) :

Le live valait son pesant de cacahuètes, avec beaucoup d’intervenants, et beaucoup d’humour, principalement au détriment de la campagne désastreuse de Theresa May (je vais y revenir). Les réactions de la salle étaient également amusantes. Extraits de mes notes au moment de l’annonce du premier exit poll :

[ BBC speaker: ] « …My giant touchscreen is being loaded with data… »
[ room bursts into laughter ]

[ BBC speaker: ] « Tories lead the vote… »
[ room softly saddens ]
[ BBC speaker: ] « …but are short of a majority. »
[ room literally explodes with joy ]

Un peu de contexte

La séquence électorale britannique de ces dernières années est impressionnante : referendum sur l’indépendance de l’Écosse en 2014, élection générale en 2015, referendum sur le « Brexit » en 2016, et élection générale cette année. Les marchés financiers, qui ont décroché depuis le « Brexit », risquent de ne pas bien accueillir non plus le résultat complexe (voir plus bas) de la semaine dernière, car ce résultat n’élimine pas la possibilité d’une autre élection générale avant 2022…

May avait promis de ne pas déclencher d’élection générale anticipée, puis finalement l’a fait, avec une avance de 20 points dans les sondages, qu’elle s’est appliquée à perdre au terme d’une période pré-électorale désastreuse pour son parti : May ne s’est pas présentée au débat télévisé, a remis le débat sur le fox hunting sur la table alors que la majorité des Britanniques n’en veut pas, et surtout, a dû défendre un programme qui ne prévoit aucune augmentation des salaires dans le secteur public et plusieurs régressions en matière de protection sociale, dont la désormais bien connue dementia tax.

Le Labour de Jeremy Corbyn a mené une campagne diamétralement inverse d’un point de vue stratégique : Corbyn s’est exposé au risque d’organiser des rassemblements de masse, a présenté un manifesto anti-austérité mais formulé dans des termes sociaux-démocrates raisonnables, et semble enfin avoir réussi à mettre une sourdine sur les accusations d’antisémitisme qui ont miné l’image de son parti l’an dernier.

La combinaison des deux campagnes semble pouvoir expliquer le revirement électoral spectaculaire mesuré par les sondages d’opinion entre l’annonce de l’élection et le vote – ce qui n’arrange pas nos affaires : il y a toute une littérature – à découvrir en partant de cet article – pour défendre l’idée que « les campagnes [électorales] ne comptent pas », ou peu, y compris au Royaume-Uni. À première vue, c’est exactement l’inverse qui s’est produit ici. Sincèrement, more research is needed!

Les résultats

Les résultats sont compliqués : les Conservateurs se retrouvent à nouveau en situation de devoir composer une coalition parlementaire – avec les unionistes irlandais ! – pour rester majoritaires à la Chambre des Communes1 ; le Scottish National Party perd sa dynamique électorale positive et régresse de 21 sièges ; et le Labour fait un très bon score de parti d’opposition, mais reste très loin d’être en mesure de revenir au gouvernement2.

Dans ce graphique de Martin Baxter, qui résume très bien à la fois la configuration électorale et les estimations des instituts de sondage, le score effectif de l’élection générale se situe dans la zone nord-ouest du cercle « low-pollsters« , à la frontière entre les zones « Con[servative] minority » et « Con[servative] majority » :

La diagonale située dans le cadran nord-est du graphique est importante, car elle trace la zone de « pure two-party vote« , où tous les partis hors « Tories » et Labour obtiennent 5% ou moins des suffrages. En 2010, mais surtout en 2015, Patrick Dunleavy (et d’autres) avaient annoncé la mort des lois de Duverger au Royaume-Uni ; les résultats de cette année confortent cette idée, vu que la coalition au pouvoir sera, à nouveau, et malgré le mode de scrutin, multi-partite3.

Ultime satisfaction

En traînant un peu sur Twitter pour y lire les excellentes notes de Martin Goodwin ou de Kiko Llaneras, je suis tombé sur la non moins excellente synthèse de Mathieu Gallard, et en particulier sur ce graphique, où l’on aperçoit, miracle, enfin, des intervalles de confiance (à ±4 points) dans une prévision électorale !

Il semble que les vives critiques adressées aux instituts de sondage britanniques suite à leur plantage complet de 2015 aient servi à quelque chose. C’est un stéréotype culturel que je tiens de mon séjour en Écosse : dans bien des domaines, les Britanniques sont meilleurs que « nous » lorsqu’il s’agit de publiquement reconnaître leurs failles pour tenter de les corriger.

Addendum #1 : pour le détail des résultats, je conseille cet autre fil Twitter de Mathieu Gallard, très en forme, et qui va certainement aussi faire du super boulot ce soir.

Addendum #2 : cette excellente analyse de Fabien Escalona se concentre sur le succès électoral du Labour de Corbyn. Fabien semble minorer certains des éléments internes au parti qui ont limité son audience électorale depuis 2015 : je pense tout particulièrement à la gestion par Corbyn des accusations d’antisémitisme. Mais hormis sur ce point, je souscris pleinement à son diagnostic, que j’aurais été bien incapable de formuler de manière aussi précise et aussi juste.

Addendum #3 : et un dernier lien vers ce billet de blog sur le two-party system, qui se porte plutôt bien en 2017 (au moins par rapport à 2015). Et cet autre billet vaut aussi le détour. Ah, et aussi ce billet, sur la prédiction réussie du modèle YouGov MRP.

Addendum #4 : et voici le briefing de la Chambre des Communes sur l’élection générale 2017 (via Mathieu Gallard), en espérant qu’un jour, la France pense à se doter d’un Parlement capable de produire des documents d’aussi bonne qualité…

  1. Comme le rappelle Henry Farrell dans le lien fourni au sujet du DUP, ce n’est pas la première fois que les conservateurs (anglais) font alliance avec les unionistes, et cela ne présage rien de bon pour l’application du « Brexit » en Irlande du Nord. []
  2. On peut spéculer sur la perspective de voir le maire de Londres, Sadiq Khan, dont l’image publique est excellente, se présenter pour le parti (néo-)travailliste dans les années qui viennent. []
  3. Certaines analyses parlent d’un retour au bipartisme en Angleterre – et d’un retour au quadripartisme en Écosse –, mais c’est un bipartisme perturbé : certes, les Liberal Democrats ne redeviennent pas un troisième « parti effectif » et les deux principaux partis passent au-dessus des 80% des suffrages, mais la répartition des sièges est suffisamment partagée entre eux pour n’en consacrer aucun sans l’aide de partis d’appui (même le Labour cherche des alliés dans l’opposition). []

A post-mortem of the French 2017 presidential election, 2/2

Suite et fin de mon post-mortem présidentiel 2017. Spécial dédicace à Diane et Frédéric, et un grand merci à Guillaume, qui m’a fait découvrir l’histoire politique de la Finlande, ainsi qu’à Emmanuel, qui se souvient bien de ses cours de lycée.

Mise à jour : fin de billet complétée, notamment suite à l’annonce du retrait de Marion Maréchal-Le Pen, et aussi parce qu’il me manquait la partie sur Emmanuel Macron, sortie plus tard que prévu des tiroirs.

tl;dr: The French 2017 presidential election was won by the spiritual son of centre-left French politician Dominique Strauss-Kahn and centre-right politician François Bayrou, both of whom failed to to claim the presidency before him, each in different ways.1

The newborn weighted precisely two votes out of three at birth and seems healthy. His godfathers, centre-left François Hollande and centre-right Valéry Giscard d’Estaing, two former French presidents who lasted one mandate only, must also feel very proud.

Visual summary of both rounds

More seriously now, let’s summarise the full presidential election ‘horse race’ in two graphs. Just in case you forgot about the first round, the results and their trail of pre-election “voting intentions” polls looked like this:

In the second round, the results and the “second-round scenario” answers to those same polls ended up looking like that, thereby predicting a certain victory for Emmanuel Macron, even under high abstention and/or voter uncertainty:2

Second-round polling accuracy

I owe both of the graphs above to Matthieu Gallard, who was kind enough to also share (and comment) similar graphs for the second-round polls of six out of the seven last presidential elections: 1974, 1981, 1988, 1995, 2002 (no data: read why), 2007 and 2012.

As a cursory look through the entire series of graphs should make obvious, the mean absolute error of second-round polls in French presidential elections has always been rather low, even though the polls have a strange tendency to lose, rather than gain, polling accuracy in the last days before the election.3

Second-round turnout and ‘valid’ votes

The polls also correctly predicted that second-round turnout (74.5%) was going to be lower than that of the first round (77.7%), even though it is important to understand what exactly happened here:

  • 1.5 million voters (3% of the registered electorate) completely ‘evaporated’ between both rounds. As a result, the second round of the 2017 presidential election totalled only 600,000 more effective votes than that of the previous presidential election.
  • The number of blank votes in the second round reached a record count of 3 million (over 4.5 times what is was in the first round). Although much smaller, the number of null votes was also multiplied by over 3.6 and reached over 1 million ‘spoilt’ votes.4
  • Adding up lower turnout and high amounts of blank/null voting counts yields a very low count of 31.3 million ‘valid votes’ (suffrages exprimés), of which 20.7 million went to Emmanuel Macron and 10.6 million to Marine Le Pen.

The record-high amounts of ‘invalid’ votes has lots of possible consequences, and lots of possible explanations:

  • None of the governing parties were on the second-round ballot, which made it impossible for any of these parties to coordinate the formation of a second-round front républicain against the Front national.
  • Unlike many of his coalition partners, such as the French Communist Party, far-left candidate Jean-Luc Mélenchon refused to endorse Emmanuel Macron, as apparently did a non-trivial fraction of his first-round electorate.5
  • In parallel, Marine Le Pen promised to make right-wing candidate Nicolas Dupont-Aignan her Prime Minister if she won the election, but Dupont-Aignan’s decision to ally with Marine Le Pen did not necessarily reflect his voters’ preferences.

All of these explanations might be true to some extent, but even on large samples, the subgroups of voters corresponding to each working hypothesis are rather small, which makes it difficult to test anything reliably before we get more data from the forthcoming legislative elections—read on for more on that.

Consequences of Marine Le Pen’s defeat

Between the two rounds of the presidential election, the candidate of the Front national increased its raw number of voters from 7.6 to 11.4 million, beating its own historical record from two years ago:

Two years ago, in the 2015 regional elections, the Front national managed to win new voters between the two rounds of the election. The same thing happened this year, and by that standard, the second-round score of the Front national is a tremendous success for Marine Le Pen, as her party now seems able to appeal to new voters in the decisive phase of a two-round election.

In line with what we know from studying extreme-right voters, opinion polls tend to locate most of the new second-round Front national votes within the right-wing first-round vote, and then within first-round abstentions. Given the derelict state of the French right-wing, which just lost its second presidential election in a row on top of being financially in dire straits and lacking a manifest leader, one might expect that dynamic to continue.

The right-wing won less than 9.5 million votes in the first round,6 and was absent from the second round. As a consequence, and even without adjusting for turnout (which was higher in the first round), more people voted for the extreme-right Front national in this election than for the non-extreme right wing. Again, from that angle, the French extreme-right might legitimately claim that its second-round performance was a great success.

The paradox here is that all the “good news” listed above sum up to a dilemma for the Front national, which might follow two hardly reconcilable electoral strategies:

  • On the one hand, the party can continue backing Marine Le Pen’s “neither right-wing or left-wing” strategy until she gets 51% of the second-round vote on her own.
  • On the other hand, the party might feel that its immediate success with right-wing voters justifies switching to a coalition-based strategy, with the Front national trying to supplant the traditional right-wing as the main player.

The first strategy outlined above – called ni droite ni gauche in French7 – can be argued for by observing the very weak state of both governing parties, and by considering Emmanuel Macron as the ‘most desirable opponent’ for an extreme-right populist party. The legislative elections might also lend additional support for that strategy if the Front national manages to eliminate candidates of all parties in the second round.

The second strategy above – called “union des droites” in French – has many staunch supporters within the Front national, and even has some kind of an official representative in the person of Marion Maréchal-Le Pen (see update below). That segment of the party is either unconvinced that Marine Le Pen will ever win a relative majority of second-round votes,8 or bothered by some of the most liberal items of her moral agenda, or both.

In the worst case scenario for the Front national (and probably the best-case scenario for its opponents), the two competing strategies listed above have the potential to cause the party to split—which has happened before, and might well happen again.9 The results of the forthcoming legislative elections, which will be the last electoral results before 2020, could be determinant in that regard.

Update, May 10: just a few hours after I published the paragraph above, Marion Maréchal-Le Pen created a conflict within the Front national by “temporarily” (her word) resigning from politics, citing a mix of personal and political reasons for doing so.

Consequences of Emmanuel Macron’s victory

Let’s finish this post with the president-elect.

Except for his very young age, none of Emmmanuel Macron’s ‘static attributes’ are entirely new to French politics. Quite a few bankers have risen to high positions in French political history, and as hinted in my opening paragraphs and in a previous post, ambitious Centrist French politicians close to the median voter are also far from new.

There are, however, many new ‘dynamic attributes’ in Emmanuel Macron’s successful presidential candidacy (which, again, I failed to imagine as possible):

  • Emmanuel Macron’s En Marche ! movement won the presidential election autonomously, without forming any formal alliance with one or more of the existing governing parties. That’s a first, and a remarkable one.

    Until now, even Charles de Gaulle or Valéry Giscard d’Estaing had formed such alliances to win their respective presidencies. Similarly, most foreign examples of fast-winning presidents, such as Martti Ahtisaari‘s run for the Finnish presidency in 1993-1994, also relied on at least one governing party to do so.10

  • By going from zero past mandates to president without touching base between those two steps, Emmanuel Macron has also set a precedent that threatens not only the organisation of governing parties, where long political careers are a requirement for higher office, but also one of the main benefits of the Front national, which also offers short time-to-office career tracks for aspiring mandate-seekers.

    This, to me, is the most interesting dynamic set in motion by Emmanuel Macron: his election is a black-swan event within the overall and ever-increasing trend that has led to the professionalisation of French politics up to a very high degree. It will be very interesting to see if the legislative elections ‘upgrade’ this change to a system-level one, or if things will “revert to type.”

  • Last, by calling for a sweeping change in political representation, Emmanuel Macron has framed his candidacy as that of an ‘anti-system candidate for the educated.’

    The ‘system’ here stands for the existing party system and for some of its incumbent elites: even though Emmanuel Macron himself is hardly anti-elitist or a populist candidate, his campaign – as well as many others – did take advantage of the electoral disconnection between governing parties and voters.

    To some extent, the position described above also echoes that of Charles de Gaulle’s Rassemblement du Peuple Français at its onset, in the immediate aftermath of the Second World War.

    In its early days, the Gaullist movement, which – unlike En Marche ! – allowed its members to keep their existing party affiliations, also presented itself as a catch-all ‘movement’ with the ability to transcend the existing party system of the French Fourth Republic, and also went on to win large fractions of the vote in its first years of existence, although in rather different historical circumstances.11

    Similarly, and to a minor but still significant extent, there is also a parallel to be drawn between Emmanuel Macron’s current strategy and that of Pierre Poujade’s Union de défense des commerçants et artisans, which shook up the French Fourth Republic in its last years.12

    To be clear, Emmanuel Macron does not seem to carry any of the violently anti-Semitic traits of Poujade’s movement, which offered Jean-Marie Le Pen his first electoral mandate in 1956. That aspect of Poujade’s politics, along with other legacies of French fascism, rests with the Front national, with which Emmanuel Macron is programmatically at odds.

    Yet just like the Poujadist movement, En Marche ! movement – just renamed République En Marche ! a few hours ago – also presents itself as a threat against the established balance of power, with the ability to completely disrupt the status quo by removing many incumbents from the political mainstream. In that respect, Emmanuel Macron is the last piece of the strange ‘anti-system’ puzzle that has assembled over the years, and that now contains populist and proto-fascist right-wing parties, anti-capitalist left-wing alternatives, and now a pro-EU Centrist former banker.

That’s it for now. For a broader overview of the French political space in its current shape, I recommend this excellent article by Fabien Escalona (in French, subscription required). This quick recap by Verónica Hoyo and William M. Chandler (in English) also has most of the important facts sorted out.

  1. François Bayrou ran three times, in 2002 (arriving 4th), in 2007 (arriving 3rd) and in 2012 (arriving 5th), while « DSK » did not manage to run at all due to, well, you know why. []
  2. On that topic, you might have stumbled upon this New York Times op-ed, or this other column. As explained in an earlier post (in French), both texts are utterly wrong, for rather obvious reasons, and uninformed to the point of embarrassment. []
  3. Both links in this paragraph lead to Will Jennings’ Twitter account. Along with the other Twitter accounts mentioned in the first part of my post-mortem, I highly recommend following his tweets if you want to become an accomplished elections nerd. []
  4. Following an electoral reform passed in 2014, ‘blank’ and ‘null’ votes are now counted separately under French electoral law, yet blank votes still do not count as ‘valid’ votes. On top of that, anecdotal evidence shows that very few people know about the law and its effects anyway. []
  5. The irony of Jean-Luc Mélenchon, a former Troskyist, refusing to rally against an extreme-right candidate, was not lost on anyone. []
  6. 8.9 million votes if one counts only François Fillon and Nicolas Dupont-Aignan; 9.3 million if one also counts minor candidates François Asselineau and Jacques Cheminade. []
  7. See also Zeev Sternhell’s book on French fascism, who uses the expression in its title. The book was controversial among historians when it came out, but seems to have won the argument over the years. []
  8. After all, so far, Marine Le Pen has never won over more than one registered voter out of four, and her second-round performance, which would have won only 45 legislative constituencies out of 577, indicates that she even lost some voters in large urban areas between both rounds. On top of that, the 2015 regional elections attest that governing parties can deny the Front national any second-round success. []
  9. One reason for that is the sentiment that the Front national is coming close to reaching its ‘maximum electoral potential:’ its vote has stayed geographically stable for two decades or so, and has not increased much among either pensioners or educated voters. []
  10. I owe this analogy to my dear friend Guillaume. I know close to nothing about Finnish politics myself. []
  11. I owe this analogy to my dear friend Emmanuel. My point about En Marche ! asking its members to drop their previous party affiliations is based on an article published in the Canard enchaîné on April 26, 2017, which indicates that En Marche ! will not endorse candidates from other political parties in the forthcoming legislative elections. More will be known very, very soon. []
  12. On the Poujade movement, see this book by Romain Souillac, which is well covered in this radio show, as well as this other radio show. []

A post-mortem of the French 2017 presidential election, 1/2

Deux derniers billets en anglais sur l’élection présidentielle, et désolé si vous avez vu passer le brouillon de ce billet dans le flux RSS ou sur Twitter en avance de quelques jours. Aujourd’hui, première partie, où l’on apprend – scoop ! – que je n’avais pas prévu les résultats du premier tour. Suite et fin quand j’aurai terminé de trier mes sources.

tl;dr: I suck at electoral predictions, but that’s alright (and hardly new).

Before going further, let me disclose here that, after discovering that I was still registered to vote (…), I voted for Benoît Hamon, having always voted for the Socialist candidate when voting in France (I have also voted in Scotland; different story).

Let me also underline that, unlike my guesstimates, French opinion polls performed extremely well in the first round of this election, in line with their good – albeit, by some aspects, bizarre – historical record:1

The graphic above shows both governing parties (Socialists and Republicans) losing their candidate on the first round of a presidential election—which is unprecedented, and a potential transformative event for French politics, which had organised around a two-party quasi-duopoly. As of today, and for a few years at least,2 that status quo is gone:

Make sure to read Fabien Escalona and Donatien Huet’s analysis of the vote – in French, subscription required – to understand the details of the French political order.3 And to further understand how the French extreme-right Front national might now think of its position in that order, I recommend this op-ed by Florent Gougou, also in French.

Electoral post-mortem

I have blogged quite a lot about the ongoing French presidential election. Two of my earlier posts contained attempts at guessing the respective vote shares of Marine Le Pen, the candidate of the extreme-right Front national, and of Emmanuel Macron, a former minister of the outgoing government and the candidate of the centrist En Marche ! movement.

Both guesstimates were for the first round only, which is why I am posting this post-mortem now and not later. I did not produce similar second-round guesstimates because I do not believe that my guesstimation technique, which was based strictly on past electoral performance, can produce anything reliable about the second round; however, my first-round predictions should have resulted in a second round between Marine Le Pen and François Fillon, the right-wing candidate.

For more 2017 presidential election posts, click on this tag. Most topically, here are my electoral ‘pre-mortems’ (in French): pre-mortem, part 1, pre-mortem, part 2.

First-round turnout

  • Assumed 80%
  • Actual 77.7%
  • Δ = –2.7 percentage points

Both of my guesstimates assumed a turnout rate of 80%, based on the hight turnout rates of many past presidential elections. The actual turnout of 77.7% and the 950,000 blank or null votes in the first round reduced the effective electorate to 36 million voters, which is barely 200,000 more voters than the first round of the previous presidential election.

First-round vote for Marine Le Pen

  • Guesstimated 20% (lower bound)
  • Actual 21.3%
  • Δ = +1.3 percentage points

My guesstimate of Marine Le Pen’s vote share put her at a strict minimum of 19-21% of the vote, assuming a turnout rate of 80%. This guesstimate was based strictly on the past electoral performance of the Front national in 2012 and 2015.

The same blog post included a computation of the minimum raw votes for Marine Le Pen of roughly 7 million out of 45 million registered voters from the 2015 regional elections. The correct total number of registered voters in 2017 (47 million) was published only after I posted my guesstimate.

In the end, Marine Le Pen attracted 7.6 million votes, which makes my guesstimate pretty spot-on, largely thanks to the fact that my two computation mistakes (underestimated number of registered voters, overestimated turnout) partly cancelled each other out.

My guesstimate of Marine Le Pen’s vote share was conservative in comparison to what polls (and friends) announced: see, for instance, the better informed estimate of 21-28% by my dear friend-and-co-blogger Joël. Generally speaking, Marine Le Pen’s first-round electoral performance was clearly below expectations.

First-round vote for Emmanuel Macron

  • Guesstimated 18.5% (upper bound)
  • Actual 24%
  • Δ = +5.5 percentage points

My guesstimate of Emmanuel Macron’s vote share put him at a strict maximum of 7 million votes, or 18.5% of the vote, assuming a turnout rate of 80%. This guesstimate could not be based on the (non-existent) past electoral performance of Macron’s movement: instead, I used centrist ex-candidate François Bayrou‘s past record as a proxy for the Macron vote.

In the end, Emmanuel Macron attracted over 8.6 million votes, completely smashing my guesstimate and its corollary—that Emmanuel Macron would not be on the second round of the election, which the polls had – correctly – predicted for several weeks.

My guesstimate was to seriously underestimate the propensity of left-wing voters to flock towards non-Socialist candidates: the official candidate of the party, Benoît Hamon, scored barely 2.2 million votes (6.6% of the vote share), far below left-of-left competitor Jean-Luc Mélenchon, and far below Emmanuel Macron himself.

The first-time-ever component of the election – a (Socialist) president not seeking reelection – certainly played a role in making left-wing voters much more volatile than I would have ever imagined, but my guesstimation technique would probably have been excessively conservative even absent of that factor.

First-round vote for Jean-Luc Mélenchon

In the last two weeks before the first round, opinion polls announced that far-left Jean-Luc Mélenchon was doing really fine with potential voters, and could potentially steal François Fillon’s predicted third place, after Marine Le Pen and Emmanuel Macron. Some of my friends asked me about this, and asked for a guesstimate of his vote share.

To do so, I took his best electoral performance ever, which was his 2012 presidential election performance, where he received almost 4 million votes (11% of the vote share). Note that, at the time, Mélenchon was already supported by the same far-left coalition as he was in this election.

I then formulated three assumptions: stable voting behaviour among his past electorate, identical turnout, and uniform gains among newly registered voters, i.e. 11% of 80% of newly registered voters voting for Mélenchon. That gives you an estimate of 0.11 × 47 = 5.17 million votes, which is a huge increase of over 1 million of new Mélenchon voters since 2012.

This guesstimate of almost 5.2 million votes for Jean-Luc Mélenchon was my upper bound, because under lower turnout, one should expect the far-left vote to decrease, not increase. Furthermore, even a highly dynamic political party like the Front national did not register such an increase in raw votes between 2012 and 2015.

The staggering error in my guesstimate says something of how successful Jean-Luc Mélenchon was at attracting the left-wing vote in the first round of the election. The only correct part in my guesstimate was that he had no real chance of making it to the second round, and was instead competing for the third – and potential “kingmaker” – position.

Last first-round observations

The unusual setup and highly eventful nature of the French presidential election made it a very interesting event to follow, and many observers, including Alexandre Afonso, Jules Bonnard and Mathieu Gallard, provided first-class live reports of what was happening.4

My favourite visualization of the first-round results is this map by Alexandre Léchenet, which shows the modal (first-among-voters) candidate over geographic areas resized to their relative population:

This map is helpful to realise some of the most powerful social cleavages behind the vote: Emmanuel Macron clearly owes a large electoral debt to the urban middle class, which is more educated and therefore also more likely to vote.

  1. Note that my general views about opinion polls, which use nonrandom samples, are insensitive to high polling accuracy on one or more event(s). []
  2. The next ‘intermediary’ elections will take place in three years, and the next presidential and legislative elections in five. []
  3. For complements, see also this article (in French) by Simon Labouret and Florent Gougou, as well as Pierre Martin’s electoral reports (also in French), published in Commentaire. []
  4. Of course, the electoral campaign and first-round results also produced its fair share of implausible counterfactuals, useless fantasy maps and ecological fallacies. Too bad. []

Présidentielle 2017 : pre-mortem de premier tour, 2/2

Suite et fin de mon pre-mortem de premier tour. Le post-mortem (en anglais) suivra dans deux quatre jours après le second tour (pas le temps avant).

Derniers rappels sur le contexte

La première spécificité de la « PR 2017 » est que le président sortant ne se représente pas à la fin de son premier mandat. C’est inédit, et c’est bien la seule variable qui fasse structurellement exception ici : à part ça, l’élection se déroule au moment prévu, avec un nombre de candidats similaires à d’autres élections présidentielles de la Cinquième République, et dans un climat économique et social qui ne diffère pas radicalement de celui d’autres élections.

La deuxième spécificité de cette élection, qui est liée à la première, est que les partis de gouvernement sont tous les deux menacés d’être éliminés dès le premier tour, et la troisième, qui est liée à la seconde, est qu’un candidat annoncé comme un possible candidat de second tour, Emmanuel Macron, n’a aucun passé électoral qui permette de concevoir cette situation, mais seulement des défections.

Reste enfin le score du Front national, dont la candidate est annoncée comme candidate de second tour avec un forte probabilité, et même comme possible candidate majoritaire au terme du second tour. Au terme de mon examen de son score possible de premier tour et des arguments (spécieux) lui accordant une victoire au second, cette dernière possibilité me paraît toujours très, très, très faiblement probable.

Pronostics lus çà et là

Mon précédent billet contient mon pronostic personnel, établi sur la base très conservatrice des seules élections intermédiaires de 2015, et qui tranche nettement avec les sondages… ainsi qu’avec les pronostics que j’ai pu lire sur Facebook ou Twitter.

En prenant les pronostics de trois amis électoralistes, qui se reconnaîtront peut-être s’ils tombent sur ce billet, j’obtiens le pronostic de premier tour suivant :

  • Marine Le Pen : 26-28%1
  • Emmanuel Macron : 20.5–25%
  • François Fillon : 17–21%
  • Jean-Luc Mélenchon : 17–19%
  • Benoît Hamon : 5–8%
  • Nicolas Dupont-Aignant : 3–4%
  • Philippe Poutou et Nathalie Arthaud : 2–2.5%
  • Autres candidats : 1–1.5%

Marine Le Pen arrive en premier rang dans les trois pronostics, suivie d’Emmanuel Macron ou François Fillon, à scores quasi-égaux dans un pronostic, avec 5 et 8 points d’avance respectivement pour Emmanuel Macron dans les deux autres.

L’observation que je retiens dans ce pronostic est évidemment le pourcentage de suffrages allant à Emmanuel Macron : sur un électorat de 47 millions, il faudra au moins 7.5 millions de suffrages, légèrement au-delà du maximum de 7 millions que j’envisage de mon côté, pour le qualifier pour le second tour, dans un scenario à participation hyper-faible pour une élection présidentielle – scenario non réfuté par les sondages les plus récents, qui annoncent un taux de participation plus près de 70% que de 80%.

Quelques remarques supplémentaires, désormais.

Sur les « retraits » de candidatures

Dans les semaines qui ont précédé le premier tour et au lendemain des débats télévisés, j’ai lu plusieurs appels au retrait de tel ou tel candidat, et l’on ne manquera pas, dans quelques heures, de lire des contrefactuels du type « et si candidat x s’était retiré en faveur de candidat y, en additionnant les scores, on arrive à… et donc… ».

Pour rappel, puisque l’information va bientôt céder le pas à la spéculation sauvage, demander à tel ou tel candidat (de gauche) ne sert à rien, à moins que le candidat ne puisse rembourser par lui-même ses frais de campagne ! Quant aux promesses de remboursement en cas de retrait, je renvoie à la petite phrase de Charles Pasqua sur les promesses en politique.

La Constitution encadre à peu près la possibilité de retrait, et au-delà du délai prévu d’une semaine, le retrait est résolument l’option la moins attractive dans absolument tous les scenarios pour le candidat, et ce d’autant plus que le retrait du candidat oblige également à réécrire les accords conclus par son parti en vue des élections législatives.

Sur l’incertitude des sondages d’opinion

Sur l’autre thème porteur du moment, désormais :

Sur la même période et en réaction aux mêmes événements, quand les sondages annonçaient des progressions de +4 points pour tel ou tel candidat, j’ai lu des commentaires devenus récurrents depuis l’année dernière : « ah, mais les sondages se sont plantés sur le Brexit et Trump, donc ils auront tout faux à nouveau ! »2

Raisonnement très étrange que j’ai déjà tenté de mettre à mal, et qu’Arthur Charpentier a aussi démonté point par point, et cela bien que nous ayons tous deux de sérieuses réserves sur le calcul de probabilités à partir des sondages d’opinion.

Ces différentes remarques illustrent l’imagination débordante que suscite « la présidentielle » dans les esprits, et pourrait presque faire croire qu’on en dispose pas de documents sérieux pour documenter (1) comment se tient l’élection, et (2) comment les électeurs (et les candidats) se comportent.3

Le travail des instituts de sondage est loin d’être exempt de critiques, mais encore faut-il savoir identifier quels aspects de leur travail pose réellement problème. De mon point de vue, la « cuisine » sondagière est surtout critiquable pour ses redressements, qui peuvent notamment aboutir à ce problème :

Ces problèmes – de redressements effectués pour s’aligner sur les concurrents – ne sont qu’un aspect du problème des redressements : sur des échantillons constitués par quotas, tout redressement est par nature en violation des lois de probabilités qui permettent de pondérer un échantillonnage aléatoire—mais j’en ai déjà parlé, à de nombreuses occasions4.

Puisque j’ai cité Nate Silver, qui regarde les sondages français depuis les États-Unis, voici deux articles très recommandables d’auteurs américains sur « notre » élection présidentielle : Harrey Enten, « The French Election Is Way Too Close To Call », et G. Elliott Morris, « What to Know About the France 2017 Election, Part 1 — Polls and Predictions ». Et bien sûr, allez aussi faire un tour chez Arthur Goldhammer.

Full disclosure conclusive

Les lecteurs les plus attentifs savent que l’auteur de ce billet s’apprête à voter pour Benoît Hamon, après avoir découvert qu’il était encore inscrit sur les listes malgré une longue période de négligence totale du sujet, et après lecture du programme de Benoît Hamon, de celui de Jean-Luc Mélenchon, et de celui d’Emmanuel Macron. L’auteur a même parcouru le programme de François Asselineau, pour rendre service à une amie outre-Atlantique qui s’interrogeait sur l’offre politique autour du « Frexit ».

Je vous laisse mon intention de vote à des fins de transparence totale, et certainement pas avec l’idée d’influencer votre propre décision d’aller voter ou pas, et de voter pour tel(le) ou tel(le) candidat(e). En outre, le fait que j’ai lu les programmes de certains candidats, par exemple, ne signifie pas que je vote en fonction du programme : j’ai lu les programmes que l’on m’a tendu (biais socio-géographique), et j’en ai mis quelques-uns directement à la poubelle.

De mon point de vue, mon comportement électoral aux élections passées est un bien meilleur prédicteur de mon choix actuel que n’importe quel autre facteur—et mes prévision électorales reflètent d’ailleurs ce très fort biais en faveur des choix passés.

Dans tous les cas, entre les emplois fictifs, les costumes, le revenu universel et l’alliance bolivarienne (pardon ?), on aura vraiment eu le droit à tout au cours de la campagne :

Rideau !

Mise à jour, 17 mai : je découvre à l’instant l’excellente analyse de Léopold Mebazaa qui démontre très proprement le phénomène du herding dans les sondages français sur l’élection présidentielle. Pour les curieux, les données de son article sont sur GitHub.

  1. Soit la borne la plus haute de l’estimation de Joël, et sur la base d’une sous-estimation dans les sondages qui serait plus probable que le scenario inverse (ce qui reste à voir). []
  2. Idée que l’on retrouve en tête de cette tribune sur les biais des échantillons des sondages. La tribune est excellente… si l’on en retire les quelques lignes d’ouverture : d’une part, les sondages américains avaient bien annoncé la victoire d’Hillary Clinton au vote populaire ; d’autre part, les sondages britanniques sur le « Brexit » donnaient un résultat plus incertain que ce que l’on a pu en voir dans la presse française. Les sondages britanniques sont, en réalité, en difficulté depuis plus longtemps que le referendum du « Brexit », même si leurs performances passées furent respectables. []
  3. C’est bien évidemment faux—de même d’ailleurs que les remarques qu’on a pu lire, çà et là, sur la non-reconnaissance du vote blanc : la loi précise bien, depuis 2014, que ce vote est reconnu, mais non comptabilisé dans les exprimés. La nuance est de taille. []
  4. Il semble, malheureusement, que les erreurs les plus graves continuent de se glisser dans les médias, telle que cette vidéo du Monde, qui contient des tonnes d’approximations — et le cas n’est pas isolé. []

Sondages : Roosevelt battrait Macron au second tour

Je ne peux m’empêcher de partager cet article, sans en dire plus :

The Literary Digest poll of 1936, which incorrectly predicted that Landon would defeat Roosevelt in the 1936 US presidential election, has long been held up as an example of how not to sample. The sampling frame was constructed from telephone directories and automobile registration lists, and the survey had a 24% response rate. But if information collected by the poll about votes cast in 1932 had been used to weight the results, the poll would have predicted a majority of electoral votes for Roosevelt in 1936, and thus would have correctly predicted the winner of the election. We explore alternative weighting methods for the 1936 poll and the models that support them. While weighting would have resulted in Roosevelt being projected as the winner, the bias in the estimates is still very large. We discuss implications of these results for today’s low-response-rate surveys and how the accuracy of the modeling might be reflected better than current practice.

Si le sujet de la pondération des données d’enquêtes vous intéresse, je ne peux que recommander cet autre article :

Et si vous voulez lire plus de choses sur les sondages, allez voir chez Arthur Charpentier.

Polls, surveys and the known unknowns of the French presidential election

Un dernier billet en anglais, et je ne dirai plus rien sur l’élection présidentielle de cette année. Je ne pense même pas faire de “post-mortem” du premier tour, vu que mes prédictions n’ont rien d’exceptionnel (loin de là) et que les réponses les plus intéressantes viendront après les résultats – et l’étude des reports de voix – des législatives.

If you have read any of my two previous blog posts on the respective electoral prospects of Marine Le Pen and Emmanuel Macron in the upcoming French presidential election, you might have noticed that I did not use either polling or survey data to produce my predictions, relying entirely instead on past elections.

This is because I hold strong views about both opinion polls and electoral survey data when it comes to France. I detail these views below, as some kind of a methodological appendix to my previous posts.

Note that my electoral predictions are nothing but semi-educated guesses, and that my methodological views are just the same: I have no scientific credentials in the fields of electoral studies, opinion polling or survey methodology.

Like the previous ones, this post is written in English because it is addressed to friends who do not read French, but who keep asking about the French presidential election and the risk that it might turn into a local equivalent of the Brexit “Leave” vote or Donald J. Trump’s election to the U.S. presidency.

Update, 24-25 March: added a section on non-random polls, including a link to (and several citations from) the UK 2015 “polling miss” report, as well as a link to the French Polls Committee.

French opinion polls

As in any other wealthy representative democracy, opinion polls provide a large share of the electoral horse race spectacle. And just like anywhere else, these opinion polls have sometimes been awfully wrong, even though their average historical record is, by and large, broadly accurate (more on that later).

There are lots of reasons why one might be skeptical of opinion polls in general, and French opinion polls in particular. The classical Bourdieusian critique applies here, and French political scientist Alain Garrigou has done lots to popularise it: polls are partly performative, insofar as they frequently ask questions that respondents had not thought about before. In the context of a two-round election, this particularly applies to hypothetical “second-round” voting scenarios.1

Another problem, which is perhaps more prevalent in French opinion polling than elsewhere, has to do with sampling. In recent years, the issue of whether opt-in Internet sampling creates representative samples has attracted a lot of discussion, but an older and, in my view, more problematic issue, has to do with quota sampling, another non-probabilistic method that is widely used by French polling institutes, principally because it is much less expensive than proper random sampling.

The main problem with quota sampling is that it creates a layer of opacity around the precise re-weighting methods used to make the poll as representative as possible of the target population – an operation called “redressement” in the jargon of French polling institutes, which develop their own “in-house” methods for doing so, and then sit on it like some kind of industrial secret.2

This is a huge issue: even when quota sampling seems to be able to “emulate” simple random sampling, we have no idea how that happens. To me, this issue makes the defence of quota sampling hopeless, perhaps even more than the trivial observation that the standard error of a quota sample has no formal relationship to sample size, which makes the presentation of error margins in quota-based polling results highly misleading to the public.3

Troubles with non-random polls

Allow me to elaborate for a few paragraphs on the point made above.

The idea of “emulating” simple random sampling is, at its core, fundamentally flawed. Nature cannot be fooled: pseudo-randomness is non-randomness. Even simple random samples will contain nonrandom (missing data) components, which are hard enough to deal with. Adding non-randomness “at the root” of the sampling procedure is just madness in the present context.

And yet this is something that polling companies seem to be okay with, and not just in France: just next door, the British polling market seems to be configured exactly the same way, and even though they have less things to predict over there since the party system contains less (national) elements, they also, from time to time, get it completely wrong, precisely because their samples are non-random.

Here’s the conclusion of the committee that worked on the serious polling miss that occurred in the UK before the 2015 general election, which I mentioned earlier as the true starting point of the “Brexit-Trump” series of failed predictions:

… the primary cause of the polling miss in 2015 was unrepresentative samples… The statistical adjustment procedures applied to the raw data did not mitigate this basic problem to any notable degree. The other putative causes can have made, at most, only a small contribution to the total error. (p. 4, emphasis in original)

And a few pages down in the same document:

… all GB pollsters in 2015 took a common approach to sampling and estimation: they assembled a quota sample of eligible individuals, which was then weighted to known population totals. They asked sample members their vote intention and likelihood of voting, derived a predicted sub-sample of voters, and produced weighted estimates of vote intention for this sub-sample. (p. 22).

And then the report goes on to explain how the BES and BSA surveys are fundamentally different, because they use simple random sampling, ad because “both surveys produced good post-election estimates of the difference in vote shares between the Conservatives and Labour” (p. 27; important word here: “post” means that the surveys were run after the election, so considerations about turnout and voting uncertainty did not apply).

Issues with non-random quota samples are discussed in detail in the Francophone Wikipedia entry for opinion polls. The entry has multiple issues to do with its sources and the balance of its sections, but it otherwise makes a good read on the topic of why opinion polling is a problematic source of information, especially when it comes to voting behaviour. And if you need an example of a recent opinion poll to illustrate how black-boxed polling methodology is, here’s one.4

What can electoral polls be used for?

The flaws outlined above do not mean that opinion polls cannot produce either interesting forecasts or useful information. Measures of voting uncertainty, for instance, can be accurately measured through polls, although their generalisation to the electorate is still a very risky exercise absent of detailed information on the composition and re-weighting/redressement of poll samples. Still, compared to past elections, it seems that voting uncertainty is currently very high, especially on the left.

Another interesting use of opinion polls for electoral purposes consists in aggregating as many polls as possible, re-weight their results according to what we know of their past performance and overall reliability, and then use these results to forecast electoral results. This style of electoral forecasting has been popularised by Nate Silver‘s forecasts of U.S. elections, some of which were highly accurate, some of which were far off the mark.5

There is currently one attempt to build a Silver-style, “polls-only” forecast of the French 2017 presidential election, which came to my knowledge thanks to a tweet by François Massot. The forecast is produced by Victor Poughon and published on the “Depuis 1958” (“Since 1958”) website. The current result of the forecast, which is based on transparent polling data, unambiguously predicts that Emmanuel Macron will win the election:6

The unequivocal result of the 2017 forecast is, to me, less interesting than the “retrofit” of the model to previous elections. The Bayesian methodology of the forecast shows the model fit to the 2012 presidential election, with the difference between the actual (red) and predicted (black) results:

I also find this kind of exercise interesting because it produces a rough estimate of how much of the vote might go to the first-tier (20%+ vote shares), second-tier (10-20% vote shares) and third-tier candidates (5% or less vote shares). In 2012, these shares respectively represented around 55%, 38% and 7% of the first-round vote. For 2017, the model predicts 44%/44%/12% vote shares, which is consistent with estimates of higher voting uncertainty:

Final observations: note how wide the credible intervals reflected in the predicted vote shares shown above are. Also note that the model does not say anything about turnout, which might be lower this year. Last, note that the model cannot inform us about who is voting for whom, i.e., about voter realignments (which can also differ between presidential and legislative elections). For all these reasons, we need other forms of electoral data to supplement what polls might provide.

French electoral surveys

Especially perhaps in a multi-party system, highly detailed and sophisticated electoral surveys are much, much more important than polls to get the data generating process – i.e. what makes people cast a vote, and for whom – right. Do we have (access to) such surveys in France?

The answer is: kind of. Here are, for instance, some results from the most recent CEVIPOF/Ipsos/Steria electoral survey, the Enquête Électorale Française 2017, which also show high rates of voting uncertainty, especially when it comes to the left-wing candidates:

What credit might we lend to that survey? There is, in fact, no way to tell, except for CEVIPOF or Ipsos/Steria insiders. The survey data are not public, and the kind of sketchy technical explanations provided in the media hint that even for the official partners of the survey, French newspaper Le Monde and the Jean-Jaurès Foundation, the exact methodology that produced the data is a black box.

Lack of transparency from private polling institutes is one thing; but partly-publicly funded research is different, or at least, in my view, should be. The conclusions that insiders are drawing from the Enquête Électorale Française 2017 are highly valuable, but they do not allow outsiders to learn anything from the survey data itself, or to reuse the data in different ways. In short, French electoral data, just like many other forms of data, suffer from serious availability and cumulativity issue.

Further problems with survey data

There are several other problems with “flat microdata” (repeated cross-sections), which make up most of polling and electoral survey data.

First, to answer some questions, we need proper longitudinal (panel) data, that is, data that are produced by asking repeated questions to the same respondents at different points in time. Second, for many other questions, we need relational (network) data, of the kind produced by family-wide sampling (when the entire household is sampled). Last, in the many cases (including voting) where geography matters, we often need spatial (geolocated) data.

A few years ago, my co-blogger Joël called for the launch of a longitudinal French National Election Study, suggesting that the survey should allow multilevel analysis—that is, a form of analysis that can observe and control for the existence of hierarchical (e.g. local/regional/national) patterns:

Such a survey exists in the United States. France has no equivalent survey; the French national statistics agency does not seem to be interested in running one; and the European Social Survey, which has many of the required qualities that such a survey would call for, can only imperfectly fit the bill. Last, the French Ministry of the Interior cannot produce such a survey, out of fear of being accused to produce a partisan survey.


As a result of the many different limitations outlined above, there is no data source to turn to when one wants to analyse voting intentions in France. Sure, there are polls and electoral surveys, but neither of them are currently transparent enough to allow for independent reanalysis. Instead, what we have in France is a small club of (French) election specialists, working on disparate data sources that only very imperfectly cumulate.

Things might be getting better: France has a data repository, the Réseau Quételet, which lets researchers access some of the existing electoral studies produced under the supervision of various research teams. Note, however, that the current “French Election Studies,” a by-product of the Comparative Study of Electoral Systems (CSES), are limited to presidential-election years, and are available only for 2007 and 2012, even though they seem to have been running for longer than that.

Further reading

This blog has many posts about opinion polls, some of which focus on (older) French presidential elections: see here, here and here.

For a fuller assessment of French survey data, take a look at the appendix in Vincent Tiberj’s last book on cohort and generational effects among French citizens. The book itself is absolutely excellent, and the appendix provides a useful list of surveys – many of them cross-national – that contain data on French social and political attitudes7

  1. And outside of electoral periods, one might argue that polls co-produce the presidential approval ratings that they claim to measure. Notwithstanding, I do not agree with Garrigou that prohibiting electoral polls is the way to go. []
  2. Furthermore, there is no French equivalent of either the ICPSR or the Pew Research Center, which means that there is no repository of polling data to turn to in order to inspect raw survey data. As a consequence, the entire system of French opinion polls is effectively black-boxed. []
  3. The committee in charge of regulating polls has stated that polls should come with an empirical “margin of uncertainty” rather than a theoretical one, which actually makes things worse, since “empirical” can mean pretty much anything here. []
  4. The linked example characteristically provides none of the methodological explanations that should come with any polling result: how were the samples collected, how was nonresponse addressed, what the denominators are for population/electorate estimates, etc. []
  5. For another example of aggregated-polls-based forecasting, see Chris Hanretty’s forecast of the Italian 2013 general election. []
  6. Note that the 2017 forecast goes beyond the first round of the election, and actually attempts to predict second-round results. Also note that Macron has always been high in the approval-rating polls, which likely played a role in his candidacy. In past elections, Alain Poher and Édouard Balladur also ran for the French presidency (in 1969 and 1995 respectively) with similarly high approval ratings in the polls. Both lost. []
  7. Namely: EB, EVS, ESS, EES, CEVIPOF and TriÉlec 2012 surveys, CNCDH Racism Barometer, Insee Participation Survey, Ined TeO survey, and the 1967 French National Election Survey by Philip E. Converse and Roy Pierce. One might consider adding the ISSP and the Insee EDP (census extract) to the list. []