[Note: Part 1, posted last night, gave us an intro to the simulation methodology]

by Marshall Flores

With that out of the way, let’s begin our simulation. “Hold on to your butts.” I’m going to divide the entire BP nomination process into three rounds, each round consisting of multiple steps.

Round 1: Initial Allocation, Victory Threshold, and the Surplus Rule

We begin the first round of voting by separating the 505 top 10 lists I’ve compiled into piles of films receiving #1 votes. It turns out that although 357 films received votes from this sample of lists, only 80 films received #1 votes. The top 20 leaders in terms of #1 votes are:

After we sort the ballots by their #1s, we then have to perform a few calculations. First, we’ll determine the initial victory threshold, the level of #1 support a film needs to get BP nominated off-the-bat. The victory threshold is approximately 9.1% of the total ballots cast (which is then rounded up to the nearest whole number), so 0.091 * 505 = 46 #1 votes.

Another number we need to determine is called the surplus threshold, which relates to a feature unique to IRV called the surplus rule. I consider the surplus rule an incentive for voters to consider both consensus and passion. The underlying idea is that if there is a strong degree of consensus on what are the #1 films, then other films that have support down the line on these ballots will benefit to a certain extent as well. In other words, the surplus rule is a potential reward for those voters who agree on a #1 film to a significant amount, by giving an additional degree of consideration to their 2nd or 3rd choices.

How does the surplus rule work? First, we have calculate the surplus threshold that a film needs to even qualify for the rule. This threshold is approximately 11% of the total ballots (and again, we round up), so 0.11 * 505 = 56 #1 votes. Note that any film meeting this level also meets the victory threshold, and as a result, *automatically* gets nominated for BP! As we can see in the table above, 12 Years and Slave and Gravity both meet the surplus threshold. So not only do we have two films with which the surplus rule applies, but we also have our first two BP nominees!

Next, and this is the wonkiest aspect of the surplus rule, we reallocate the votes of 12 Years a Slave and Gravity received to the #2 or #3 choices on those ballots at a reduced weight. This weight is calculated by determining how much each film surpassed the victory threshold, and then dividing these surplus #1 votes by the total amount #1 votes each film received.

Quite a mouthful, huh? Well let’s compute the vote weights in order to get a better idea. In the case of 12 Years, we have 69 #1 votes, which is 23 votes higher than the 46 vote victory threshold. As a result, the #2 choices on those ballots which listed 12 Years as #1 will receive 23 / 69 = 0.33 of a vote. Calculating along similar lines, the #2 choices on those ballots listing Gravity at #1 will obtain 19 / 65 = 0.29 of a vote.

Round 2: Allocating Surplus + Elimination of Films

We then start Round 2 by distributing the surplus votes. Our updated Top 20 below:

Although many films in our Top 20 benefited from the surplus rule, it’s not enough for any of them to meet the victory threshold. We now have to start eliminating films from contention. The elimination threshold is 1% of the total votes cast, or 0.01 * 505 = 6 votes. Films with vote totals below this level are removed from the BP equation and their votes will be redistributed to the next highest ranked film still in contention at full weight. This rule does give an incentive to voters in supporting any personal long shot favorites without wasting their ballots.

At this point, only 19 of 80 films are above the elimination threshold, so we eliminate 61 films from competition and attempt to transfer their votes. If a vote from an eliminated ballot cannot be reallocated, i.e. all the other ranked films on the ballot are no longer in contention, then the vote gets discarded and the total number of ballots in play is reduced. We end up discarding 12 ballots entirely, lowering our total vote count from 505 to 493.

Round 3: Allocating Eliminated Votes, 5% Nomination Threshold

We now enter the 3rd and final round of voting in our simulation. At this point, a film no longer needs to hit the 46 vote victory threshold to get nominated. Now, it just needs to garner at least 5% support of the ballots still in play, or 0.05 * 493 = 25 votes. Once the ballots eliminated in Round 2 are redistributed, any and all films receiving at least 25 votes *will* be BP nominated, while the rest are not. The nomination process then concludes.

Our updated and final results:

So our simulation results in an 8 film BP lineup!

12 Years a Slave
The Act of Killing
Before Midnight
Blue Is the Warmest Color
Gravity
Her
Inside Llewyn Davis
Leviathan

All 8 films were in that Top 20 table that was ranked using a weighted system, but it’s very clear that #1 votes are paramount in obtaining a nomination. For example, Leviathan, which received about 50 total mentions less than either Frances Ha or American Hustle, ended up being BP nominated due to having more #1 votes than Frances Ha and Hustle *combined*. Practically speaking, it seems that a film needs to have been initially listed as the first choice on at least 3.5-4% of all ballots in order to have a shot at getting nominated.

These results are in line with other voting simulations that have been conducted by Steve Pond and others. When I ran this simulation last year (with just 207 Top 10 lists), I also arrived at 8 nominees. I can only speculate on how AMPAS has been able to come up with 9 BP nominees the past couple of years, but my intuition has come up with a couple of possible reasons. The first is that AMPAS has around 6000 voters – a much larger population of votes than the 500 in this simulation and others. The second – since voters are *not* required to rank additional films beyond their #1 choice, I would guess that there’s a high number of discarded ballots in Round 2. Discarding ballots ends up *lowering* the 5% threshold a film needs to reach in Round 3 to get nominated.

In the end, we ended up with 8 BP nominees, despite there being many possibilities for BP this year due to a strong overall crop of films. I would suspect that many would be pleased with a lineup as diverse as this one. AMPAS probably won’t be as diverse or bold with its lineup, but hey, we can all dream, right? 🙂

An overall recap of the 3 rounds in our voting simulation can be found can be found here

This concludes this week’s very special (and long!) episode of Statsgasm. The process of nominating films for Oscar is a pretty involving and complex one, and arguments can certainly be made in favor of scrapping the current system and reverting back to a simpler one. But for now, instant-runoff voting is the name of the game.

As usual, feel free to ask any questions in the comments below, or contact me at marshall(dot)flores(at)gmail(dot)com and on Twitter at @IPreferPi314. Next week on Statsgasm we will return our focus to AD’s prediction models.

A very happy New Year to you all!