Adding Diversity to the Hugo Awards

One of the claims the various puppies have made is they're getting more works into consideration for the Hugo Awards...  I thought this was an interesting claim, especially given the effectiveness of a strong slate, so I thought I'd look at the data.  I can only find details back to 2011, anybody who has links to the earlier nomination data, I'd quite like it.

So 2011 is a clearer pre-puppy year, back in the bad old days when either middle-aged priviliged white men, or a sekrit cabal of Social Justice Warriors ruled the Hugos - it was also a smaller con, being in Australia.

As our pre-puppy, cSWM/SJW dominated reference we had 833 Ballots Cast for Best Novel, with 23 works up for consideration.

Each year, since then, the number of ballots cast has increased - interestingly, the number of books nominated has not, in fact, in each of the following years just 16 works have been nominated... meaning that the ratio of ballots to works has gone from 36:1 in 2011, to 100:1 last year...  strike 1 for the puppies.

I next looked at the ratio of Best Novel nominations to nominations for the Short Story categories - interestingly, by and large they're relatively static - more or less half the people who nominate a book for Best Novel nominate a short.

But here's another thing, in the last Post Puppy year we had the following numbers:

Novelette: 24
Novella: 22
Short Story: 17

Let's look at the state of play post puppy?

Novelette: 17-16-15
Novella: 15-15-16
Short Story: 18-17-15

Apart from Short Story which hasn't changed, the number of individual works has actually fallen too.

One of the original puppy claims was that works were not getting onto the Hugo Ballot, this is certainly true, but the implication of the claim was they were not being nominated - the data doesn't support that.  They were being nominated, they just didn't have enough people doing so.

The drive to get more people to nominate Puppy books has actually reduced the diversity and range of options being nominated, NOT, increased it.  Which rather dismisses the idea that the puppies do not and have no voted the slate as a bloc and read their own things and made their own decisions - if they have been doing that, then actually they've been therefore reading pretty much the same stuff as the cSWM/SJW 'cabal'.

I'd be interested to look at the data going back, but on the face of it, the drive to get more people to nominate and increase numbers has actually had the reverse effect and that strong slates actually make things even worse.

EDIT 1: A puppy supporter has pointed out to me that 2012 was also pre-puppies too - I'd need somebody to check if that was around the time a lot of Book Bloggers were getting upset about cSWM domination of the event.  Either way, it's interesting that despite an increase in the numbers of nominations, the numbers of works being nominated hasn't changed a lot with or without the Puppies.  I would like pre-2011 data so we can see what the normalized ratios were pre Aussiecon.

EDIT 2: The data I'm seeing seems to cut off at about 3% of votes cast - so I don't actually have full nominations here, but the numbers making that cut off seem very very consistent year in and year out.  Having more works spread out doesn't indicate much of a conspiracy.  Just people tend to like the same things.

EDIT 3: It's really hard to pull out the numbers of unique works because they're not always given or not given in a format I can be arsed to pull out.  However, interestingly a couple of things emerge:

1. The ratio of ballots to unique works in Best Novel is constant at about 2 - you can increase the voters, they all seem to behave the same way - the number of works that make the 3% of the votes cast list that is conventionally shown dropped in 2011 and has stayed the same


2. The Short Story categories have changed - if the ratios of votes to unqiue works had stayed constant then they'd have been 300+ unique works in one of the categories last year, instead there were only 200

In short, the more people we have taking part, the easier the numbers suggest it becomes to actually log roll a category with a strong slate.  I wasn't expecting that.

There is also precious little evidence that there has really been any real concerted activity in the past to game the awards - but we knew that.

My Hugo Numbers - a bit tidier

Firstly, my original methodology was a bit wonky, I was working off the published nominations that got more than about 3% of the votes cast.  Unfortunately, not very many years have published details on the number of Unique Works nominated compared to the Ballots.  I have 3 relatively easy to look at ones though.  2009, 2013 and 2014.

Tables don't seem to be easy here, so I'll do what I can:

Total Ballots cast Best Novel | Novelette | Novella | Short Story
2009 - 629 | 373 | 337 | 448
2013 - 1113 |616 | 587 | 662
2014 - 1595 | 728 | 847 | 865

Unique Works Nominated Best Novel | Novelette | Novella | Short Story
2009 - 335 | 233 | 122 | 470
2013 - 475 | 252 | 135 | 569
2014 - 648 | 290 | 206 | 578

I've already covered the works garnering about 3% of the total votes and what happened to them.

Most interesting for me: the ratio of unique works nominated to the number of ballots cast for Best Novel has stayed REALLY very consistent over this whole time.

The actual ratio of people who vote for Best Novel and then vote for a short category is also relatively consistent.  What is not consistent is effect on the works being nominated - relatively speaking fewer and fewer short stories are being nominated over time.

What is unclear to me is how many unique works are actually published in each category and how much of this is simply diminishing returns in a field that already was fairly widely read.

Either way, given these results and how often in recent years some of the short categories have struggled to get enough stories which hit 5% it's indicative of just how effective a strong 'slate' is and how little evidence there really is of much of one historically.