Adding Diversity to the Hugo Awards

[daveon]

One of the claims the various puppies have made is they're getting more works into consideration for the Hugo Awards...  I thought this was an interesting claim, especially given the effectiveness of a strong slate, so I thought I'd look at the data.  I can only find details back to 2011, anybody who has links to the earlier nomination data, I'd quite like it.

So 2011 is a clearer pre-puppy year, back in the bad old days when either middle-aged priviliged white men, or a sekrit cabal of Social Justice Warriors ruled the Hugos - it was also a smaller con, being in Australia.

As our pre-puppy, cSWM/SJW dominated reference we had 833 Ballots Cast for Best Novel, with 23 works up for consideration.

Each year, since then, the number of ballots cast has increased - interestingly, the number of books nominated has not, in fact, in each of the following years just 16 works have been nominated... meaning that the ratio of ballots to works has gone from 36:1 in 2011, to 100:1 last year...  strike 1 for the puppies.

I next looked at the ratio of Best Novel nominations to nominations for the Short Story categories - interestingly, by and large they're relatively static - more or less half the people who nominate a book for Best Novel nominate a short.

But here's another thing, in the last Post Puppy year we had the following numbers:

Novelette: 24
Novella: 22
Short Story: 17

Let's look at the state of play post puppy?

Novelette: 17-16-15
Novella: 15-15-16
Short Story: 18-17-15

Apart from Short Story which hasn't changed, the number of individual works has actually fallen too.

One of the original puppy claims was that works were not getting onto the Hugo Ballot, this is certainly true, but the implication of the claim was they were not being nominated - the data doesn't support that.  They were being nominated, they just didn't have enough people doing so.

The drive to get more people to nominate Puppy books has actually reduced the diversity and range of options being nominated, NOT, increased it.  Which rather dismisses the idea that the puppies do not and have no voted the slate as a bloc and read their own things and made their own decisions - if they have been doing that, then actually they've been therefore reading pretty much the same stuff as the cSWM/SJW 'cabal'.

I'd be interested to look at the data going back, but on the face of it, the drive to get more people to nominate and increase numbers has actually had the reverse effect and that strong slates actually make things even worse.

EDIT 1: A puppy supporter has pointed out to me that 2012 was also pre-puppies too - I'd need somebody to check if that was around the time a lot of Book Bloggers were getting upset about cSWM domination of the event.  Either way, it's interesting that despite an increase in the numbers of nominations, the numbers of works being nominated hasn't changed a lot with or without the Puppies.  I would like pre-2011 data so we can see what the normalized ratios were pre Aussiecon.

EDIT 2: The data I'm seeing seems to cut off at about 3% of votes cast - so I don't actually have full nominations here, but the numbers making that cut off seem very very consistent year in and year out.  Having more works spread out doesn't indicate much of a conspiracy.  Just people tend to like the same things.

EDIT 3: It's really hard to pull out the numbers of unique works because they're not always given or not given in a format I can be arsed to pull out.  However, interestingly a couple of things emerge:

1. The ratio of ballots to unique works in Best Novel is constant at about 2 - you can increase the voters, they all seem to behave the same way - the number of works that make the 3% of the votes cast list that is conventionally shown dropped in 2011 and has stayed the same


2. The Short Story categories have changed - if the ratios of votes to unqiue works had stayed constant then they'd have been 300+ unique works in one of the categories last year, instead there were only 200

In short, the more people we have taking part, the easier the numbers suggest it becomes to actually log roll a category with a strong slate.  I wasn't expecting that.

There is also precious little evidence that there has really been any real concerted activity in the past to game the awards - but we knew that.

Comments

Unclear

[(Anonymous)]

There are bunch of things that confuse me in this post, but the 2011 Worldcon was in Reno. It was the 2010 Worldcon that was in Australia.

You can find the voting statistics for 2010 at http://web.archive.org/web/20100917163329/http://aussiecon4.org.au/hugoawards/files/2010HugoVotingReport.pdf. (The Aussiecon 4 web site has been taken down, but it was archived.)

The WSFS Constitution (section 3.11.4) requires a Worldcon to publish nomination statistics "including in each category the vote counts for at least the fifteen highest vote-getters and any other candidate receiving a number of votes equal to at least five percent (5%) of the nomination ballots cast in that category, but not including any candidate receiving fewer than five votes." So the number of works that get listed as runners-up for nominations in the Hugo statistics will vary slightly from year to year, but will also depend on how much work the Hugo administrators want to do in publishing the list.

For example, there were 1,827 nomination ballots submitted for Best Novel this year. If the administrators wanted to publish only the nomination count for the top 15 novels and any other novel receiving at least 5% (that is, at least 91 nomination votes), they could do that. Or they could go deeper down the list, and publish the nomination counts for all novels receiving at least 0.3% of the nomination votes (that is, at least 5 nomination votes). Or they could go somewhere in between.

But, in any event, they have already revealed that 587 different novels were listed on those 1,827 ballots. (See https://chaoshorizon.wordpress.com/2015/04/06/how-many-puppy-votes-breaking-down-the-hugo-math/.) Many of them were probably listed by only one to four people and thus will not be listed in the final statistics in any event.

In summary, I'm not sure that the data you are using supports the conclusion you are trying to draw from it.

--J. Kreitzer

Re: Unclear

[daveon.livejournal.com]

See the latest edits - I have gone back and looked for the unique nominated works in total for each year, not easy because it's not always been tabulated. The numbers for Best Novel seem to remain constant, the Shorts don't.