I need your help for the Best Anime of 2012 votes!

Topic started by sickVisionz on June 16, 2012. Last post by AURON570 2 years, 8 months ago.
Post by sickVisionz (4,327 posts) See mini bio Level 24

As mentioned in another thread, I'm getting some preliminary work done for the Best of 2012 voting that's going to happen in a few months. The vote counting system is being done via Google Spreadsheets and Forms and at this point it's pretty much an automated process and is near completion. However, there is one thing I wanted to get community feedback on:

Weight System

Here's the way I currently have it setup:

  • Last year I got the impression that some were nominating shows but with an attitude of, "well I only saw 5 shows. I thought one was really good and the rest were meh, but there are five slots so I have to fill them all up with something." This year I want to stress that only "Top 5 Quality Content" should make a list and if you can only think of X number of shows that deserve mentioning, only nominate X number of shows.
  • Tying in with that, a one (fifth place) is almost a nothing vote compared to what first place would be, but fifth place isn't and shouldn't be a nothing quality series. I wanted some type of balance where a show that gets a small number of first place votes can't totally monster over a series that has significantly more votes in other ranks, especially when every show nominated should be a beloved series by the voter. First should be powerful, but not dominating... unless there a series gets a lot of them.
  • The higher the lowest score, the more wiggle room there is to make separation between the other ranks while still keeping the ratio of first to last the same.
  • Having said that, first place is first place and while I don't want it to completely obliterate everything else, I did want for it to be a powerful vote. That's why there is separation between first and second and it's why it's 3x the weight of fifth place.

How it all plays out

Having said all of that, this is how it all plays out at the end of the day. I ran the new weights through last years numbers.

Chart Legend

  • Series = the name of the series. They are listed in the order that they won in the 2011 system (Madoka at the top = 1st place with everything beneath it being one position lower in rank). This ranking also correlates with the "2011 Wgt" category.
  • 1sts - 5ths = the total number of votes given in those ranks. Example, Fate/Zero have 5 under 1sts means that it got 5 first place votes.
  • Votes = the total number of votes a series received. Example, Tiger & Bunny has a Total of 5 (1 second place + 2 third places + 2 fourth places = 5 total votes).
  • Wgt Total = weighted total. This is the weighted score using the current system (24,15,10,7,5). It's calculated by multiplying each rank x votes and summing it. Example, Persona 4 has 2 second place votes (2 votes x second place weighted value of 15 = 30) + 2 fifth place votes (2 votes x fifth place value of 5 = 10) which add up to equal a weighted total of totaling to 40 (30+10)
  • 2011 = the weighted total using the 2011 weights (7,6,5,4,3). Example, Guilty Crown for 3 third place votes (3 votes x last year third place weight of 5 = 15) and 1 fifth place vote (1 x last year fifth place weight of 3) which total to 18 (15+3)


This is with the weighting listed at the top (24,15,10,7,5). Originally I used a different scale (9,6,5,4,3) and the same thoughts listed immediately underneath the "Weight System" header, but after running real votes through it, certain things stood out as being off base. Looking at last year's weighted totals also showed questionable rankings that I felt needed to be adjusted. Note: the following is purely based off of voting and not the quality of the series...

  • Madoka should monster everything else. It's tied for tied for most first places and it's probably the most well rounded when it comes to the other ranks. Last year's weights made things closer but I didn't feel that actually represented the difference that it should have over the competition.
  • Persona 4 should beat Guilty Crown. Not by a lot, but it got the votes in the spots that counted imo and the math should add up. AnoHana vs Hanasaku should remain a very close match up based off of the votes.
  • Tiger & Bunny vs Beelzebub should show both the power of a first place vote and how a handful of middling votes can neutralize it.
  • I felt that Wandering Son's placement in the prior year showed how in an effort to not make first place votes dominating, I hadn't made them powerful. Something about two first place votes and a third getting beaten by the rankings of the four series above it just didn't feel right.

But, that's just my views on it. I want to make the 2012 votes even better than before and I want and need to hear the communities thoughts on what is probably the most crucial and critical aspect of the whole event. So, what do you all of think of this?

Also, if you want to mess around with it and help me beta test things (I'd be super thankful for this) use these links: View the Titles and Results & Vote for Titles. None of the voting will be counted toward the real Best of 2012 (which will take place in December and January) so feel free to vote as many times as you want and however you want. This is mainly for people to see how the weights impact score and whether or not it seems fair or needs tweaking. It's also for me to see if there are any errors in the spreadsheet and test out things like making sure I get a notice if a user votes for the same series multiple times and other things.

Post by Wales (133 posts) See mini bio Level 21

I think it works out well and the system itself works too. Interface on voting is pretty straightforward and easy to get into as well as you explain it well on how people should vote given the circumstance. However, the one thing I would ask for a little clarification on is the weight system. this being my first time looking at it and giving my own two cents in the field, the area confuses me slightly. I'm sure any other user who is looking at this for the first time might need some clarity. Not a dumbed down version but short, sweet and to the point as we say in the education world.

Post by takashichea (15,985 posts) See mini bio Level 25
Online Now

I somewhat understand the numbers, but I have to read it again a couple times to get it. I get how you weight an anime series, but I just got lost when I look at the example. I love the Google Docs approach in voting versus how we used lists. We'll see how much an anime series rank in 1st place and below.

When do you want to spread the word and get people voting?

Post by sickVisionz (4,327 posts) See mini bio Level 24

@Wales: The weighting system and the calculation page are just behind the scenes stuff that I'm exposing so people can see how it impacts the rankings of a title and can do their own tests to see things like how many X votes equal a Y vote and does that seem fair/sensible. That page won't be visible in the real one that people use at the end of the year. All they will see is the big series list.

As far as the reasoning behind why it's as it is rather than something else (like First = 5 points, Fifth = 1 point), it's just a way of assigning values to the ranks where there is more flexibility and I go over it a bit in the OP of this blog. Last year's system was pretty rushed by me and in hindsight there are things that didn't seem right. I listed a few of them in the OP. This year I tried to come up with a weighting system that I could run the old data through and get results that seemed more in line with my own internal logic about how votes in X rank vs Y rank should play out and didn't leave me scratching my head certain results.

Post by sickVisionz (4,327 posts) See mini bio Level 24

@takashichea: This is just a beta version and all of the votes will be erased once the real voting period starts (from late December 2012 to early January 2013). Right now I just want as many folk testing the system so that we can all see how the voting works and examples of how the weighting affects things in real-time so that any oddities can be seen by all, suggestions given, etc can be given and discussed well before the real voting begins. Additionally, I want to see if there are any flat out errors in the system. As many people voting right, the better it will be for the real process.

I'll clarify the example in the OP.

Post by DocHaus (938 posts) See mini bio Level 18

Just voted all five slots for Gintama just to see how (if?) it handles any repeat votes (whether joking or intentional, it can still skew the results).

Also, wouldn't it just be easier to simply weigh the votes on a 1-5 point scale (5 points for 1st place, 4 for 2nd, etc) or is there a reason you went for more of a range?

Post by takashichea (15,985 posts) See mini bio Level 25
Online Now

I voted, too.

Post by AURON570 (573 posts) See mini bio Level 15

I always thought it was a 5,4,3,2,1 point system. Honestly I think I'd be fine with any weighted system, as long as all the data is summarized and made available at the end of voting.

Post by AURON570 (573 posts) See mini bio Level 15

I just voted. I'm not sure if you have considered this already, or if google docs have a function to deal with this, but people submitting english names and japanese names of the same anime. For example, Kids on the Slope is the english name of Sakamichi no Appolon.

Post by sickVisionz (4,327 posts) See mini bio Level 24

@AURON570: I tried to make it clear on the form that you should only use titles as written in the spreadsheet. I'll add a link/text to it on the form.

Post by AURON570 (573 posts) See mini bio Level 15

@sickVisionz: whoops, I totally didn't read those notes at the top of the form, didn't know you only wanted us to use the first 20 titles on the spreadsheet for this test. >.> Welllll, I guess this is what testing is for right? >.>

Mandatory Network

Submissions can take several hours to be approved.

Save ChangesCancel