41 coaches online • Server time: 19:17
Forum Chat
Log in
Recent Forum Topics goto Post Creating a custom to...goto Post Secret League Americ...goto Post DOTP Season 4
SearchSearch 
Post new topic   Reply to topic
View previous topic Log in to check your private messages View next topic
mrt1212



Joined: Feb 26, 2013

Post   Posted: May 24, 2015 - 20:38 Reply with quote Back to top

So I was thinking about one of the other threads and the outsized impact that a few select coaches have on the aggregate win percentages for various teams.

Examples are Smallman with his Pact smallkosp, BillBrasky with his Chaos WMDs in the Box, DukeTyrion with his Nurgle Slime Barons and CameronHawkins with his Hanson's Roughriders (Lizardmen) where their overall records are far better than the box population's winning % with respective races.

The question I have is whether you discount the large impact that one or two coaches have on aggregate winning % or if you approach it as fully relevant since most those coaches play a significant amount of games currently (CH withstanding).

My feeling on the matter is that throwing out unqualified winning %s when there are players who account for a large portion of any team's games does a disservice to the discussion. I mean, I can't be the only who is curious as to the impact that long lived teams in the hands of coaches with tons of experience does to the overall picture.

Are there statistical methods we can use to determine the impact that individual members of the population exude on the aggregate? I don't know much about statistical methods but I'd like to learn more.

A secondary question is whether its beneficial to ever exclude data based on how stale it might be? I'd hate to fit the data to reach an opinion but I feel like when we're looking at data from 2013, it's not really relevant to the current state of the box or what we might expect. Does this kind of thing depend on the size of the data and what you're trying to achieve (in this case an idea of what you could expect to see in the next month)?

Help me understand the world better armchair statisticians
koadah



Joined: Mar 30, 2005

Post   Posted: May 24, 2015 - 20:57 Reply with quote Back to top

If you are worried about win% then maybe.

If you worried about "too many..." then too many is too many whoever is playing them Twisted Evil

_________________
Image
O[L]C 2016 Swiss! - April ---- All Stars - Anniversary Bowl - Teams of Stars - 13th March
Harad



Joined: May 11, 2014

Post   Posted: May 24, 2015 - 21:12 Reply with quote Back to top

So it depends on the question you are trying to answer. For example, if you want to know how good the average (mode, mean, median) coach is in the box with a certain race then you may choose to ignore them depending on how imbalancing their effect is. Alternatively, if you are trying to answer, how likely am I to lose a game against a certain race in the box then you should include their numbers as they represent the population of people you could play against in the ratio you are likely to play against them.

So all the statistics are valid, it just depends on what you are intterogating them for.
Wizfall



Joined: Dec 09, 2011

Post   Posted: May 24, 2015 - 21:29 Reply with quote Back to top

Very few impact like 1%, maybe 2% at the very best (i'm sceptic though), at least for popular race.
Take lizards for example, they are in the top 2/3 both here and on Cyanides in a B environment.
More or less the same win % for everyone here and on Cyanide too.
(the last time i checked a long time ago).
CanvasBack



Joined: Jan 15, 2007

Post   Posted: May 24, 2015 - 21:39 Reply with quote Back to top

You always have to look at the context when using statistics. A first generation team in Box will not have faced the same challenges while developing that a new starting team will.

You can compare a coach's performance with any team and compare it to the aggregate performance of the rest of the population using ordinary statistic's methods. Just remember to subtract the individual coach's stats out first.

_________________
"When you have shot and killed a man you have in some measure clarified your attitude toward him. You have given a definite answer to a definite problem. For better or worse you have acted decisively. In a way, the next move is up to him." R.A. Lafferty
Grod



Joined: Sep 30, 2003

Post   Posted: May 24, 2015 - 22:33 Reply with quote Back to top

Tricky. If you take out a few top coaches you should take out a few at the bottom as well. A relevant stat might be the average win rate of every coach (who has played at least 1 fame) with that race. This means a coach who has played 1 game counts the same as a coach that has played 1000 (quite different from current stat). It tells you the coach mean skill level with a race rather than the race mean skill level.

_________________
I am so clever that sometimes I don't understand a single word of what I am saying.

Oscar Wilde
Medon



Joined: Jan 28, 2015

Post   Posted: May 24, 2015 - 22:50 Reply with quote Back to top

A possible idea: Don't calculate win% of all matches, but the win% of all coaches. So take the win% of every coach of that specific race, and then take the average (or median) of that. In that way all coaches have the same weight factor it doesn't matter if you've played 1000 games or 100 games. The result is the expected win% of the 'average coach' if he plays with that race. Data selection on e.g. 'minimum 10 games played' or 'minimum TV 1300' could be made to calculate the expected win% of the average coach who played more than 10 games and had a TV of more than 1300. At least it would be interesting for me, since I qualify for the 'average coach'Smile Don't know if it would be interesting for you Legends out there...
Wreckage



Joined: Aug 15, 2004

Post   Posted: May 29, 2015 - 18:28 Reply with quote Back to top

Medon wrote:
A possible idea: Don't calculate win% of all matches, but the win% of all coaches. So take the win% of every coach of that specific race, and then take the average (or median) of that.


Man, you are so right.
This is how it should be done. Well spotted.
mrt1212



Joined: Feb 26, 2013

Post   Posted: May 29, 2015 - 18:36 Reply with quote Back to top

Wreckage wrote:
Medon wrote:
A possible idea: Don't calculate win% of all matches, but the win% of all coaches. So take the win% of every coach of that specific race, and then take the average (or median) of that.


Man, you are so right.
This is how it should be done. Well spotted.


Time to tap on Koadah? Laughing

This reminds me in a way of how Football Outsiders does their "DVOA" stats:

http://www.footballoutsiders.com/info/methods

It's all about where you are relative to league average based on situation.
WolfyDan



Joined: Aug 02, 2003

Post   Posted: May 30, 2015 - 00:44 Reply with quote Back to top

For the 1st question, I'd say to not even bother. There are so many issues with undertaking even simple stats work on something as varied as this that your results will almost certainly come down to technique selection. Even a relatively small number of games can have a big impact for certain types of analysis. You couldn't even split it into TV groups because that would confuse matters even more as the TV system is quite terrible.

For the 2nd question, the trick here is to not presuppose anything. Best bet would be to GLM the data and then add time as a random factor to check for consistency. If it holds then keep the data in and look for inflationary trends to add over the top. Otherwise drop.


Medon wrote:
Don't calculate win% of all matches, but the win% of all coaches.

Sadly this is unlikely to work due to the high correlation you are introducing. Almost every decent stat technique would get confused as to what is driving the win rate.
jimimothybodles



Joined: Mar 31, 2004

Post   Posted: May 30, 2015 - 01:42 Reply with quote Back to top

Plus you can't take an average of averages. Nonsensical. On the original question posed, averages in data are easily skewed by outliers, hence should be removed.

Its about actual data vs normalised data. The first reveals the past, the second (through statistical modelling) allows future predictions.

_________________
"Give a man a match, and he'll be warm for a day.
Set a man on fire, and he'll be warm for the rest of his life."
~Ancient Snotling Proverb
strikereternal



Joined: Oct 02, 2009

Post   Posted: May 30, 2015 - 04:34 Reply with quote Back to top

mrt1212 wrote:

Time to tap on Koadah? Laughing

This reminds me in a way of how Football Outsiders does their "DVOA" stats:

http://www.footballoutsiders.com/info/methods

It's all about where you are relative to league average based on situation.


We could take a page from baseball/sabermetrics and calculate WARC (Wins Above Replacement Coach) Very Happy

In reply to the original post: have we already established what impact those coaches have on the overall win percentages for their races?
licker



Joined: Jul 10, 2009

Post   Posted: May 30, 2015 - 04:52 Reply with quote Back to top

mrt1212 wrote:
So I was thinking ....


Usually dangerous.

mrt1212 wrote:
Examples are Smallman with his Pact smallkosp, BillBrasky with his Chaos WMDs in the Box, DukeTyrion with his Nurgle Slime Barons and CameronHawkins with his Hanson's Roughriders (Lizardmen) where their overall records are far better than the box population's winning % with respective races.


The coaches (and or the builds) are also better.

mrt1212 wrote:
The question I have is whether you discount the large impact that one or two coaches have on aggregate winning % or if you approach it as fully relevant since most those coaches play a significant amount of games currently (CH withstanding).


What % of total games by those races do any of those coaches actually have though?

mrt1212 wrote:
My feeling on the matter is that throwing out unqualified winning %s when there are players who account for a large portion of any team's games does a disservice to the discussion. I mean, I can't be the only who is curious as to the impact that long lived teams in the hands of coaches with tons of experience does to the overall picture.


No, it's entirely possible that you are the only one who cares.

mrt1212 wrote:
Are there statistical methods we can use to determine the impact that individual members of the population exude on the aggregate? I don't know much about statistical methods but I'd like to learn more.


Yes there are.

mrt1212 wrote:
A secondary question is whether its beneficial to ever exclude data based on how stale it might be? I'd hate to fit the data to reach an opinion but I feel like when we're looking at data from 2013, it's not really relevant to the current state of the box or what we might expect. Does this kind of thing depend on the size of the data and what you're trying to achieve (in this case an idea of what you could expect to see in the next month)?


You can of course use a running average, but what is the point? In general I wouldn't exclude any data unless there was a change in the conditions you are measuring. Thus you would probably accept all data from when the MM change was introduced, and could exclude all data prior, though the rules were the same, the ability to min/max and pick on rookie teams has gone away (or at least been reduced).

mrt1212 wrote:
Help me understand the world better armchair statisticians


I'm not sure what you're really trying to understand. What the impact of a coach with a 70% win rate in 1000 games has on a population of 25000 games that have a win rate of 55%? I mean... sure, you could run the numbers, but isn't it obvious what the impact is generally?
zakatan



Joined: May 17, 2008

Post   Posted: May 30, 2015 - 07:05 Reply with quote Back to top

jimimothybodles wrote:
Plus you can't take an average of averages. Nonsensical. On the original question posed, averages in data are easily skewed by outliers, hence should be removed.

Its about actual data vs normalised data. The first reveals the past, the second (through statistical modelling) allows future predictions.


Except when outliers are a significant chunk of the data. If you remove 1% of the data, you're avoiding the skew. If you remove 10% you are cooking...

_________________
Image
WolfyDan



Joined: Aug 02, 2003

Post   Posted: May 31, 2015 - 14:07 Reply with quote Back to top

Ok, so I know I said that this was a bit of a waste of time, but it got me thinking. And although I still maintain it is a massive pain in the butt, that is never a reason by itself to not do something. So if I were to do something like this, here's how I'd go about it.


1. Factor selection
The first thing that you would need to do is come up with a list of anything you can imagine that might influence the result of a game. And I think that there would literally be thousands of them. You'd have things about the teams, the players, the dice rolls, anything. You'd also have to look at combinations. So a highlight list of 10 things might be:

1. Number of +ST players
2. Number of +ST players more than your opponent
3. Race of team being played
4. Race of opponent team
5. Rank of player overall
6. Rank of player for that team
7. Number of games played by that player with that team
8. Number of times Blitz rolled by that team in the game
9. Number of 1's rolled
10. Total number of block skill in team
etc.


You would need help from the community I am sure. The key at this stage is to try and be too wide with your selection. Don't listen to people that say they have anecdotal evidence or opinions that something doesn't have an effect. If we're going to do this then we should do it scientifically and collect evidence. But as there is no way to do that in advance for the factor selection then we'll have to use the wisdom of masses to come up with a list.


2. Data Prep
You would need a database of all team-games played. This means every game will have 2 entries, one for each team. You'd need a unique reference (I'd say the game number with H or A suffixed), and the result. You wouldn't need the score but that would be interesting too. Everything else after that are the above factors.

I've no idea what the website's DB format is like, or how easy it might be to append the factors to the data set. Also there might be some historic issues (knowing how many games someone has played now is a lot easier than knowing how many they played at the time that game was played).

This is the toughest but most important step. Crap data means crap results. This could take half a year to build, and would need a lot of help from the community I expect.


3. Information Value
1000+ factors are way too many to sensibly deal with, so the first stage is to get a rough estimate of which factors are more predictive by correlating them to the win rate. Random forest is a better method but would take too long.

There's a process called Information Value that can help to assess this. This applies a score to each factor - the higher it is the more correlated it is to predicting a result.

http://documentation.statsoft.com/STATISTICAHelp.aspx?path=WeightofEvidence/WeightofEvidenceWoEIntroductoryOverview


4. Group and Trim
The next stage is to overcome the combinations of the above factors. Some factors will be heavily correlated, and this will lead to them getting very similar IVs. For example 'total number of +ST' is likely to be highly correlated to 'total number of +ST more than opponent'. So as they are often the same thing it makes little sense to include both.

Therefore you'd need to group your factors and then take the highest IV of each group. This ensures that you have a good range of different effects that you can interact. After that you should also trim anything with a poor IV to ensure only important factors are being considered. Ideally at this stage you'd want no more than 100 factors left.


5. Generalised Linear Modelling
http://en.wikipedia.org/wiki/Generalized_linear_model

There are plenty of down sides to this technique, but the big upside is that you can strip out the effect of a factor in the rest of the data. This allows you to see where the true reasons for winning a game lies.

So you might find that the 'total number of games played' is predictive, and by adding that to the model it strips the effect out of every other factor. This means that anything left over (residual) must be explained by other things. Any factor that has the effect reduced to 0 has been knocked out, and you know that what you thought was a good influence is actually explained by other things.

Again random forest is the best measure, but realistically it would take too long. So really you'd need a few good analysts doing an exploration and feeding back results. Once a concensus is reached then you can build the final model.

You'd also need to add in time as a random factor to ensure that anything you see is consistent and not just a fluke.


6. TV calc
The great thing about this method is that you can begin to calculate the true value of the team. For example you might find that +ST has the same increase in Win% as playing a team with 38TV more than you. This would allow you to rebalance the +ST TV to +38 rather than +50. With enough data and enough factors considered you could even personalise this to specific players.

You do this by taking your final model and seeing what the over the top effect of these factors are (stripping out any correlated factors). A pain in the butt, but certainly achievable. From this you could completely redesign the TV system. Skills like Pass Block might only add 2 or 3 TV, whilst Block might be +24 unless the player already has dodge in which case it is +28. ClawPOMB might be +300 Twisted Evil


So yeah, all you need is the data, some GLM software, and a couple of experienced modellers and I think you could do this in about 12 months.
Display posts from previous:     
 Jump to:   
All times are GMT + 1 Hour
Post new topic   Reply to topic
View previous topic Log in to check your private messages View next topic