Page 1 of 2 1 2 LastLast
Results 1 to 20 of 24

Thread: Stats: Why does no one look at variance?

  1. #1
    Raptors Republic Starter
    Join Date
    Nov 2009
    Location
    Etobicoke
    Posts
    983
    Post Thanks / Like

    Default Stats: Why does no one look at variance?

    I always see people looking at stats as averages, or extrapolated averages (e.g. Per 36 minutes). How come no one ever looks at the variance of those stats?

    For example, the standard deviation for FG% or rebounds would tell us a lot about consistency and, therefore, reliability. If one player shoots 40% one night and 50% another, whereas another player shoots 45% both nights, who would you rather have?

    I'm not sure if the advanced metrics take this into account, but I haven't read that they do. I guess this is more a question for Blake, but just wondering, is all...

  2. #2
    Raptors Republic All-Star slaw's Avatar
    Join Date
    Jul 2010
    Location
    Calgary
    Posts
    2,096
    Post Thanks / Like

    Default

    Couple of things:

    - I think your underlying premise that consistency isn't evaluated or considered is wrong.
    - Single games are too small sample sizes to draw conclusions from (just look at the Bulls/Nets and Thunder/Rockets series). There are dozens of variables that could explain a poor performance.
    - It would only mean much if the variance was big and consistent. My guess is that few players fit this bill. And the ones that do, well, we already evaluate them correctly.
    - It all gets factored in to the aggregate number anyway, so, poor performances are reflected in aggregate numbers.

    The best example of this on the Raps is Derozan. People talk about how he has improved various parts of his game. This is true. But if you look at him as a whole player, and his production levels, the various tweaks and improvements have made little to no difference in overall productivity or efficiency. So, despite the fact he had some big games and seemd to be a better player, he has the same impact on games he had last year and the year before.

  3. #3
    Raptors Republic Starter
    Join Date
    Nov 2009
    Location
    Etobicoke
    Posts
    983
    Post Thanks / Like

    Default

    Quote slaw wrote: View Post
    The best example of this on the Raps is Derozan. People talk about how he has improved various parts of his game. This is true. But if you look at him as a whole player, and his production levels, the various tweaks and improvements have made little to no difference in overall productivity or efficiency. So, despite the fact he had some big games and seemd to be a better player, he has the same impact on games he had last year and the year before.
    The new 100 words on Demar post is actually what sparked this thought, "DeRozanís shooting percentage were nothing short of a rollercoaster" (Ryan McNeil). You don't necessarily need to look at variance on a game-to-game basis. You can look at DeRozan's shooting percentages each month of the season and compare them to someone similar.

    I'm just very critical of aggregate numbers. If one player gave you exactly 18 points per game, every single game, then everything is easy to model. But even though aggregate numbers incorporate poor performances, they don't tell us how often those poor performances are, or how often those amazing performances are. Again:

    - Player A averages 18 ppg, with a range of 12-20
    - Player B averages 18 ppg, with a range of 4-36

    Despite their aggregate averages being similar, they are very different players!

  4. #4
    Super Moderator CalgaryRapsFan's Avatar
    Join Date
    Apr 2010
    Posts
    4,329
    Post Thanks / Like

    Default

    Quote Marz wrote: View Post
    - Player A averages 18 ppg, with a range of 12-20
    - Player B averages 18 ppg, with a range of 4-36

    Despite their aggregate averages being similar, they are very different players!
    Even just doing that would have flaws, because no player scores points in a vacuum. There are many factors to consider; some directly related to the player, some indirectly related to a player and some unrelated to the player.

    For instance, if there's a game where a player is being guarded by an elite lockdown defender and some of his teammates have very favourable matchups, should the player be 'penalized' if the coach draws up plays to exploit those advantages and the player follows the plan by deferring to his teammates? Like most stats, just looking at them on the surface can be very misleading and doesn't explain the "why" behind them.

    For a starter who plays significant minutes for most/all games throughout a season, I think the law of averages will account for the 'positive' and 'negative' games related to a specific stat. Although DeRozan had some games that made him look like a budding star, his averages really show that he actually made very little improvement over last season. Also, for any player, you can easily get one impression from 'results' type stats, but an entirely different impression when efficiency is factored into the analysis.


    DeRozan: 2012-2013 (2011-2012) Stat Comparisons

    PPG: 18.1 (up from 16.7), but needed 15.0 shots per game (up from 14.3)

    3PT: 28.3% (up from 26.1%), but identical 0.4 makes / 1.5 attempts per game (both this season and last season)

    FT: 83.1% (up from 81.0%), but nearly identical 4.3 makes / 5.2 attempts per game (had 5.3 attempts per game last season, so he actually got to the line less this season)

    His peripheral stats this season were all similar to last season, but definitely improving:
    RPG: 3.9 (3.3 last season)
    APG: 2.5 (2.0 last season)
    STL: 0.9 (0.8 last season)
    BLK: 0.3 (0.3 last season)
    TO: 1.8 (2.0 last season)

    All of that was a result of playing more MPG this season (36.7, up from 35.0 last season) and being a bigger focal point in the team's offensive gameplan.


    These are just the raw stats. It's up to each individual fan to interpret/analyse them however they see fit, compare them to their own subjective 'eye test' and come to their own conclusion about DeRozan (or any other player).
    Last edited by CalgaryRapsFan; Thu Apr 25th, 2013 at 12:28 PM.

  5. #5
    Raptors Republic Rookie
    Join Date
    Dec 2012
    Posts
    15
    Post Thanks / Like

    Default

    Why are you guys so quick to shut this idea down? I think it's a fantastic idea. How 'bout it, Blake? Would be grateful for an analysis!

    The counter-arguments didn't convince me. If a player defers then it will affect their FGA, but you could look at the variance of TS%, and I think it would say a lot when comparing scorers. Statisticians already find big discrepancies with home and away performance, so why not look at variance in general.

  6. #6
    Super Moderator CalgaryRapsFan's Avatar
    Join Date
    Apr 2010
    Posts
    4,329
    Post Thanks / Like

    Default

    Quote red monkey wrote: View Post
    Why are you guys so quick to shut this idea down? I think it's a fantastic idea. How 'bout it, Blake? Would be grateful for an analysis!

    The counter-arguments didn't convince me. If a player defers then it will affect their FGA, but you could look at the variance of TS%, and I think it would say a lot when comparing scorers. Statisticians already find big discrepancies with home and away performance, so why not look at variance in general.
    I certainly think it is worth investigating, as a type of advanced stat.

  7. #7
    Raptors Republic All-Star Primer's Avatar
    Join Date
    May 2012
    Posts
    1,189
    Post Thanks / Like

    Default

    Quote red monkey wrote: View Post
    Why are you guys so quick to shut this idea down? I think it's a fantastic idea. How 'bout it, Blake? Would be grateful for an analysis!

    The counter-arguments didn't convince me. If a player defers then it will affect their FGA, but you could look at the variance of TS%, and I think it would say a lot when comparing scorers. Statisticians already find big discrepancies with home and away performance, so why not look at variance in general.
    I agree. As someone who doesn statistical modeling for a living, this analysis would be very meaningful and useful. For example, I model shipments of goods to home depot stores nationwide. The amount of goods demanded varies greatly throughout the year. If I were to just look at the aggregate as say, we sell 50 bags per store per week on average, so lets send every store 50 bags a week. This would be horrible, as there are parts of the years where stores need 200 bags, and parts where they need none. It's much better to drill down and look at everything on a weekly basis. This doesn't directly correlate to basketball, but let me try to explain.

    For easiness, I'll use Pts/game, but any stat could work here, and best would probably be a graph of Pts/Game and TS% so you could see that relationship as well. If you were to graph a player by Pts/game, with each game being a data point, then you could set a threshold of acceptable number of points, say 15, and then count the number of games above and below the threshold. This would be important, because if DD is alternating 25pt games with 5pt games, then we clearly have a greater probability of winning the 25pt games, and a greater probability of losing the 5pt games. Whereas if he were averaging 15pts a game every game, he'd be giving us the same probability of winning each night. To avoid the pitfalls of small sample size, you look at an entire season and calculate winning percentages for above and below the threshold. This could really magnify the value of having a consistent 15pt scorer, versus one who alternates 25pt and 5pt games. You could use TS% to do the same analysis, and look for links between the two. It may turn out a player who averages 15pts every single night is far more valuable than a player who averages 18pts over the season, but has a large amount of games scoring below 10pts.

    I'd love to see this analysis done. It wouldn't be perfect at first, but a little time in excel expermienting with different graphs and thresholds could provide some interesting results on how valuable consistency is versus random huge games. A player who goes off for 30 against the Bobcats but then goes for 5 against the Celtics is probably less valuable than a player who scores 15 against both of them.

  8. #8
    Raptors Republic Starter
    Join Date
    Jun 2010
    Posts
    959
    Post Thanks / Like

    Default

    Quote Marz wrote: View Post
    The new 100 words on Demar post is actually what sparked this thought, "DeRozanís shooting percentage were nothing short of a rollercoaster" (Ryan McNeil). You don't necessarily need to look at variance on a game-to-game basis. You can look at DeRozan's shooting percentages each month of the season and compare them to someone similar.

    I'm just very critical of aggregate numbers. If one player gave you exactly 18 points per game, every single game, then everything is easy to model. But even though aggregate numbers incorporate poor performances, they don't tell us how often those poor performances are, or how often those amazing performances are. Again:

    - Player A averages 18 ppg, with a range of 12-20
    - Player B averages 18 ppg, with a range of 4-36

    Despite their aggregate averages being similar, they are very different players!
    Yes, but even that doesn't really tell much of the real story. Let's say I told you that there was player who averaged a little over 10 points, with a range of 0 to 35 as a range. Let's call this guy player A. Now, let's take another guy (player B), who averages 10 points, with a range of 0 to 24 as a range. Sounds like sort of similar players, right? But just based on those numbers, do you have a picture of who this guy is? Is he a guy who scores about nine points most games, and then had just one crazy night that totally throws his range out of whack? Or is he a guy who fluctuates wildly from night to night?

    So let's define average games as average +/- 20% (I'm just throwing a number out here). Now, I'll break down our mystery player A's game: 38% of his games were above average, 46% were below average, and 15% were average. So now we're starting to see that he's the sort of player who fluctuates wildly.
    Mystery player B's game breaks down this way: 32% of his games are above average, 30% are average, and 38% of his games are below average. His performance is much more consistent, and he actually produces a greater percentage of average or better games, despite having a lower upper-end to his range.
    By breaking down games into these levels (and ideally you could use more than just 3 levels... 5 might be ideal), and perhaps using a more comprehensive stat, like the gamescores available on basketball-reference.com (I know, I know, it's a Hollinger stat), you'd get a more complete picture of how often these players win you games, how often they lose you games, and how often they turn in a consistent performance.
    If you're curious, player A is Alan Anderson, player B is Amir Johnson.

    edit: yeah, I like the idea Primer's suggestions here too, particularly trying to incorporate the impact of highs and lows and consistent scores in terms of actual wins.
    Last edited by octothorp; Thu Apr 25th, 2013 at 03:18 PM.

  9. #9
    Raptors Republic Starter
    Join Date
    Nov 2009
    Location
    Etobicoke
    Posts
    983
    Post Thanks / Like

    Default

    Quote octothorp wrote: View Post
    Yes, but even that doesn't really tell much of the real story. Let's say I told you that there was player who averaged a little over 10 points, with a range of 0 to 35 as a range. Let's call this guy player A. Now, let's take another guy (player B), who averages 10 points, with a range of 0 to 24 as a range. Sounds like sort of similar players, right? But just based on those numbers, do you have a picture of who this guy is? Is he a guy who scores about nine points most games, and then had just one crazy night that totally throws his range out of whack? Or is he a guy who fluctuates wildly from night to night?

    So let's define average games as average +/- 20% (I'm just throwing a number out here). Now, I'll break down our mystery player A's game: 38% of his games were above average, 46% were below average, and 15% were average. So now we're starting to see that he's the sort of player who fluctuates wildly.
    Mystery player B's game breaks down this way: 32% of his games are above average, 30% are average, and 38% of his games are below average. His performance is much more consistent, and he actually produces a greater percentage of average or better games, despite having a lower upper-end to his range.
    By breaking down games into these levels (and ideally you could use more than just 3 levels... 5 might be ideal), and perhaps using a more comprehensive stat, like the gamescores available on basketball-reference.com (I know, I know, it's a Hollinger stat), you'd get a more complete picture of how often these players win you games, how often they lose you games, and how often they turn in a consistent performance.
    If you're curious, player A is Alan Anderson, player B is Amir Johnson.

    edit: yeah, I like the idea Primer's suggestions here too, particularly trying to incorporate the impact of highs and lows and consistent scores in terms of actual wins.
    Your levels are essentially looking at the "variation" from the average. Not in the strict mathematical sense, but you've basically helped describe my point: looking at how a stat fluctuates is so much more meaningful than a single number.

    I, too, work in modelling, but also in generating numbers synthetically that correspond to those models. And whenever I use an aggregate number, without taking into account variation, my generated numbers always have lower fidelity. Hence the idea.

    @Primer I like those ideas. How do we get Blake to do the leg work again? :P

  10. #10
    Raptors Republic All-Star Soft Euro's Avatar
    Join Date
    Jun 2011
    Location
    Utrecht, The Netherlands (Yes, that's Europe!)
    Posts
    1,920
    Post Thanks / Like

    Default

    A problem is that there are so many reasons for variance undependent of a players consistency.

    For example, if it's a big who's getting a lot of his points from PnR situations, his points will be influence by the way the opposing team defends the PnR. The same holds true, of course, for the ball handler.

    It's also highly depended on the role a player has on the team, just as it's depended on the game plan. If there are favorable matchups on the floor, a coach will try to exploit those. That doesn't mean anything for the consisticy of that player and some coaches try to make more use of this than others.

    If we take DeRozan as an example, his variance is also related to the way he is being defended (and the quality of that defender). In fact, I think that's one of the most important factors in his case. Does that mean that this makes him inconsistent? No, the explanation for that part of his variance stems from the variance in defenders.

    And I think there are countless other examples to show other influences besides consistency for a stat like this.

    All in all, I think there are too many other factors besides a player's consistency causing noise in a stat like this.

  11. #11
    Raptors Republic Starter
    Join Date
    May 2012
    Posts
    848
    Post Thanks / Like

    Default

    Quote Soft Euro wrote: View Post
    For example, if it's a big who's getting a lot of his points from PnR situations, his points will be influence by the way the opposing team defends the PnR. The same holds true, of course, for the ball handler.

    It's also highly depended on the role a player has on the team, just as it's depended on the game plan. If there are favorable matchups on the floor, a coach will try to exploit those. That doesn't mean anything for the consisticy of that player and some coaches try to make more use of this than others.
    The simple solution to this is compare 'variance' as per positions. For example, compare a PF to a PF, a SF to a SF and so on. Even though 'variance' provides a more accurate indication of a players consistency, it's OBVIOUSLY not been considered important enough to be used. Nothing to prove this but in this day and age, it's highly unlikely that the idea has not been explored.
    Attitude Is A Choice.

  12. #12
    Raptors Republic All-Star Soft Euro's Avatar
    Join Date
    Jun 2011
    Location
    Utrecht, The Netherlands (Yes, that's Europe!)
    Posts
    1,920
    Post Thanks / Like

    Default

    Quote Eric Akshinthala wrote: View Post
    The simple solution to this is compare 'variance' as per positions. For example, compare a PF to a PF, a SF to a SF and so on. Even though 'variance' provides a more accurate indication of a players consistency, it's OBVIOUSLY not been considered important enough to be used. Nothing to prove this but in this day and age, it's highly unlikely that the idea has not been explored.
    That only solves a small part of the problems. I could add tons of other examples about context influencing a players production, especially if we focus on points.

    What do you mean by a "more accurate indication". More accurate than what? And why would it be accurate at all? Too much noise.

  13. #13
    Raptors Republic Starter
    Join Date
    May 2012
    Posts
    848
    Post Thanks / Like

    Default

    Quote Soft Euro wrote: View Post
    What do you mean by a "more accurate indication". More accurate than what? And why would it be accurate at all? Too much noise.
    While other factors are considered, the most common stats that determine a players offensive consistency are PPG and FG%. As Mars who started this thread states as example, if player A shoots 40% one night and 50% another as opposed to player B who shoots 45% on both nights, player B is more consistent, This can be determined when 'variance' is considered. Hence a more accurate indication of consistency.
    Attitude Is A Choice.

  14. #14
    Raptors Republic All-Star Primer's Avatar
    Join Date
    May 2012
    Posts
    1,189
    Post Thanks / Like

    Default

    Another interesting stat to look at would be on/off +/-. If there is a large variation in that from night to night, it shows how that players inconsistency is affecting the team as a whole. Even for someone like DD, who currently has a higher off court +/- than on court, the game by game analysis would at least show when he is doing worse and better, not necessarily good and bad since you could argue its always bad when the +/- favors you being off the court. If DD has lots of nights with a +6 on court, followed by lots of nights with a -5 on court, then you know his inconsistency is causing the team to lose games, and they'd be better with a player who consistently gets +1 On court every night, or even +0 On court every night, as those -5 On court games probably equal losses. Maybe every player is wildly inconsistent in those stats, I'm not sure. That's why the analysis would be cool to see.

  15. #15
    Raptors Republic All-Star Soft Euro's Avatar
    Join Date
    Jun 2011
    Location
    Utrecht, The Netherlands (Yes, that's Europe!)
    Posts
    1,920
    Post Thanks / Like

    Default

    Quote Eric Akshinthala wrote: View Post
    While other factors are considered, the most common stats that determine a players offensive consistency are PPG and FG%. As Mars who started this thread states as example, if player A shoots 40% one night and 50% another as opposed to player B who shoots 45% on both nights, player B is more consistent, This can be determined when 'variance' is considered. Hence a more accurate indication of consistency.
    Ok, I get it, but I don't particularly agree that you'll find out who's more consistent that way. It would be nice if someone could give an example of just a comparison between two players.

  16. #16
    Raptors Republic Starter
    Join Date
    Nov 2009
    Location
    Etobicoke
    Posts
    983
    Post Thanks / Like

    Default

    I'm not saying using variance is the be-all end-all indicator of consistency. I was just interested as to why it's never considered when discussing stats, at least from what I've read. In statistics, an average with a standard deviation of 3 vs. one with a standard deviation of 20 is different. I'd assume this would be important information to consider when looking at aggregate stats of a player.

    Whether or not variance tells us a player is consistent or not is still an open question, but I believe it would be a good metric to look at.

  17. #17
    Raptors Republic Starter
    Join Date
    Jun 2010
    Posts
    351
    Post Thanks / Like

    Default

    I agree that looking at variance is an interesting concept, and consistent with statistical principles used in many other fields.

    Alan Anderson seems like a "high variance" kind of guy.

  18. #18
    Raptors Republic Starter
    Join Date
    Nov 2009
    Location
    Etobicoke
    Posts
    983
    Post Thanks / Like

    Default

    I'm not very good with Excel, and I don't have much time, but here's some raw data. All from basketball-reference game logs for the 2012-2013 season. Feel free to draw your own conclusions.

    Code:
    DeMar DeRozan
    
    Metric	FG	FGA	FG%	3P	3PA	3P%	FT	FTA	FT%	ORB	DRB	TRB	AST	STL	BLK	TOV	PF	PTS
    
    Average	6.68	15.01	0.44	0.41	1.46	0.22	4.33	5.21	0.81	0.59	3.32	3.90	2.49	0.93	0.29	1.84	2.11	18.11
    
    STDEV	2.76	4.47	0.12	0.75	1.31	0.31	3.14	3.41	0.20	0.73	1.70	1.94	2.09	0.97	0.59	1.32	1.19	6.87
    Code:
    Alan Anderson
    
    Metric	FG	FGA	FG%	3P	3PA	3P%	FT	FTA	FT%	ORB	DRB	TRB	AST	STL	BLK	TOV	PF	PTS
    
    Average	3.63	9.48	0.34	1.46	4.38	0.27	1.94	2.26	0.85	0.52	1.75	2.28	1.58	0.74	0.11	1.23	2.02	10.66
    
    STDEV	2.77	4.57	0.19	1.60	2.76	0.26	2.10	2.34	0.22	0.73	1.38	1.61	1.32	0.93	0.31	1.13	1.54	7.85
    Code:
    Amir Johnson
    
    Metric	FG	FGA	FG%	3P	3PA	3P%	FT	FTA	FT%	ORB	DRB	TRB	AST	STL	BLK	TOV	PF	PTS
    
    Average	4.15	7.48	0.54	0.06	0.16	0.40	1.68	2.31	0.73	2.80	4.74	7.54	1.51	1.00	1.36	1.44	3.72	10.04
    
    STDEV	2.62	3.90	0.22	0.24	0.46	0.44	1.58	1.95	0.30	2.05	2.62	4.03	1.32	0.97	1.29	1.21	1.42	5.99
    Not sure how to include an attachment on this forum, but I can share the full excel file (it has other stats and information in it).

    EDIT: Bleh, don't know how to disable word-wrap...
    Last edited by Marz; Fri Apr 26th, 2013 at 08:23 AM.

  19. #19
    Raptors Republic Starter
    Join Date
    May 2012
    Posts
    848
    Post Thanks / Like

    Default

    Quote Soft Euro wrote: View Post
    Ok, I get it, but I don't particularly agree that you'll find out who's more consistent that way. It would be nice if someone could give an example of just a comparison between two players.
    Here's an example. I'm using Demar Derozan and Rudy Gay as examples. I'm comparing their points over a 5 game stretch. I hope it helps.

    DEMAR:

    GAME PTS. SHOOTING FG%

    1 20 8/17 47%
    2 16 7/16 44%
    3 15 6/13 46%
    4 20 8/16 50%
    5 12 5/15 33%


    RUDY:

    1 30 12/30 39%
    2 22 8/22 36%
    3 20 8/14 57%
    4 25 11/28 39%
    5 30 12/22 54.5%


    Demar averages 16.6 PPG at a FG% of 44%.

    Rudy averages 25.4 PPG at a FG% of 44%.

    Even though Rudy has more PPG and their overall FG% are identical, Demar is the more consistent scorer based on his game to game FG%. Rudy had excellent shooting in two games but poor shooting in three whereas Demar had one poor game but four good games which makes him the more consistent scorer.
    Last edited by Eric Akshinthala; Fri Apr 26th, 2013 at 10:00 AM.
    Attitude Is A Choice.

  20. #20
    Super Moderator thead's Avatar
    Join Date
    Jan 2012
    Location
    Toronto
    Posts
    3,018
    Post Thanks / Like

    Default

    the more I read into this the more I think you are on to something.

    to put this in the simplest of terms. Is it better to have a guy that averages 15 pts a game but does it by scoring 10 one game and 20 the next at 45% clip. Or a guy that scores 14 - 16 a game at a 45% clip. One is consistent and one can shoot you either in to or out of a game

Page 1 of 2 1 2 LastLast

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •