The validity of plus/minus has been debated frequently here at Raptors Republic with several misconceptions around both “basic/raw” and “adjusted” measures.

The NBA posts raw +/- scores here. A few days ago, it was noted the DeMar DeRozan had the worst plus/minus in the league – what does this mean?. And how is it that Andrea Bargnani can have 23 points with a -21 rating one night and 25 points and +29 the very next game?!

So, what is this measure and why is it (or not) important?
“Raw”, “Basic” or “Simple” plus/minus is what you see in the NBA box scores.

It’s simply the difference between points scored (+) as a team versus points allowed (-) while that player is on the floor.

Why is it good?
In theory, it measures the player’s total impact on the floor. While the focus for many looking a the box score is “how many points did player x get?”, this metric attempts to capture his ultimate influence on the outcome of the game – did player x leave the game with the team scoring more than the opponent?

What are the issues with it?
The primary one is it doesn’t account for the “environment”. Does the player usually play with good or poor teammates? Does he usually play against the opponent’s starters? Bench? or Garbage time?

To illustrate:
Carlos Arroyo has a +/- of +144. Deron Williams? “Only” +44
So is Arroyo a better player? Let’s look at his “environment”

Arroyo has played ~600 minutes with ALL of Wade, Bosh, and James on the floor with him.
Williams has played the most minutes (779) with the “big three” of Kirilenko, Jefferson, and Millsap.

It’s quite clear that Arroyo’s +/- is driven by simply being on the floor with three of the best in the league.

How are the Raptors players doing?

Given our record (and our 24th ranked efficiency differential), it’s no surprise the majority of our players are in negative territory. Only Joey Dorsey and Julian Wright are in positive territory. Given they have limited minutes under their belts and they have often played against the opponent’s second units, its hard to draw any conclusions. It’s also not a surprise that Bargnani, DeRozan and Evans have the worst +/- given they’ve played the most starting minutes – thus against the opponent’s best lineups.

What can we conclude about “raw” +/-? Not to rely on it. Game by game and even over the course of the season, it’s hazardous to draw firm conclusions from “good” or “bad” +/- scores.

A better metric – “adjusted” plus/minus


Several people know I am a big fan of this metric. It’s foundations go back to Dan T. Rosenbaum’s work in 2004. Aaron Barzilai runs a site that I use very often – basketballvalue.com – which updates these values on a nightly basis.

Here’s why I’m a fan: Mr. Rosebaum developed adjusted +/- to attempt to address these questions:

Good players lead their teams to wins. Lots of players can fill up a stat sheet, but only the great ones are difference-makers. The objective of a basketball game is not to accumulate points or rebounds or assists, but to win. What statistic do you have for that?

So what is it? How is it measured?

Adjusted +/- ratings indicate how many additional points are contributed to a team’s scoring margin by a given player in comparison to the league-average player over the span of a typical game (100 offensive and defensive possessions). – Aaron Barzilai, Ph.D. and Steve Ilardi, Ph.D.

To put in simple terms, it adjusts the raw/basic +/- to factor in the quality of players that a given player plays with and against.

Let’s go back to our previous example between Carlos Arroyo and Deron Williams. What does the Adjusted +/- statistic say?

Does this seem more consistent with their abilities? It appears so.

How much should we rely on it to rate players?

It is important to note that the adjusted +/- rating is not a “holy grail” statistic that perfectly captures each player’s overall value. – Barzilai and Ilardi.

The authors properly warn about large standard errors unless a large sample size is used – noting that even a full season is often inadequate (we most often focus on the two year metric for this reason). It also suffers from skewed sampling: “it can be difficult to accurately tease out the individual effects of two players who almost always appear on the court together.”

Certainly there are anomalies due to these issues, but don’t most measures have problems as well? Should we rank players simply by PPG, ignoring the efficiency of which they get those points as well as their impact on defense?

Why do I like it? It’s a great tool to screen players and start “filling in the blanks”. For example, last year Amir Johnson screened well. He’s still at the top of all Raptors for 2 year adjusted +/-. If nothing else, you start asking questions: “does he really impact the game in a positive way?” “Well, he doesn’t score much, but he’s efficient.” “Hmm, he’s one of the top offensive rebounders per minute (second chance points!)”. “He fouls a ton, but some of that is due to help defense – no easy baskets.” “He runs the floor well creating easier opportunities for himself and teammates.” “His hustle may just inspire teammates to play a little harder on D”. Etc.. You don’t have to agree, but these metrics get us thinking on what might explain the data and if there’s something to it or not.

This is also why I was pushing for Julian Wright early on. His 2 year adjusted +/- is generally 1st or 2nd along with Johnson. And by watching the games, you could see – despite some offensive limitations – he was making a difference: hunting down loose balls, drawing charges, sticking to his man, making nice passes, etc. What’s happened when he plays more? One stat: in the three games that he’s played more than 21 minutes he’s averaged approximately 7 pts, 6 rbs, and a +14 rating. ALL were wins on the road (two against good teams). Bundle this data together, watch some tape and we should conclude that its worth playing him more to either: confirm if our analysis is correct or this is all just a fluke. It’s worth a test.

Free Ju Ju (0:55 mark).

Email me: [email protected] or find me on Twitter.

  • Statement

    Nice article Tom,

    I’m a fan of adjusted +/- as well. Berri however, rips it to shreds on his website. Berri’s Wins Produced metric is quite nice as well, though I like to look at all of them (PER, Off/Def Ratings, 82games on/off, adjusted +/-, Wins Produced) to fully get a handle on a player.

    • http://twitter.com/Liston Tom Liston

      Great point. Good idea to look at many metrics – see what’s consistent or tells another side to the story.

      And Berri shouldn’t throw stones. He ranked DeMarcus Cousins and Evan Turner much higher than John Wall.
      http://dberri.wordpress.com/2010/05/18/is-john-wall-the-obvious-number-one-pick-for-the-wizards/

      • Statement

        I hear you Tom,

        I will say that in subsquent articles, he has indicated that PAWS isn’t the greatest predictor, but does better than other potential predictors.

        Given the relative lack of explantory power in PAWS, it’s reasonable that he would be more inaccurate using it.

        • http://twitter.com/Liston Tom Liston

          That’s true. Only meant to imply that no metric is the “holy grail” and Berri isn’t perfect either.

  • A_socrates

    Does anyone keep stats along the lines of points for a player vs. points for the player they are gaurding?

    • http://twitter.com/Liston Tom Liston

      Yes, at 82games.com. Has Net PER as well.
      Amir: http://www.82games.com/1011/10TOR11.HTM#bypos
      Andrea: http://www.82games.com/1011/10TOR15.HTM#bypos
      etc…

      • http://twitter.com/Liston Tom Liston

        82games.com just changed the link for Andrea, use this:
        http://www.82games.com/1011/10TOR16.HTM#bypos
        (The above now links to Joey Dorsey)

      • tom

        The thing about net PER — can’t it be kind of deceptive for someone like Andrea who guards both PF’s and C’s? So in a hypothetical example if Dwight Howard is shredding up Amir while Bargs is locking down Brandon Bass, does Bargs appear to take the hit on Opp PER?

        If my understanding is correct, it’s probably more relevant for someone like Jose, who plays and guards PG pretty much 99% of the time.

        • http://twitter.com/Liston Tom Liston

          In your example it would for sure. But how often does Andrea guard the opponent’s PF rather than C? I don’t recall it being that often since our PFs are a bit undersized to begin with and thus I don’t see them guarding C that often.

          • tom

            Fair enough — I was just thinking of a few cases where he didn’t start out guarding the opposing C (Gasol, Amare, Howard, Nene) but there probably aren’t too many more cases like that. Then again, those are 4 of the top 5 Centres we’ve faced so far this year in terms of PER (Horford being the other).

            Just to be clear though, not trying to defend Bargs’ opp PER (it appears to be dreadful any way you slice it). Just saying it’s probably more reliable for some players than others.

            • tom

              btw, I *think* Bargs was guarding Odom not Gasol in the fist Laker game, but I could be wrong on that…

  • Nilanka15

    Thanks for clearing up all the misconceptions. Nicely done!

  • http://twitter.com/EC_Rapsfan D. Gillis

    I think nothing beats watching the game, if you can, and then asking yourself if that player played well or not, regardless of what the Box or advanced metrics says. Personally, I’d be hesitant to make any claims about a player i haven’t watched play many times, regardless of what metrics say.

    Seems to be always people looking for a new and better mousetrap, where, if the current tools were all that valuable there wouldn’t be a need. Also all of these metrics fail to account for synergy and fit. Why can Beasley play well on the Wolves, but not on the Heat? It’s that kind of stuff that goes un-noticed for two long. Not saying this it you Tom, but I’m remind of a saying about how people often use stats like a drunk uses a lightpost, for support rather than enlightenment.

    Math gives you absolutes, so people erroneously think basketball stats prove something. But they don’t. They just help to provide context.

    • Nilanka15

      What we can conclude is that no matter how hard we try, or how passionately we feel about a certain topic, at the end of the day, it is nothing more than opinion.

      We can site in-game references, anecdotes, and basic/advanced statistics until we’re ready to pull our hair out, but no two people will see the same sequence of events unfold the same way. That’s what makes discussions on this site so interesting.

    • Statement

      East Coast Raps Fan,

      I agree that statistics are only useful as a tool and not the be-all-end-all of decision making. However, it is a documented position in psychology that the human mind can process only 7 different events at a given time. In basketball there are so many moving parts that it is actually humanly impossible to make an accurate record of all events. Hence, the need for statistics.

      Optimally, I would want my front office to have good scouts/coaches to watch the players, but not completely ignore the advanced statistics and incorporate them into their decision making.

      Take a measure such as True Shooting Percentage. Clearly, this is an improvement over raw field goal percentage as it takes into account 3-pt attempts, 2-pt attempts, and free-throw attempts in it’s calculation. To me, this is a very useful stat and should be looked at as the benchmark when valuing the shooting efficiency of a player.

      When advancements like this happen, they should not be ignored and should be incorporated into the front-office decision making process.

      All in all, watching the games is good, but to ignore/not understand the statistics provided by the game is folly.

    • http://twitter.com/Liston Tom Liston

      Absolutely correct.

      But adv stats *may* remind you of what you may have forgotten. Or “urge” you to re-watch the tape for certain sequences. Or, in fact, find line-up synergies are often uncovered by the stats geek (check MIT Sloan Sports Conf keynote panel for discussion on this) that the coaches didn’t think of.
      See here: http://basketballvalue.com/teamunits.php?year=2010-2011&sortnumber=17&sortorder=DESC&team=TOR
      or here: http://www.82games.com/1011/1011TOR2.HTM

      Not you (b/c I know you) but many only remember (or see on SportsCentre) the highlights – almost exclusively offensive production. And they’ll believe things like Hedo Turkoglu is a clutch shooter when he’s not. (It’s b/c there are no YouTube videos showing the 15 clutch shots he misses – only the 3 big makes. Because he has the ball all the time in these situations he’s bound to make some…. but no one recalls all the misses).

      If you ran the stats, you’d see that he shot 29.2% from three in clutch situations and only 38.6% overall last year. Nowitzki is clutch. Turkoglu, not so much. Yet 90% of fans would call him a clutch player. But WHAT ABOUT TURKOGLU IN ORLANDO 2 years ago? Even worse: 20% from three, and 35.7%. Thesis: jack ’em up a lot in clutch situations as even if you only make a couple, you’ll get the label and a fat contract. Not convinced – look at his playoff overall clutch stats that big year: http://www.82games.com/0809/playoffs/08ORL9.HTM#clutch

      Ugly.

      These are a few reasons why its very useful to use adv stats as A TOOL – it helps you ask the right questions and look for the right answers.

      Just like Moneyball. GMs and Scouts usually overweight athleticism, the long ball (NBA: high volume scorers, etc.) This is why the Spurs draft so well, for example.

      Finally, and this is the my clincher: I have yet to watch a game where a fellow fan can actively watch all 10 players on the floor at the same time. Often an away-from-the-ball play can mean the difference for a score. Watching the games often don’t give you the whole picture unless you watch hours of tape afterwards.

      • Statement

        Tom,

        See my comment above about the theory (reasonable as of now) that humans can only process 7 different visual events at a given time.

        It appears that you and I are like-minded, maybe you could hook me up with an equity analyst job too.

        • http://twitter.com/Liston Tom Liston

          Yes – sorry, meant to follow up to credit you – only saw your response after I posted mine… and I “stole” your theme. Well said.

          • Statement

            No problem,

    • yertu damkule

      taking your example of beasley a bit further, advanced metrics help to not only determine why he’s playing well with the wolves, but also why that isn’t (or hasn’t) translating to team success.

  • cesco

    Tom , could you explain the following discrepancy between the +/- stats of basketballvalue.com (adjusted) and NBA.com (net +/- total) .
    basketballvalue adj +/- (1 year) : Reggie +5.88 , Andrea -5.25
    NBA.com net total this year : Reggie -105 , Andrea -125

    These +/- stats seem like comparing apples to bananas , the figures for Andrea make sense but for Evans , they don’t make sense . Another crappy advanced stat in my opinion.

    • Theswirsky

      how is that like comparing 2 different things? Because the results aren’t what you’d like to see?

      • cesco

        I asked a question to Tom , he may or may not have an explanation . I didn’t ask for stupid remark from a Bargnani hater .

        • Theswirsky

          I’m just asking how it “seems like comparing apples to bananas”?

          • cesco

            On one stat (NBA.com) , both players look bad , on the other stat (basketball value) one player look bad , the other look good . Both stats deal with +/- of players yet they seem to be contradictory . I just wanted to know if there was an explanation to that.

            • http://twitter.com/Liston Tom Liston

              The two stats can look contradictory. For example, if you or I played with Bosh, Wade and James all the time, we may have a positive +/- !! However, if we “adjust” for the fact we’re playing with superstars, it would show we have a negative impact.

    • http://twitter.com/Liston Tom Liston

      Yes. NBA.com is the cumulative total of +/- over all games to date. It does NOT take into account the quality of players Bargnani or Evans is playing with nor against.

      Basketball value *attempts* to “normalize” these numbers to essentially what these players would impact the score over 100 possessions (roughly a normal NBA game) basically assuming he’s playing with 4 average players against 5 average players from the other team.

      • cesco

        Thanks for the explanation . Still Reggie played most of the time with Andrea and after he was injured , Andrea played most of the time with Amir who has a very high ‘normalized’ +/- . I still see something fishy in that normalized +/- stat.

        • Statement

          cesco,

          I understand your point, the adjusted +/- has high standard errors so it is required to look over an extended time period (i.e. more than 1 year). 1 year encapulates Bargs and Reggies time together. So lets look at the adjusted +/- of greater than one year

          So.,…Bargs 2-year adjusted +/- = -5.59 (this would incorporate this years and last years data)

          So… Bargs 2-year adjusted +/- (incorporating data from 2009 – 2010) = -6.32

          Lets see what Bargs 2-year adjusted +/- was for the year before (2008-2009) = -6.29

          So basically, if you develop a moving average over time by this measure, bargs has been shit by this measure since at least 2008

          But this is only one measure

          Lets look at Wins Produced (intelletually different than adjusted +/- and a competitor to this type of analysis)

          Andrea Bargnani this year = -1.5 Wins Produced

          Andrea Bargnani last year = -2.7 Wins Produced

          Andrea Bargnani year before last = -2.4 Wins Produced

          Remember – these measures aren’t from “Bargnani Haters” these are normal attempts to quantify the value of players. Wins Produced doesn’t care who Bargnani or Lebron are, it just tries to separate the good (Lebron) from the garbage (Bargnani)

          • cesco

            I think you missed completely what I was saying . I did not question Andrea stat but Reggie should be in the minus also since they played mostly together , Reggie did the REBOUNDING and some scoring , Andrea the SCORING and some rebounding. This advanced stat is garbage (and you are a beautiful Andrea hater).

            • Statement

              Beautiful…..really…..Thanks. Seriously though cesco, I don’t mind debating you on Bargs because you don’t resort to namecalling, good on you sir.

            • Theswirsky

              “since they played mostly together ”

              but not always together… and hence your difference.

        • http://twitter.com/Liston Tom Liston

          Reggie and Amir’s D and rebounding are reasonably similar, but the difference is Amir in transition and just overall on offense. Amir is much better at the pick-and-roll game, for example. Amir also has quite a few more blocks (and alter shots, not measured) than Reggie.

  • Theswirsky

    “doesn’t account for the “environment”.”

    I agree thats a problem. Compare one team to another.

    But what about within a team, the environs. will be much more similar.

  • Danielmba2003

    I have a very hard time putting too much faith in these stats. Just watch the games and enjoy. It doesn’t take a Phd to understand who a good player or bad player is or who had a good game or a bad game. The stats are just tools to help the team.

    I don’t think by looking at any stats you know the true value of a player. You have to watch the games. These stats aren’t inelastic and don’t necessarily follow a player from one team to the next. For instance, look at Hedo. His number, like adjusted +/- or any other advanced stat like PER or whatever are all over the place from team to team. I looked up other players too, and when players move teams, the advanced stats change, sometimes dramatically.

    The stats, adjusted +/- or whatever, are situational and are a product of the environment in which the player plays. They are not a true measure of a player. While these stats can be useful, it is the interpretation and purpose they are used for that leaves much to be desired.

    Leave these stats for the intended audience, the coaches and team management, and leave the enjoyment of the game to us, the fans.

    • http://twitter.com/Liston Tom Liston

      That’s why I categorize these posts under “stats”, so its clear. Some fans are quite interested in looking at different metrics to confirm what they believe (or find some unique insight), some simply just love to roll with the flow of the game and make judgments.

      Different strokes…

      • raptordan

        There have been several articles on the this site under the guise of stat analysis that are used as a tool to bash Bargnani. Kudos to you for this article being educational and insightful and not yet another way to say that Bargs is bad on defense.

        • yertu damkule

          so, no articles that break down games can reference stats? or just these really tough to understand stats? what about ppg & rebounds & assists…are those ok?

          sigh. an mba just ain’t what it used to be…

          • Statement

            try reading Fooled By Randomness.

            a (rich) Wharton MBA basically spends most of the book discrediting the degree.

            Phd in engineering all the way!!

            • http://twitter.com/Liston Tom Liston

              Taleb is brilliant.

  • Ajar

    DeRozan’s awful raw +/- is obviously due to him being a sophomore player who is only a mediocre defender, but is the starting SG on a similarly mediocre team. He gets to guard the Ray Allens and Kobe Bryants while trying to score predominantly in the paint against the other team’s starting lineup (i.e. their best bigs, if they aren’t out on the perimeter guarding Bargs).

  • Dookielover

    f the stats. just watch the game. emar erozan.

  • yertu damkule

    it’s too bad there isn’t a way to statistically quantify how bargs’ supporters feel about advanced metrics versus how bargs’ haters feel about them.

    i’m thinking it would be fairly complicated, kinda like this:

    bargs fan: advanced metrics are stupid and are only used by haters to bag on bargs.
    bargs hater: advanced metrics are a useful tool proving that bargs sucks a bag of hammers.

    reality, as always, lies somewhere between…

    • Statement

      On point

      Which side of the fence do you lay on? You can’t say “somwhere between….”

      • yertu damkule

        i absolutely can say ‘somewhere between.’

        i lie on the side of ‘bargnani would have the most positive impact on team success if he were used as a 30 minute-per-game offensive sparkplug off the bench’ (think 7-foot jason terry). i think part of why his advanced metrics look as they do is because he’s playing against, for the most part, the opposition’s best frontcourt (i.e. being defended by & having to defend the opposition’s ‘best’ players). i believe his skillset is best suited to coming off the bench.

        because i know that that will NEVER happen (as long as both he & BC are in TO), i feel like i’m in the middle of my fabricated debate (between the haters & the fanatics). i see his skill, but i also see the shortcomings.

  • sangaman

    +-1 is a team stat as Tom pointed out. So may on this blog use this stat incorrectly to bludgeon bargs and others. The same players with a different coach can get better results. A needed defensive metric would be player to player for and against- in direct match-ups. And as never tire of saying physical play is hidden in stats….

    • yertu damkule

      i think you’re missing the point (somewhat)…adjusted +/- essentially is an attempt to account for the quality of one’s teammates & quantify their individual impact (i.e. all things being equal, does player X help or hinder their team’s chance of success). putting a player on a different team will likely adjust their raw numbers, but should not alter (too much) their adjusted +/-.

      • cesco

        But Reggie played mostly with Andrea against the other team starters yet he has an adjusted/normalized +5.88 and Andrea -4.08(not -5.25 like I said above) . I agree with Andrea stat which support the net total from NBA.com (both in the minus) . I mean Reggie did the REBOUNDING , Andrea did the SCORING , why such a discrepancy on the ‘adjusted’ stats , it does not make any sense at all . That is a barbage stat .

        • yertu damkule

          i’m speaking in generalities, and you’re looking at a unique & far too small sample size (i.e. the pairing of reggie & bargs) in isolation. remember, the purpose of ‘adjusting’ is to account for the quality of all one’s teammates with whom they share time on the court.

          • cesco

            I agree that they spent some playing time with different team mates but there should not be such a discrepancy if the majority of time they are playing together . Without the NBA.com stat you would think that Andrea was the biggest culprit of the two for the team losses but the NBA.com stat show the opposite (see the per minute negative +/- which favor Andrea) .

            • yertu damkule

              i guess i just see it differently. i didn’t want to get into specifics, because it would just come across as more bargnani-bashing.

              that being said, i think that them playing together would actually be why there is such a disparity. to wit: the one thing that reggie does well while on the court, he does EXCEPTIONALLY well, to the degree that it offsets his subpar contributions in other aspects (offense, and to a lesser extent, defense), and thus contributes to an adjusted +/- that indicates that his net impact, when the quality of his teammates is taken into consideration, is an overall positive.

              bargnani’s offensive contributions (the ‘one’ thing he does well) are certainly nothing to scoff at, but his offensive impact on the game is not as great as reggie’s rebounding impact (i.e. to oversimplify, reggie is a better rebounder than andrea is a scorer). further, because andrea is not an elite scorer, the other areas of the game in which he is sub-par lessen his offensive impact, to the point that – over a large sample size – his overall contribution to team success, when the quality of his teammates is factored in, is a net negative.

              no one designed these stats with the intention of trying to find new & creative ways to slag bargnani, and there are of course flaws in it (and any statistical measure if not placed in proper context). but as we look into each & every advanced metric, there is a consistent pattern, and that is that very few (if any) show him (bargs) in much of a positive light.

              do the ‘haters’ use these as *proof* of his suckitude irresponsibly? probably. but is doing so any more irresponsible than dismissing all this data because it doesn’t line up with what we want to believe?

    • http://twitter.com/Liston Tom Liston

      There are, and the data doesn’t look good for Bargnani in this case.
      http://www.82games.com/1011/10TOR16.HTM#bypos
      Net negative PER.