Does Coaching Stability Matter?

The stare that could drop a thousand panties will return in 2014-15. Does it matter?

For just the second time in the history of the franchise, the Toronto Raptors will enter the 2014-15 season with a coach in his fourth year at the helm. Like with the roster, the coach’s chair has been a carousel throughout the 19-year history of the Raptors, with few names ever succeeding long enough to put an indelible stamp on the team’s identity.

And while the words of general manager Masai Ujiri, which have preached culture and chemistry, don’t necessarily promise continuity, head coach Dwane Casey is “the guy” until he’s not. Given how inconsistent the results have been during his three-year tenure, it raises the question as to what we can reasonably expect from him moving forward.

Casey So Far
2011-12 – Coming off of a lockout with a preseason that lasted all of two games, the incoming Casey had the deck stacked heavily against him. A new coach, eight new players and a lack of preparation time – plus a retroactively-stated desire to tank for a better draft pick – led to a 23-43 record and the second-worst offense in the NBA. Still, this group managed to rank 14th in defensive efficiency, and there was optimism that Casey could at least scheme through a lack of talent on one end of the floor.

2012-13 – This time around, there were 10 new names on the team at any given point, and the team drastically shook things up with the acquisition of Rudy Gay. It was a transition year of sorts, and one that ended with some promise despite a 34-48 record. The defense sank to below-average but the offense rose slightly above it, and it was again tough to get a grasp on Casey’s acumen.

2013-14 – A new general manager, more shake-ups and 11 new faces throughout the year made for another tumultuous season, but this one brought far more success than the previous two. Casey became one of just a handful of coaches who could boast top-10 marks on both ends of the floor, and his fostering of a strong team identity was part of the reason the Raptors surprised with a 48-34 record. He earned his extension, in short.

A Rudimentary Statistical Look at Casey’s Performance
Over the past three seasons in total, the Raptors rank 16th in the NBA in offensive efficiency and 14th in defensive efficiency. Despite finishing 105-125 over those three years, the Raptors have been completely average during Casey’s tenure. And while we can’t account for the impact of a roster that was constantly in flux, we can at least compare those results to expectation given who Casey had to deploy.

To see how Casey performed relative to expectation, I found the weighted average Offensive and Defensive RAPM for the team over the past three years. There are issues with this method, of course – RAPM isn’t perfect and many don’t understand it, so some commenters will surely scoff at it right off the bat. That’s fine, we’re just exploring here, anyway. There’s also the issue that RAPM is based on lineup performance and, since these players all played with the Raptors for at least some of their careers, their Raptor time will affect their RAPM. Finally, this won’t account for any mistakes Casey has made in lineup deployment, as we’re looking at who did play, not who should have. Nonetheless, I think it’s a worthwhile comparison to make.

(Note: Ben Uzoh, Dwight Buycks, Julyan Stone, Justin Dentmon, Quincy Acy and Solomon Alabi, who combined for 1,220 minutes in the sample, were not included as they did not meet the RAPM minimums.)

PlayerMPORAPMDRAPM
DeMar DeRozan8236-0.5-1
Amir Johnson60921.74.1
Kyle Lowry48822.50.8
Jonas Valanciunas3764-1.6-0.9
Terrence Ross33980.8-2.7
Jose Calderon30722.1-2.9
Ed Davis2621-3.8-0.5
Andrea Bargnani2035-0.1-3.1
Alan Anderson1956-0.9-1.2
Rudy Gay17850.5-1.2
James Johnson1561-3.10
Linas Kleiza14330.3-0.8
Landry Fields13590.10.7
Aaron Gray1346-2.2-0.2
Greivis Vasquez13142.1-1
John Salmons1281-1.2-0.9
Patrick Patterson111800.5
Tyler Hansbrough978-3-0.8
Leandro Barbosa9461.6-2.5
John Lucas827-2.9-0.2
Gary Forbes713-0.1-1.4
Jerryd Bayless7050.7-3.4
Chuck Hayes575-0.94.1
Steve Novak540-0.8-2.4
Rasual Butler453-3.1-0.9
Mickael Pietrus386-1.41.2
Jamaal Magloire374-1.70.9
Dominic McGuire230-3.6-2.2
Anthony Carter209-1.7-0.5
Nando De Colo193-2-0.2
Sebastian Telfair185-1.4-2.2
D.J. Augustin820.1-3.2
Austin Daye33-1.4-1.1
TOTAL54682-0.08-0.40
ACTUAL55902-0.27-0.13

In short, the results of this don’t tell us much we probably didn’t already assume – Casey has done a slightly negative job on the offensive end and a slightly positive one on the defensive end, though again, all of the caveats ahead of the table apply. This was just an experiment, we’re not “taking advanced stats and using them in place of watching the game” or ignoring issues with the data – that’s all laid out, this is just descriptive, and I think the offense-defense result backs up what most would have assumed anyway.

Does Continuity Matter
Knowing all of that, I was curious as to whether keeping a coach in place even matters. That is, can a coach exhibit consistency in team’s performance? Obviously, a guy like Gregg Popovich can play a stapler 20 minutes a game and he’ll get Third-Team All-NBA votes, but for most coaches, that impact is less obvious, especially since rosters tend to turn over quite a bit.
casey Once again, we can’t control for the amount of roster turnover these teams have all had (well, we probably can, but I don’t have that kind of time or data), but we can at least compare how teams have done with and without coaching changes. It’s worth noting that bad teams are more likely to change coaches than good ones (and therefore more likely to change rosters), we’d expect teams that keep the same coach to be more consistent than those who change coaches. Just how much that’s true, however, is what I wanted to find out.

For that, we have my homies Andrew Lynch and Ian Levy to thank. Thanks to Andrew, I have era-adjusted O-Rating and D-Rating for every team dating back to 1973-74 and thanks to Ian, I know in which seasons teams underwent coaching changes.

First, let’s have a look at offense.
alloffense1 What we see here is that the correlation between a team’s offense year to year is moderately strong – a team’s offense the year before can explain about 40 percent of the variance in its offense the following year. How does this differ for teams with coaching changes?
offensecoaches2What we see here is that teams who retained their coaches had more stable offenses than those who did not. Teams with the same coach can explain about 44 percent of their offensive rating simply by looking at the prior year, while that’s only true for 28 percent of offensive variance for teams with new coaches.

Now, let’s turn to defense.
alldefense1What we see here is similar to on offense, though defenses are slightly less consistent year over year than offenses are. Again, let’s compare teams with and without coaching changes.
defensecoaches2Like with offense, teams that keep a coach in place are more consistent than those who see changes, though the R-squared falls from 44 percent on offense to 42 percent on defense.

So What?
Okay, so I threw some data and graphs at you, but what does it mean for the Raptors moving forward? Well, not necessarily a whole lot. We know that the team’s performance on both ends of the floor should be more stable than if there had been a coaching change, but that also accounts for less than half of the variance in team performance year over year. That is, Casey’s continued presence, and the assumption of roster stability that comes with performing well enough to save a coach’s job, is only half the battle.

The other half is, of course, where the focus of the offseason lands. Roster upgrades, system improvements and player development will all have a huge impact on how the Raptors look next season. We know that stability is just one factor, and our focus at Raptors Republic the next few weeks will now turn to how to make the team better.