Advanced Metrics 101: PER, WS, WP, xRAPM

When I was little, I used to get the newspaper for the sports section. I enjoyed reading the color pieces, but somehow I’d always end up skipping them to peruse the box scores splayed across the margins. The numbers always fascinated me; I couldn’t watch a game being played between the Timberwolves and the Lakers, but I could look over the box score and see that Kevin Garnett had another 20/10 night. From the ink-stained margins, I could recreate the entire game in my mind.

Nowadays, I no longer need to hunt down copies of the Toronto Star; I can just fire up my laptop and watch an illegal stream NBA league pass. However, I haven’t stopped looking at box scores; now, I’m looking at them more than ever.

But the times they have a-changed. 20/10 just doesn’t mean what it used to. The NBA landscape has shifted, and “advanced” statistics like Player Efficiency Rating and Win Shares have emerged to the forefront of NBA analysis. It can be confusing and intimidating to the casual fan, and they must be especially perplexing to newcomers. The 11-year old version of me would’ve had a pretty difficult time imagining the game of a 15.0 PER player.

Therefore, I’ve decided to create a primer for the four major advanced box score statistics: Player Efficiency Rating (PER), Win Shares (WS), Wins Produced (WP) and (x)Regularized Adjusted Plus-Minus (xRAPM). I’m going to briefly break down how they’re computed, their strengths and flaws, provide the league average, and give some sense to the distribution for these numbers. In no way is this meant to be definitive; a primer is a primer.

 

Player Efficiency Rating (PER)

Developed by John Hollinger in the late 90’s, PER is definitely the most publicized advanced statistic today. Therefore it should come as no surprise that Hollinger wasted no time before riding his golden metric to fame and fortune at ESPN. With the support of ESPN’s platform, PER has been widely featured in everything from TV broadcasts to AP reports.

How is it computed?

The basic idea behind PER is pretty straightforward; to get an idea of a player’s total contributions, you add up the good, like field goals made, free throws made, assists, rebounds, blocks, and steals, and you take out the bad, like missed shots, turnovers, and personal fouls.

Each number in the box score has a corresponding linear weight 1 which translates the contribution into an effect on points per game 2. Points-per-game translates to offensive and defensive efficiency, which is highly correlated with wins. Therefore, you input the box score, and you get out a single number that captures a player’s total contributions on a per-minute basis.

It should be noted that PER is adjusted for minutes played. Therefore, a player like Demar Derozan who averages 18 points on 34 minutes per game isn’t necessarily a better scorer than say, Chris Copeland, who scores 8.7 points on 15 minutes per game.

PER is also adjusted for pace, so a player like Paul George isn’t punished relative to a player like Ty Lawson simply because the Pacers play 5% slower than the Denver Nuggets.

Finally, PER is adjusted for league average. Over time, the landscape of the NBA changes as new rules and new strategies emerge. PER is always adjusted to the yearly league average. Therefore, someone like Oscar Robertson isn’t punished for playing without a three-point line. PER is smart like that.

League Average and Distribution

League average for PER is ~15. There are slight permutations across positions, but for the most part, average PER is in the range of 14-16. A PER of 20 or greater is considered to be “all-star” worthy, while a PER of 10 or lower is considered to be a bench player. 3

Examples (2012-13 Miami Heat):

Joel Anthony – 10.61 (bench)
Mario Chalmers – 13.38 (league average)
Dwyane Wade – 24.25 (all-star)

What are PER’s Strengths and Weaknesses?

PER is really easy to understand. The range and distribution is simple and intuitive. For the most part, it deals with whole numbers, and it’s easy to quote. It’s also the most commonly referenced advanced stat, and bloggers and analysts alike reference often.

However, the best evaluator of these statistics is how well they correlate to wins. PER is able to explain 72% of variation in wins. For our purposes, we can just score PER at 72%, or a B grade.

The shortcomings of PER are many. First, John Hollinger himself admits that PER is not great at quantifying individual defense. It’s hard to accept that you can explain a significant portion of defense based on just defensive rebounding, steals, blocks and fouls. Second, it’s widely argued that the model rewards inefficient shooting. Basically, the math works out to that if a player shoots greater than ~33% from the field, more shooting will improve his PER figure. PER is also susceptible to small sample size fluctuations, but that can be said for any of the following models. Lastly, the weights were -at least in part- “guesstimated” by John Hollinger.

 

Win Shares (WS)

Win shares is another popular metric statistic. The general win shares model was developed by Bill James 4, and has been tweaked by the fine folks over at basketball reference 5. It attempts to frame a player’s total contribution in the context of wins. When you see that Chris Paul produced 13.9 win shares last season, it’s literally means that Chris Paul is solely responsible of ~14 of the Clippers’ wins.

How is it computed?

Like PER, the win shares model first translates each players’ contributions into points created via linear weights. I will summarize how offensive and defensive win shares are calculated separately. The total win shares for an individual player is the sum of his offensive and defensive win shares.

Offense

First, a set of linear weights is applied to a player’s box score in order to translate his stats into points created. Then, the number of offensive possessions used by said player is tabulated.

These two figures are used to calculate “marginal offense”, which adjusts for league average. Basically, this is calculated finding the difference between that specific player’s points created and the number of points that a league average player would have created given the same number of opportunities 6.

“Marginal points per wins” is then calculated to adjust for pace, and a specific number is attributed each team.

Finally, a player’s offensive win shares are calculated by dividing his “marginal offense” by his “marginal points per win”.

Defense

First, the defensive rating for each player is tabulated. This value is used to calculate “marginal defense” for each player. “Marginal defense” is a function of how minutes were played, how many defensive possessions “used[NA1] ”, and the league average for defense. Essentially, it calculates how a player performed on defense in comparison a league average player 7.

Then, each team’s specific “marginal points per win” is calculated. A player’s specific defensive win shares are calculated by dividing his “marginal defense” by his “marginal points per win”.

Win shares are often quoted as win shares per 48 minutes played (WS48), which is simply how many wins a player produces every 48 minutes played. This allows us to compare a 20 MPG guy with a 30 MPG player.

League Average and Distribution

The league average WS per season is 5, and average WS48 is 0.100. This is consistent across all positions. A player who produces 0.050 WS48 is considered to be bench fodder, and conversely, a 0.150 WS48 or greater is considered to be all-star level production.

Examples (2012-13 Miami Heat):

Joel Anthony – 0.100 WS48, 1.2 WS (bench fodder, although average WS48 number)
Mario Chalmers – 0.120 WS48, 5.2 WS (average player)
Dwyane Wade – 0.192 WS48, 9.6 WS (all-star)

What are WS’s Strengths and Weaknesses?

Win shares are great because it specifically spits out a value that is easily translatable to team success. For example, it’s hard to intuitively understand how much a player with a PER of 18 is affecting his team’s success 8. However, when we read something like Dwyane Wade produced ~10 wins for the Heat last season, it resonates with our intuitions.

The explanatory power of the Win Shares model is comparable to that of PER. It clocks in at 70.2%, or a B- grade. It’s pretty good.

However, WS does have its weaknesses. The biggest issue is that team performance factors into both offensive and defensive win shares, which contradicts the utility of WS as an individual stat. Also, win shares gets a bit weird around the fringes. Only 12 players last season registered negative win shares, which seems low given that the Bobcats employed at least 12 players is hard to believe. Win shares are also fallible to small sample size variations.

 

Wins Produced (WP)

No, it’s not the same as win shares, but the idea is similar. You know the drill; linear weights are used to condense the box score into a single number. The “wins produced” figure literally means that the player produced those many wins.

The wins produced community is very vocal, but their figures aren’t really publicized because they lack a significant platform. You can find WP numbers at The NBA Geek.

How is it computed?

The Wins Produced model was famously developed by Dave Berri, and two other economics professors. These three ran regressions 9 for every statistic in the box score against points scored/allowed. Basically, they came up with linear weights for each value in the box score. Heard this story before?

Anyway, there are a couple of other tweaks thrown in which factor in yearly league averages, positional differences, and team performance. These tweaks are very complicated, but aren’t very significant.

I’d go into more details, but they’re gory. If you insist, you can look over them here.

League Averages and Distribution

League average wins produced for a player is ~5 wins. This is a function of minutes played and Wins Produced per 48 minutes (WP48). Since minutes are often fudged, WP48 is often quoted. Average WP48 varies slightly across positions (like, +/- 0.005), but the average is 0.100.

Examples (2012-13 Miami Heat):

Joel Anthony – 1.32 wins produced, 0.112 WP48 (bench fodder, but average rate stats)
Mario Chalmers – 4.84 wins produced, 0.114 WP48 (league average, average rate stats)
Dwyane Wade – 11.13 wins produced, 0.223 WP48 (superstar)

What are WP’s Strengths and Weaknesses?

Wins Produced tends to be a lightning rod for criticism because the results can often be very intuitively whacky. For example, the wins produced model has Danny Green producing more wins AND having a higher WP48 than Tony Parker in 2012-2013.

This traces back to how much you value shot-creation. Wins Produced believes that unless you shoot at a rate better than the average at your position, the value you create is largely negative (there are exceptions). For example, a player like Carmelo Anthony, who shot slightly below the TS% of an average small forward in 2011-12, got no overall credit for his points scored. Similarly, if you value shot creation, you’ll furrow you brow at the fact that Tyson Chandler had a WP48 that was eight times higher than Anthony’s (in 2012-13).

A lot of Chandler’s value was tied up in his rebounding, which is another slag against wins produced. The value from a defensive rebound outweighs that of a made two-point field goal.

Wins Produced also doesn’t attempt to explain “why”, it just measures “what”. This can be said for all of these stats, but WP is most often criticized for this. Also, a missed field goal costs more than a made field goal. You can look over the coefficients here. They’re pretty weird sometimes.

However, Wins Produced does have two major things going for it. First, it’s perhaps the most rigorously developed statistic. It was produced by three economists and it was published in a journal, meaning it stood up to peer review. Second, Wins Produced correlates most strongly with actual wins. 95% of variation in wins at the team level is explained by Wins Produced model 10.

 

xRegularized Adjusted Plus-Minus (xRAPM)

I’m not going to lie to you. This is the stat that I’m least familiar with. 11

The allure of plus-minus is intuitive; how well did the team do when you were on and how well did they fare when you were sitting? However, that’s  also where it’s misleading. The premise of plus-minus measures far too much context. It’s like picking up the haystack and claiming that you’ve found the needle.

So why include xRAPM? Because it’s the best plus-minus based statistic out there. PER, WS, and WP are all based on the same data and follow a similar methodology. The differences are minimal. xRAPM comes at the problem from a different way; PER, WS and WP attempts to solve the problem bottom-up, and xRAPM approaches evaluation from top-to-bottom. The truth is likely somewhere in the middle.

You can find xRAPM here. However, the stat is rarely publicized, and I know of no database that posts updated xRAPM numbers. Please alert me if you do!

How it’s computed

Here’s where it gets really tricky. As far as I can tell, there is no FAQ for the procedures like there is with PER, WS and WP, so you’ll just have to trust my readings on this stat.

xRAPM has two parts; one part plus-minus, one part box score-based. I’ll address each point separately.

The plus-minus half comes from RAPM, which is regularized adjusted plus-minus. The basis of RAPM is APM 12, which is adjusted plus-minus. APM is simply plus-minus, but it tries to parse out the compounding effects of a player’s teammates and his opposition. However, APM is highly variable and the range is simply too large. Regulation corrects for this with ridge regression 13.

The box score half comes from a nifty little boxscore summation statistic, which as far as I know, does not have an official name. In nature, it is much like PER or WS, in that there are coefficients given to each column in the boxscore. The purpose of this boxscore statistic is to improve the predictive power RAPM. The nitty-gritty of this statistic can be found here.

Combine boxscore stat, and RAPM, you get xRAPM. Got all that?

I know this doesn’t explain much. Think of it this way; the caffeine is in the coffee grounds (APM). RAPM is like putting these grounds through a coffee maker; it gives you black coffee. Adding the boxscore portion is like sugar and cream; it makes the coffee palatable.

Anyway, xRAPM is often presented as offensive and defensive xRAPM (per 200 possessions). The stat guys like to keep things confusing.

League Average and Distribution

I don’t actually have the full dataset for this stuff. Simply put, it’s hard to access and not a lot of people are using it. Nevertheless, I did calculate the averages from last season. They are broken down into the following:

(one standard deviation after //)

Average offensive xRAPM per 200 possessions: -0.62 // 1.84

Average defensive xRAPM per 200 possessions: -0.44 // 1.93

Average xRAPM per 200 possessions: -1.06 //2.55

2012-13 Miami Heat (off.xRAPM/def.xRAPM/xRAPM)

Joel Anthony: -2.7/1.8/-0.9

Mario Chalmers: -0.3/0.8/0.5

Dwyane Wade: 3.4/1.1/4.5

What are xRAPM’s Strengths and Weaknesses?

I find fault in the entire plus-minus approach. I think it accounts for way too much context. It’s like picking up the haystack and declaring that you’ve found the needle. There’s simply too many extenuating factors; how your teammates perform, who they are, who your back-up is, who you’re up against, what your role is in the offense/defense.  Plus-minus, however adjusted or corrected, measures far too much context to be considered a true individual statistic.

Perhaps Jeremias Englemann (the creator of xRAPM) tried to correct this by adding in the boxscore component. This seems to have rectified some of the inherent issues with plus-minus, and it has apparently improved its predictive and descriptive power.

It should also be noted that using regularization (the R in xRAPM) creates a bias wherein players who have not played very many minutes are pushed significantly towards the average. You can read more about the technical issues here.

However, if you like the plus-minus approach, and you dislike box score stats, this is your best bet. Then again, new plus-minus stats are created each year, so it might quickly become outdated. Who knows?

Anyway, I hope this primer made metric stats a little easier to digest. If the original premise was to allow an 11-year-old me understand these advanced statistics, then I’ve definitely failed. Hopefully you’re smarter than my former self, and this has made some sense to you. Personally, after writing this primer, I’m more confused than ever.

And remember: all these stats answer the question WHAT. HOW and WHY are much more difficult to answer. If you find intuition at odds with an advanced stat, you’re probably questioning it based on how and why, in which these stats hold no answers. That’s why we watch the games… well, that and Steve Nash’s behind-the-back passes:
 

 

[NOTE - 10/19/2013] The xRAPM section of this article was restructured and updated.2652 Oranges Squeezed

Notes:

  1. meaning no diminishing returns
  2. defensive numbers affect opponent points per game
  3. FYI – According to Hollinger, the reference guide is:

    • A Year For the Ages: 35.0
    • Runaway MVP Candidate: 30.0
    • Strong MVP Candidate: 27.5
    • Weak MVP Candidate: 25.0
    • Bona fide All-Star: 22.5
    • Borderline All-Star: 20.0
    • Solid 2nd option: 18.0
    • 3rd Banana: 16.5
    • Pretty good player: 15.0
    • In the rotation: 13.0
    • Scrounging for minutes: 11.0
    • Definitely renting: 9.0
    • The Next Stop: DLeague 5.0

  4. of Moneyball fame, and others
  5. notably Justin Kubatko
  6. Yes, it’s possible for this number to be negative; whaddup, Bargnani?
  7. Yes, this too can be negative; whaddup again, Bargnani!
  8. although it is possible to translate PER to wins
  9. which measure how one variable is effected by another
  10. although this is in retrospect; it’s not great as a prediction
Bookmark the permalink.

2 Comments

  1. Hello: I’m the creator of ASPM, and have worked with Jeremias Engelmann on xRAPM. A correction: ASPM is not used in xRAPM. A much simpler, purely linear set of box score weights were found by J.E. via cross validation to use in stabilizing xRAPM.

    Jeremias went through his derivation of xRAPM here: http://www.apbr.org/metrics/viewtopic.php?f=2&t=8025

    Daniel

  2. Pingback: The Bucks Are Coming…Soon | Squeeze The Orange

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>