## 1856Consistency Score

Expand Messages
• Feb 25, 2003
• 0 Attachment
You often hear about a player's consistency, but it always seemed
like such a subjective notion to me. Here is my attempt to quantify
consistency.

Consistency Score
CS = 1 / (sd / mean)

where

sd = the standard deviation of some measure of a players performance
over the course of a given time frame.

mean = the average of the players performance over the same time
frame. For these calculations, I actually used a trimmed mean, which
ignores the bottom and top 5% of the performances, making the measure
less sensitive to outliers.

Consistency Scores will be higher for consistent players, and
decrease towards zero for inconsistent players.

Here's an example: Karl Malone has often been called a consistent
player, and Olowakandi an inconsistent one. Let's compare their 01-02
seasons. (Legend at the bottom if you don't understand the table.)

Effective FG%

Player N Mean TrMn SDv Min Q1 Med Q3 Max CS
malone 80 46 46 12 9 39 46 53 78 3.80
olowokandi 80 41 41 19 0 29 46 54 100 2.16

I'll use an ascii pseudo-boxplot to highlight the differences
graphically. (Boxplot legend at bottom if you don't understand how I
drew it.)

Player CS

malone 3.8 |--------| + |-------|

olowokandi 2.2 |--------| + |-------------|

You can see here how Kandi's shooting varied much more over the
course of the season. Malone, the emblem of consistent performance,
had a top 20 Consistency Score in EffFG% last season. Kandi placed
133rd.

Manley Credits

Manley credits (MC), defined at the bottom, are basically the sum of
all good boxscore stats (minus misses and turnovers). It provides a
simple (and simplistic) method of assessing a player's overall
contributions. This is real Malone territory -- a stat in which he's
put up some rather remarkable numbers. By himself, Malone holds about
5 or 6 of the 200 best player-seasons in history. Last season, Malone
place 11th in the league, and Kandi 86th.

But if we ignore the advantage of Malone in terms of raw numbers, and
look instead at the amount of game-to-game variation, which one of
these players is more consistent?

Player N Mean TrMn SDv Min Q1 Med Q3 Max CS
malone 80 23.4 23.5 8.6 0 18.3 23 30 41 2.74
olowokandi 80 14.2 14.1 8.4 -5 8.3 13 19 38 1.68

Malone's Consistency Score is 2.7, and Kandi's 1.7. The scores are
most affected by the standard deviation -- which you can see above
are almost identical -- in relation to the average performance, which
in Malone's case is a very high 23.4 Manley Credits /g, and in
Kandi's case is a more mundane 14.2/g.

Graphically, the variation during the season for each player looks
like this:

Player CS

malone 2.7 |-----------| + |-------|

olowokandi 1.7 |--------| + |------------|

Both players have a similar amount of variation, as shown in the
table above in the standard deviation column. The ideally consistent
player would have an SDv of zero -- he would score the same MC in
every game that he played. A larger SDv registers the amount of
variation that player has game to game. But a player whose average
score is very close to zero would automatically have a smaller SDv,
simply because it's impossible to vary much between his average score
and zero.[*]

[* Technically, players can have a negative MC game. Olowakandi's
worst game was minus 5, for example. But this is a rare event, and so
in the interests of simplicity, I'm ignoring the possibility of
negative games.]

Consistency Scores was developed to get around the problem of simply
using standard deviations as a measure of consistency. Instead of
computing the _absolute_ deviations from a player's average (which is
basically what SDv is), Consistency Scores measures the _relative_
deviations -- that is, the amount of deviations relative to the
player's average.

[* Consistency score began as a version of what I think is called
Coefficient of Variation: SDv/mean. I used the inverse of the CV (and
used the trimmed mean instead of the regular mean) to produce the CS.
I know there are some restrictions for using CV -- IIRC, the
distributions have to be normal, there has to be a real zero, and
some other stuff I can't remember. But because it's so easy to
compute, I'm going to ignore these potential problems, and use it
anyway -- unless anyone can give me a good reason not to. My
statistical knowledge is entirely self-taught, and so I'm prone to
making the most elementary errors in statistical logic and
computation.]

The upshot of all of this is that Malone's Consistency Score is
higher than Kandi's because, even though they have a similar amount
of inconsistency to their games, Malone's inconsistency is much
smaller in comparison to his average Manley Credits / game. Malone's
average was 23.4 MC/g, and his standard deviation was 8.6.

Consistency Score
CS = 1 / (sd / mean)
= 1 / (8.6 / 23.4)
= 2.7

Kandi's average was 14.2, and his SDv was 8.4.

CS = 1 / (sd / mean)
= 1 / (8.4 / 14.2)
= 1.7

* * * * *

There's another stat I sometimes use when I have no defensive numbers
available: minutes played. This is sometimes helpful in inferring the
amount of defensive ability of non-scoring players, based on the
assumption that the players must be getting playing time for _some_
reason -- if it's not for their offense, then it must be defense.

I don't think minutes per game would be too helpful in comparing
Malone and Olowakandi, but for the sake of completeness, I'll include
it here:

Minutes

Player N Mean TrMn SDv Min Q1 Med Q3 Max CS
malone 80 38.0 38.1 5.4 25 34 40 42 53 7.07
olowokandi 80 32.1 32.3 8.3 12 25 33 39 46 3.91

Player CS

malone 7.1 |-------| + |--------|

olowokandi 3.9 |----------| + |-----|

TABLE LEGEND
------------

N -- Number of games

Mean -- Season average

TrMn -- Season average if you ignore the best 5% of the
performances, and the worst 5%

SDv -- Standard deviation

Min -- Minimum, the worst performance.

Q1 -- 1st quartile, basically, the midpoint of the worst half of
the performances

Med -- Median, the point at which half of the performances were
better and half worse

Q3 -- 3rd quartile, the midpoint of the best half of the
performances

Max -- Maximum, the best performance

CS -- Consistency score, see above for formula

BOXPLOTS
--------

0 min Q1 med Q3 Max
| | | | | |
v v v v v v

|--------| + |-------|

MANLEY CREDITS
--------------

Manley Credits = PTS+REB+AST+STL+BLK-FGMISSED-FTMISSED-TO
• Show all 7 messages in this topic