- Feb 25, 2003You often hear about a player's consistency, but it always seemed

like such a subjective notion to me. Here is my attempt to quantify

consistency.

Consistency Score

CS = 1 / (sd / mean)

where

sd = the standard deviation of some measure of a players performance

over the course of a given time frame.

mean = the average of the players performance over the same time

frame. For these calculations, I actually used a trimmed mean, which

ignores the bottom and top 5% of the performances, making the measure

less sensitive to outliers.

Consistency Scores will be higher for consistent players, and

decrease towards zero for inconsistent players.

Here's an example: Karl Malone has often been called a consistent

player, and Olowakandi an inconsistent one. Let's compare their 01-02

seasons. (Legend at the bottom if you don't understand the table.)

Effective FG%

Player N Mean TrMn SDv Min Q1 Med Q3 Max CS

malone 80 46 46 12 9 39 46 53 78 3.80

olowokandi 80 41 41 19 0 29 46 54 100 2.16

I'll use an ascii pseudo-boxplot to highlight the differences

graphically. (Boxplot legend at bottom if you don't understand how I

drew it.)

Player CS

malone 3.8 |--------| + |-------|

olowokandi 2.2 |--------| + |-------------|

You can see here how Kandi's shooting varied much more over the

course of the season. Malone, the emblem of consistent performance,

had a top 20 Consistency Score in EffFG% last season. Kandi placed

133rd.

Manley Credits

Manley credits (MC), defined at the bottom, are basically the sum of

all good boxscore stats (minus misses and turnovers). It provides a

simple (and simplistic) method of assessing a player's overall

contributions. This is real Malone territory -- a stat in which he's

put up some rather remarkable numbers. By himself, Malone holds about

5 or 6 of the 200 best player-seasons in history. Last season, Malone

place 11th in the league, and Kandi 86th.

But if we ignore the advantage of Malone in terms of raw numbers, and

look instead at the amount of game-to-game variation, which one of

these players is more consistent?

Player N Mean TrMn SDv Min Q1 Med Q3 Max CS

malone 80 23.4 23.5 8.6 0 18.3 23 30 41 2.74

olowokandi 80 14.2 14.1 8.4 -5 8.3 13 19 38 1.68

Malone's Consistency Score is 2.7, and Kandi's 1.7. The scores are

most affected by the standard deviation -- which you can see above

are almost identical -- in relation to the average performance, which

in Malone's case is a very high 23.4 Manley Credits /g, and in

Kandi's case is a more mundane 14.2/g.

Graphically, the variation during the season for each player looks

like this:

Player CS

malone 2.7 |-----------| + |-------|

olowokandi 1.7 |--------| + |------------|

Both players have a similar amount of variation, as shown in the

table above in the standard deviation column. The ideally consistent

player would have an SDv of zero -- he would score the same MC in

every game that he played. A larger SDv registers the amount of

variation that player has game to game. But a player whose average

score is very close to zero would automatically have a smaller SDv,

simply because it's impossible to vary much between his average score

and zero.[*]

[* Technically, players can have a negative MC game. Olowakandi's

worst game was minus 5, for example. But this is a rare event, and so

in the interests of simplicity, I'm ignoring the possibility of

negative games.]

Consistency Scores was developed to get around the problem of simply

using standard deviations as a measure of consistency. Instead of

computing the _absolute_ deviations from a player's average (which is

basically what SDv is), Consistency Scores measures the _relative_

deviations -- that is, the amount of deviations relative to the

player's average.

[* Consistency score began as a version of what I think is called

Coefficient of Variation: SDv/mean. I used the inverse of the CV (and

used the trimmed mean instead of the regular mean) to produce the CS.

I know there are some restrictions for using CV -- IIRC, the

distributions have to be normal, there has to be a real zero, and

some other stuff I can't remember. But because it's so easy to

compute, I'm going to ignore these potential problems, and use it

anyway -- unless anyone can give me a good reason not to. My

statistical knowledge is entirely self-taught, and so I'm prone to

making the most elementary errors in statistical logic and

computation.]

The upshot of all of this is that Malone's Consistency Score is

higher than Kandi's because, even though they have a similar amount

of inconsistency to their games, Malone's inconsistency is much

smaller in comparison to his average Manley Credits / game. Malone's

average was 23.4 MC/g, and his standard deviation was 8.6.

Consistency Score

CS = 1 / (sd / mean)

= 1 / (8.6 / 23.4)

= 2.7

Kandi's average was 14.2, and his SDv was 8.4.

CS = 1 / (sd / mean)

= 1 / (8.4 / 14.2)

= 1.7

* * * * *

There's another stat I sometimes use when I have no defensive numbers

available: minutes played. This is sometimes helpful in inferring the

amount of defensive ability of non-scoring players, based on the

assumption that the players must be getting playing time for _some_

reason -- if it's not for their offense, then it must be defense.

I don't think minutes per game would be too helpful in comparing

Malone and Olowakandi, but for the sake of completeness, I'll include

it here:

Minutes

Player N Mean TrMn SDv Min Q1 Med Q3 Max CS

malone 80 38.0 38.1 5.4 25 34 40 42 53 7.07

olowokandi 80 32.1 32.3 8.3 12 25 33 39 46 3.91

Player CS

malone 7.1 |-------| + |--------|

olowokandi 3.9 |----------| + |-----|

TABLE LEGEND

------------

N -- Number of games

Mean -- Season average

TrMn -- Season average if you ignore the best 5% of the

performances, and the worst 5%

SDv -- Standard deviation

Min -- Minimum, the worst performance.

Q1 -- 1st quartile, basically, the midpoint of the worst half of

the performances

Med -- Median, the point at which half of the performances were

better and half worse

Q3 -- 3rd quartile, the midpoint of the best half of the

performances

Max -- Maximum, the best performance

CS -- Consistency score, see above for formula

BOXPLOTS

--------

0 min Q1 med Q3 Max

| | | | | |

v v v v v v

|--------| + |-------|

MANLEY CREDITS

--------------

Manley Credits = PTS+REB+AST+STL+BLK-FGMISSED-FTMISSED-TO - Next post in topic >>