Loading ...
Sorry, an error occurred while loading the content.

1856Consistency Score

Expand Messages
  • edkupfer <igorkupfer@rogers.com>
    Feb 25, 2003
    • 0 Attachment
      You often hear about a player's consistency, but it always seemed
      like such a subjective notion to me. Here is my attempt to quantify

      Consistency Score
      CS = 1 / (sd / mean)


      sd = the standard deviation of some measure of a players performance
      over the course of a given time frame.

      mean = the average of the players performance over the same time
      frame. For these calculations, I actually used a trimmed mean, which
      ignores the bottom and top 5% of the performances, making the measure
      less sensitive to outliers.

      Consistency Scores will be higher for consistent players, and
      decrease towards zero for inconsistent players.

      Here's an example: Karl Malone has often been called a consistent
      player, and Olowakandi an inconsistent one. Let's compare their 01-02
      seasons. (Legend at the bottom if you don't understand the table.)

      Effective FG%

      Player N Mean TrMn SDv Min Q1 Med Q3 Max CS
      malone 80 46 46 12 9 39 46 53 78 3.80
      olowokandi 80 41 41 19 0 29 46 54 100 2.16

      I'll use an ascii pseudo-boxplot to highlight the differences
      graphically. (Boxplot legend at bottom if you don't understand how I
      drew it.)

      Player CS

      malone 3.8 |--------| + |-------|

      olowokandi 2.2 |--------| + |-------------|

      You can see here how Kandi's shooting varied much more over the
      course of the season. Malone, the emblem of consistent performance,
      had a top 20 Consistency Score in EffFG% last season. Kandi placed

      Manley Credits

      Manley credits (MC), defined at the bottom, are basically the sum of
      all good boxscore stats (minus misses and turnovers). It provides a
      simple (and simplistic) method of assessing a player's overall
      contributions. This is real Malone territory -- a stat in which he's
      put up some rather remarkable numbers. By himself, Malone holds about
      5 or 6 of the 200 best player-seasons in history. Last season, Malone
      place 11th in the league, and Kandi 86th.

      But if we ignore the advantage of Malone in terms of raw numbers, and
      look instead at the amount of game-to-game variation, which one of
      these players is more consistent?

      Player N Mean TrMn SDv Min Q1 Med Q3 Max CS
      malone 80 23.4 23.5 8.6 0 18.3 23 30 41 2.74
      olowokandi 80 14.2 14.1 8.4 -5 8.3 13 19 38 1.68

      Malone's Consistency Score is 2.7, and Kandi's 1.7. The scores are
      most affected by the standard deviation -- which you can see above
      are almost identical -- in relation to the average performance, which
      in Malone's case is a very high 23.4 Manley Credits /g, and in
      Kandi's case is a more mundane 14.2/g.

      Graphically, the variation during the season for each player looks
      like this:

      Player CS

      malone 2.7 |-----------| + |-------|

      olowokandi 1.7 |--------| + |------------|

      Both players have a similar amount of variation, as shown in the
      table above in the standard deviation column. The ideally consistent
      player would have an SDv of zero -- he would score the same MC in
      every game that he played. A larger SDv registers the amount of
      variation that player has game to game. But a player whose average
      score is very close to zero would automatically have a smaller SDv,
      simply because it's impossible to vary much between his average score
      and zero.[*]

      [* Technically, players can have a negative MC game. Olowakandi's
      worst game was minus 5, for example. But this is a rare event, and so
      in the interests of simplicity, I'm ignoring the possibility of
      negative games.]

      Consistency Scores was developed to get around the problem of simply
      using standard deviations as a measure of consistency. Instead of
      computing the _absolute_ deviations from a player's average (which is
      basically what SDv is), Consistency Scores measures the _relative_
      deviations -- that is, the amount of deviations relative to the
      player's average.

      [* Consistency score began as a version of what I think is called
      Coefficient of Variation: SDv/mean. I used the inverse of the CV (and
      used the trimmed mean instead of the regular mean) to produce the CS.
      I know there are some restrictions for using CV -- IIRC, the
      distributions have to be normal, there has to be a real zero, and
      some other stuff I can't remember. But because it's so easy to
      compute, I'm going to ignore these potential problems, and use it
      anyway -- unless anyone can give me a good reason not to. My
      statistical knowledge is entirely self-taught, and so I'm prone to
      making the most elementary errors in statistical logic and

      The upshot of all of this is that Malone's Consistency Score is
      higher than Kandi's because, even though they have a similar amount
      of inconsistency to their games, Malone's inconsistency is much
      smaller in comparison to his average Manley Credits / game. Malone's
      average was 23.4 MC/g, and his standard deviation was 8.6.

      Consistency Score
      CS = 1 / (sd / mean)
      = 1 / (8.6 / 23.4)
      = 2.7

      Kandi's average was 14.2, and his SDv was 8.4.

      CS = 1 / (sd / mean)
      = 1 / (8.4 / 14.2)
      = 1.7

      * * * * *

      There's another stat I sometimes use when I have no defensive numbers
      available: minutes played. This is sometimes helpful in inferring the
      amount of defensive ability of non-scoring players, based on the
      assumption that the players must be getting playing time for _some_
      reason -- if it's not for their offense, then it must be defense.

      I don't think minutes per game would be too helpful in comparing
      Malone and Olowakandi, but for the sake of completeness, I'll include
      it here:


      Player N Mean TrMn SDv Min Q1 Med Q3 Max CS
      malone 80 38.0 38.1 5.4 25 34 40 42 53 7.07
      olowokandi 80 32.1 32.3 8.3 12 25 33 39 46 3.91

      Player CS

      malone 7.1 |-------| + |--------|

      olowokandi 3.9 |----------| + |-----|


      N -- Number of games

      Mean -- Season average

      TrMn -- Season average if you ignore the best 5% of the
      performances, and the worst 5%

      SDv -- Standard deviation

      Min -- Minimum, the worst performance.

      Q1 -- 1st quartile, basically, the midpoint of the worst half of
      the performances

      Med -- Median, the point at which half of the performances were
      better and half worse

      Q3 -- 3rd quartile, the midpoint of the best half of the

      Max -- Maximum, the best performance

      CS -- Consistency score, see above for formula


      0 min Q1 med Q3 Max
      | | | | | |
      v v v v v v

      |--------| + |-------|


    • Show all 7 messages in this topic