Loading ...
Sorry, an error occurred while loading the content.
 

Re: [hackers-il] I want to believe

Expand Messages
  • Nadav Har'El
    ... First, the square root of the variance (the standard deviation) gives a more meaningful measure... Second, I don t see what the high variance has anything
    Message 1 of 2 , Feb 20, 2001
      On Tue, Feb 20, 2001, Chen Shapira wrote about "[hackers-il] I want to believe":
      > Well, this particular test, had 18 questions, a right answers gives you 6
      > points, a wrong one substracts 2.
      > Lets assume I came to the test with 75% knowledge.
      > So each question I get right with 0.75 probability.
      > The expectancy of my score is quite reasonable - 72.
      > The variance on the other hand is 6048 (18 independent questions each with
      > variance of 12).
      > Unless I had an error of calculations, this is an extremely high variance,
      > meaning that a test score has low chance of reflecting my actual knowledge.

      First, the square root of the variance (the standard deviation) gives a
      more meaningful measure...

      Second, I don't see what the high variance has anything to do with the
      correlation between the score and your knowledge! I mean, if you answered
      questions by random, then you're right - you'd expect a crazy distribution
      of random scores, ranging from -36 to 108 with a high variance. But that's
      not what happens: assume you have a intrinsic success fraction A (e.g., 0.75 in
      your previous example), and a small chance of making errors on questions
      you ought to know or succeeding in questions you ought not to know.
      This small chance means that A is no longer a constant, but rather a random
      variable with a narrow variance (not that it doesn't have anything to do
      with the variance you calculated).

      So, if A is gaussian with a mean 0.75 and standard deviation, say, 0.05,
      then you can calculate your score to be roughly
      S = 18*6*A - 18*2*(1-A)
      or
      S = 144*A - 36

      If you had a success rate A=0 (a constant distribution), you'd get -36,
      if you had a success rate A=1 (a constant distribution), you'd get 108
      (your teacher invented these weird limits)

      But in our example, where A is a gaussian with mean 0.75 and stdandard
      deviation 0.05, you'd get that the expected value of S is 144*0.75-36,
      or 72 (as you calculated), and a standard deviation of 144*0.05, or 7.2
      points.

      It's only slightly higher than the 5 point standard deviation you'd "expect"
      when you chose 0.05 as a standard deviation for A, and it is higher only
      because your range (-36..108) is higher than the range 0..100.

      So I don't see why you think this test can't reflect your knowledge.


      --
      Nadav Har'El | Tuesday, Feb 20 2001, 28 Shevat 5761
      nyh@... |-----------------------------------------
      Phone: +972-53-245868, ICQ 13349191 |I love deadlines. I love the whooshing
      http://nadav.harel.org.il |sound they make as they go by.
    Your message has been successfully submitted and would be delivered to recipients shortly.