Loading ...
Sorry, an error occurred while loading the content.

Re: [SpectrumLabUsers] Numerical precision

Expand Messages
  • Tom Becker
    ... IEEE754 32-bit floating point values are equivalent to about seven digits of decimal precision. A real-world example expresses approximately one foot of
    Message 1 of 6 , Dec 10, 2012
    • 0 Attachment
      > ... how many digits are valid...

      IEEE754 32-bit floating point values are equivalent to about seven
      digits of decimal precision. A real-world example expresses
      approximately one foot of longitude at the Equator.
      http://en.wikipedia.org/wiki/IEEE_754-2008


      Tom
    • wolf_dl4yhf
      ... The subject is very difficult, and I don t plan to dig much further into this subject... it would involved a detailed analysis of how rounding errors
      Message 2 of 6 , Dec 11, 2012
      • 0 Attachment
        Am 10.12.2012 22:48, schrieb WB6BNQ:
         

        Hi
        Wolf,

        Let me amplify the previous email a bit more.  And, again ignoring the sound card, that is, assume its reported value is absolutely correct.

        For example, when doing a Peakf function, to what number of digits is the value accurate before some limitation affects the reported value ?

        As an example, good scientific calculators, like the hp brand, provide around 12 digits of accuracy before rounding and other errors start to affect the displayed digits.



        The subject is very difficult, and I don't plan to dig much further into this subject... it would involved a detailed analysis of how rounding errors propagate through the FFT.

        http://en.wikipedia.org/wiki/Floating_point#Machine_precision_and_backward_error_analysis


        As Tom correctly noted, the precision of 32-bit floating point numbers (indeed they are IEEE754) is limited to 7 or 8 decimal digits. But most of the signal analysis functions (such as peak_f) operate on a larger number of 32-bit floats on the input, but the algorithms themselves (accumulators, etc) all use 64-bit floats.
        For the frequency measurement (peak_f), I strongly suggest to try it, which is much easier than a mathematical analysis. From a practical example, a 10 kHz audio signal taken derived from a good OCXO could be measured with an accuracy (not just resolution) of less than 1 microhertz, i.e. more than 10 usable decimal places in the measurement - even though the samples were internally stored as a sequence of 'single precision floats'.

        All the best,
          Wolf .

      Your message has been successfully submitted and would be delivered to recipients shortly.