Loading ...
Sorry, an error occurred while loading the content.

Re: [SpectrumLabUsers] Numerical precision

Expand Messages
  • wolf_dl4yhf
    ... The subject is very difficult, and I don t plan to dig much further into this subject... it would involved a detailed analysis of how rounding errors
    Message 1 of 6 , Dec 11, 2012
    • 0 Attachment
      Am 10.12.2012 22:48, schrieb WB6BNQ:
       

      Hi
      Wolf,

      Let me amplify the previous email a bit more.  And, again ignoring the sound card, that is, assume its reported value is absolutely correct.

      For example, when doing a Peakf function, to what number of digits is the value accurate before some limitation affects the reported value ?

      As an example, good scientific calculators, like the hp brand, provide around 12 digits of accuracy before rounding and other errors start to affect the displayed digits.



      The subject is very difficult, and I don't plan to dig much further into this subject... it would involved a detailed analysis of how rounding errors propagate through the FFT.

      http://en.wikipedia.org/wiki/Floating_point#Machine_precision_and_backward_error_analysis


      As Tom correctly noted, the precision of 32-bit floating point numbers (indeed they are IEEE754) is limited to 7 or 8 decimal digits. But most of the signal analysis functions (such as peak_f) operate on a larger number of 32-bit floats on the input, but the algorithms themselves (accumulators, etc) all use 64-bit floats.
      For the frequency measurement (peak_f), I strongly suggest to try it, which is much easier than a mathematical analysis. From a practical example, a 10 kHz audio signal taken derived from a good OCXO could be measured with an accuracy (not just resolution) of less than 1 microhertz, i.e. more than 10 usable decimal places in the measurement - even though the samples were internally stored as a sequence of 'single precision floats'.

      All the best,
        Wolf .

    Your message has been successfully submitted and would be delivered to recipients shortly.