Re: [SpectrumLabUsers] Numerical precision

Expand Messages
• ... IEEE754 32-bit floating point values are equivalent to about seven digits of decimal precision. A real-world example expresses approximately one foot of
Message 1 of 6 , Dec 10, 2012
> ... how many digits are valid...

IEEE754 32-bit floating point values are equivalent to about seven
digits of decimal precision. A real-world example expresses
approximately one foot of longitude at the Equator.
http://en.wikipedia.org/wiki/IEEE_754-2008

Tom
• ... The subject is very difficult, and I don t plan to dig much further into this subject... it would involved a detailed analysis of how rounding errors
Message 2 of 6 , Dec 11, 2012
Am 10.12.2012 22:48, schrieb WB6BNQ:

Hi
Wolf,

Let me amplify the previous email a bit more.  And, again ignoring the sound card, that is, assume its reported value is absolutely correct.

For example, when doing a Peakf function, to what number of digits is the value accurate before some limitation affects the reported value ?

As an example, good scientific calculators, like the hp brand, provide around 12 digits of accuracy before rounding and other errors start to affect the displayed digits.

The subject is very difficult, and I don't plan to dig much further into this subject... it would involved a detailed analysis of how rounding errors propagate through the FFT.

http://en.wikipedia.org/wiki/Floating_point#Machine_precision_and_backward_error_analysis

As Tom correctly noted, the precision of 32-bit floating point numbers (indeed they are IEEE754) is limited to 7 or 8 decimal digits. But most of the signal analysis functions (such as peak_f) operate on a larger number of 32-bit floats on the input, but the algorithms themselves (accumulators, etc) all use 64-bit floats.
For the frequency measurement (peak_f), I strongly suggest to try it, which is much easier than a mathematical analysis. From a practical example, a 10 kHz audio signal taken derived from a good OCXO could be measured with an accuracy (not just resolution) of less than 1 microhertz, i.e. more than 10 usable decimal places in the measurement - even though the samples were internally stored as a sequence of 'single precision floats'.

All the best,
Wolf .

Your message has been successfully submitted and would be delivered to recipients shortly.