Re: [SpectrumLabUsers] Numerical precision
- Am 10.12.2012 22:48, schrieb WB6BNQ:
Let me amplify the previous email a bit more. And, again ignoring the sound card, that is, assume its reported value is absolutely correct.
For example, when doing a Peakf function, to what number of digits is the value accurate before some limitation affects the reported value ?
As an example, good scientific calculators, like the hp brand, provide around 12 digits of accuracy before rounding and other errors start to affect the displayed digits.
The subject is very difficult, and I don't plan to dig much further into this subject... it would involved a detailed analysis of how rounding errors propagate through the FFT.
As Tom correctly noted, the precision of 32-bit floating point numbers (indeed they are IEEE754) is limited to 7 or 8 decimal digits. But most of the signal analysis functions (such as peak_f) operate on a larger number of 32-bit floats on the input, but the algorithms themselves (accumulators, etc) all use 64-bit floats.
For the frequency measurement (peak_f), I strongly suggest to try it, which is much easier than a mathematical analysis. From a practical example, a 10 kHz audio signal taken derived from a good OCXO could be measured with an accuracy (not just resolution) of less than 1 microhertz, i.e. more than 10 usable decimal places in the measurement - even though the samples were internally stored as a sequence of 'single precision floats'.
All the best,