37561Re: Math Problems
- Feb 1, 2009The problem isn't how the numbers are stored. It is that when a number
is stored as an integer, LB, seems to think you want to do "infinite
precision" math on it, which can take a horrendous amount of time if
there are a lot of digits. That issue mainly arises with powers, which
can build up a lot of digits quickly. In many cases, you know whether
you are dealing with such numbers. But if you are working with
user-supplied values, anything is possible.
The point is that there are cases where the programmer needs to be
sure that floating point math is performed. This is normally done by
using a floating point data type, but in LB it needs to be done a
little more indirectly. That was the purpose of the Power function I
--- In firstname.lastname@example.org, "rod_bird001" <rodbird@...> wrote:
> Just to say that Liberty will store a variable as a float whenever it
> becomes one and as an integer whenever it becomes one. Simple
> straightforward strategy.
> I'm not into maths at the level you are, I do struggle to understand
> how the strategy prevents maths operations.
- << Previous post in topic Next post in topic >>