Loading ...
Sorry, an error occurred while loading the content.

37561Re: Math Problems

Expand Messages
  • Sam Wetterlin
    Feb 1, 2009
    • 0 Attachment
      The problem isn't how the numbers are stored. It is that when a number
      is stored as an integer, LB, seems to think you want to do "infinite
      precision" math on it, which can take a horrendous amount of time if
      there are a lot of digits. That issue mainly arises with powers, which
      can build up a lot of digits quickly. In many cases, you know whether
      you are dealing with such numbers. But if you are working with
      user-supplied values, anything is possible.

      The point is that there are cases where the programmer needs to be
      sure that floating point math is performed. This is normally done by
      using a floating point data type, but in LB it needs to be done a
      little more indirectly. That was the purpose of the Power function I
      posted.

      Sam W.

      --- In libertybasic@yahoogroups.com, "rod_bird001" <rodbird@...> wrote:
      >
      > Just to say that Liberty will store a variable as a float whenever it
      > becomes one and as an integer whenever it becomes one. Simple
      > straightforward strategy.
      >
      > I'm not into maths at the level you are, I do struggle to understand
      > how the strategy prevents maths operations.
      >
    • Show all 6 messages in this topic