Loading ...
Sorry, an error occurred while loading the content.

Re: Math Problems

Expand Messages
  • Sam Wetterlin
    The problem isn t how the numbers are stored. It is that when a number is stored as an integer, LB, seems to think you want to do infinite precision math on
    Message 1 of 6 , Feb 1, 2009
    • 0 Attachment
      The problem isn't how the numbers are stored. It is that when a number
      is stored as an integer, LB, seems to think you want to do "infinite
      precision" math on it, which can take a horrendous amount of time if
      there are a lot of digits. That issue mainly arises with powers, which
      can build up a lot of digits quickly. In many cases, you know whether
      you are dealing with such numbers. But if you are working with
      user-supplied values, anything is possible.

      The point is that there are cases where the programmer needs to be
      sure that floating point math is performed. This is normally done by
      using a floating point data type, but in LB it needs to be done a
      little more indirectly. That was the purpose of the Power function I
      posted.

      Sam W.

      --- In libertybasic@yahoogroups.com, "rod_bird001" <rodbird@...> wrote:
      >
      > Just to say that Liberty will store a variable as a float whenever it
      > becomes one and as an integer whenever it becomes one. Simple
      > straightforward strategy.
      >
      > I'm not into maths at the level you are, I do struggle to understand
      > how the strategy prevents maths operations.
      >
    • rod_bird001
      Ok, bit outside my skill set. If Liberty does seem to hang try Ctrl Break, click OK on the notice and then close or kill the program
      Message 2 of 6 , Feb 1, 2009
      • 0 Attachment
        Ok, bit outside my skill set. If Liberty does seem to "hang" try Ctrl
        Break, click OK on the notice and then close or kill the program
      Your message has been successfully submitted and would be delivered to recipients shortly.