Loading ...
Sorry, an error occurred while loading the content.

Re: INT problem

Expand Messages
  • paradisejb
    ... I guessed this had to do with rounding errors, and I remembered something that Stefan Pendl posted some time ago. I tried it and it seems to work, but the
    Message 1 of 6 , Nov 3, 2011
    • 0 Attachment
      --- In libertybasic@yahoogroups.com, "paul.r.gorton@..." <paul.r.gorton@...> wrote:
      >
      > Good Morning,
      >
      > I have a program in which the user is asked to input a number between 1.01 and 9.99. To check whether they have used extra decimals, I use the following:-
      >
      > print #main.ratio, "!contents? RAT";
      > if int(RAT * 100) <> (RAT * 100) then print #main.ratio,"Ratio Error": wait
      >
      > This works fine most of the time but certain 'correct' values produce an error. For example 8.7 where int(8.7 * 100) produces 869!
      >
      > Is there an easy way round this?
      >
      > Regards, Paul
      >
      > PS. This will make a change from all the talk about LBB
      >

      I guessed this had to do with rounding errors, and I remembered something that Stefan Pendl posted some time ago. I tried it and it seems to work, but the other methods might be simpler.

      Here is the rounding function:

      number=8.7
      print Round(number*100)
      end

      function Round(number)
      ' returns a rounded number
      Round = int( number+((number>0)-(number<0))/2 )
      end function
    • Stefan Pendl
      ... I haven t posted this function, so I can t receive credit for it. ... This is known as the floating point error and is the result of not all numbers can be
      Message 2 of 6 , Nov 3, 2011
      • 0 Attachment
        >
        > I guessed this had to do with rounding errors, and I
        > remembered something that Stefan Pendl posted some time ago.
        > I tried it and it seems to work, but the other methods might
        > be simpler.
        >

        I haven't posted this function, so I can't receive credit for it.

        ----

        This is known as the floating point error and is the result of not all numbers can be represented in binary format.

        The best way to avoid this problem is to change how you create the whole number from the real number.

        1) retrieve a string from the text box
        2) split that string by the period
        3) check the length of the second string


        '---code start (watch for line wraps)

        num$(1) = "0.01"
        num$(2) = "8.7"
        num$(3) = "9.876"
        num$(4) = "1"
        num$(5) = "10.4"
        num$(6) = "11.34"

        for i = 1 to 6
        Print num$(i);

        select case CheckNumber(num$(i))
        case -1
        print " has too many leading numbers"

        case 0
        print " is valid"

        case 1
        print " has too many trailing numbers"
        end select
        next
        print "finished..."
        end

        function CheckNumber(NumberAsString$)
        LeadingNumbers$ = word$(NumberAsString$, 1, ".")
        TrailingNumbers$ = word$(NumberAsString$, 2, ".")

        select case
        case len(LeadingNumbers$) > 1
        CheckNumber = -1

        case len(TrailingNumbers$) > 2
        CheckNumber = 1

        case else
        CheckNumber = 0
        end select
        end function

        '---code end

        ---
        Stefan Pendl
        http://stefanpendl.runbasichosting.com/

        Liberty BASIC 4.04 Pro ... http://www.libertybasic.com/assist.html
        Liberty BASIC 4.04 ....... http://www.libertybasic.com/lb404setup.exe

        Liberty BASIC Bug Tracker ... http://libertybasicbugs.wikispaces.com/

        Books at http://www.lulu.com/ and http://www.amazon.com/
        Alyce Watson ... APIs for Liberty BASIC
        Carl Gundel .... Beginning Programming with Liberty BASIC

        Windows 7 Home Premium 64-bit SP1
        AMD Turion X2 RM-70 2GHz, 4GB RAM
      • paradisejb
        ... My apologies. You are absolutely right, Stefan. That function was posted by cundo in the JB Building Blocks page for INT. Jack
        Message 3 of 6 , Nov 3, 2011
        • 0 Attachment
          --- In libertybasic@yahoogroups.com, "Stefan Pendl" <pendl2megabit@...> wrote:
          >
          > >
          > > I guessed this had to do with rounding errors, and I
          > > remembered something that Stefan Pendl posted some time ago.
          > > I tried it and it seems to work, but the other methods might
          > > be simpler.
          > >
          >
          > I haven't posted this function, so I can't receive credit for it.
          >
          > ----


          My apologies. You are absolutely right, Stefan. That function was posted by cundo in the JB Building Blocks page for INT.

          Jack
        Your message has been successfully submitted and would be delivered to recipients shortly.