Loading ...
Sorry, an error occurred while loading the content.

Re: INT problem

Expand Messages
  • Rod
    You might try using() isnce an error will always create a higher value. rat=8.7 print val(using( #.## ,rat)) if val(using( #.## ,rat))
    Message 1 of 6 , Nov 3, 2011
    • 0 Attachment
      You might try using() isnce an error will always create a higher value.

      rat=8.7
      print val(using("#.##",rat))
      if val(using("#.##",rat)) < rat then
      print "Ratio Error"
      else
      print "Value OK"
      end if


      --- In libertybasic@yahoogroups.com, "paul.r.gorton@..." <paul.r.gorton@...> wrote:
      >
      > Good Morning,
      >
      > I have a program in which the user is asked to input a number between 1.01 and 9.99. To check whether they have used extra decimals, I use the following:-
      >
      > print #main.ratio, "!contents? RAT";
      > if int(RAT * 100) <> (RAT * 100) then print #main.ratio,"Ratio Error": wait
      >
      > This works fine most of the time but certain 'correct' values produce an error. For example 8.7 where int(8.7 * 100) produces 869!
      >
      > Is there an easy way round this?
      >
      > Regards, Paul
      >
      > PS. This will make a change from all the talk about LBB
      >
    • paradisejb
      ... I guessed this had to do with rounding errors, and I remembered something that Stefan Pendl posted some time ago. I tried it and it seems to work, but the
      Message 2 of 6 , Nov 3, 2011
      • 0 Attachment
        --- In libertybasic@yahoogroups.com, "paul.r.gorton@..." <paul.r.gorton@...> wrote:
        >
        > Good Morning,
        >
        > I have a program in which the user is asked to input a number between 1.01 and 9.99. To check whether they have used extra decimals, I use the following:-
        >
        > print #main.ratio, "!contents? RAT";
        > if int(RAT * 100) <> (RAT * 100) then print #main.ratio,"Ratio Error": wait
        >
        > This works fine most of the time but certain 'correct' values produce an error. For example 8.7 where int(8.7 * 100) produces 869!
        >
        > Is there an easy way round this?
        >
        > Regards, Paul
        >
        > PS. This will make a change from all the talk about LBB
        >

        I guessed this had to do with rounding errors, and I remembered something that Stefan Pendl posted some time ago. I tried it and it seems to work, but the other methods might be simpler.

        Here is the rounding function:

        number=8.7
        print Round(number*100)
        end

        function Round(number)
        ' returns a rounded number
        Round = int( number+((number>0)-(number<0))/2 )
        end function
      • Stefan Pendl
        ... I haven t posted this function, so I can t receive credit for it. ... This is known as the floating point error and is the result of not all numbers can be
        Message 3 of 6 , Nov 3, 2011
        • 0 Attachment
          >
          > I guessed this had to do with rounding errors, and I
          > remembered something that Stefan Pendl posted some time ago.
          > I tried it and it seems to work, but the other methods might
          > be simpler.
          >

          I haven't posted this function, so I can't receive credit for it.

          ----

          This is known as the floating point error and is the result of not all numbers can be represented in binary format.

          The best way to avoid this problem is to change how you create the whole number from the real number.

          1) retrieve a string from the text box
          2) split that string by the period
          3) check the length of the second string


          '---code start (watch for line wraps)

          num$(1) = "0.01"
          num$(2) = "8.7"
          num$(3) = "9.876"
          num$(4) = "1"
          num$(5) = "10.4"
          num$(6) = "11.34"

          for i = 1 to 6
          Print num$(i);

          select case CheckNumber(num$(i))
          case -1
          print " has too many leading numbers"

          case 0
          print " is valid"

          case 1
          print " has too many trailing numbers"
          end select
          next
          print "finished..."
          end

          function CheckNumber(NumberAsString$)
          LeadingNumbers$ = word$(NumberAsString$, 1, ".")
          TrailingNumbers$ = word$(NumberAsString$, 2, ".")

          select case
          case len(LeadingNumbers$) > 1
          CheckNumber = -1

          case len(TrailingNumbers$) > 2
          CheckNumber = 1

          case else
          CheckNumber = 0
          end select
          end function

          '---code end

          ---
          Stefan Pendl
          http://stefanpendl.runbasichosting.com/

          Liberty BASIC 4.04 Pro ... http://www.libertybasic.com/assist.html
          Liberty BASIC 4.04 ....... http://www.libertybasic.com/lb404setup.exe

          Liberty BASIC Bug Tracker ... http://libertybasicbugs.wikispaces.com/

          Books at http://www.lulu.com/ and http://www.amazon.com/
          Alyce Watson ... APIs for Liberty BASIC
          Carl Gundel .... Beginning Programming with Liberty BASIC

          Windows 7 Home Premium 64-bit SP1
          AMD Turion X2 RM-70 2GHz, 4GB RAM
        • paradisejb
          ... My apologies. You are absolutely right, Stefan. That function was posted by cundo in the JB Building Blocks page for INT. Jack
          Message 4 of 6 , Nov 3, 2011
          • 0 Attachment
            --- In libertybasic@yahoogroups.com, "Stefan Pendl" <pendl2megabit@...> wrote:
            >
            > >
            > > I guessed this had to do with rounding errors, and I
            > > remembered something that Stefan Pendl posted some time ago.
            > > I tried it and it seems to work, but the other methods might
            > > be simpler.
            > >
            >
            > I haven't posted this function, so I can't receive credit for it.
            >
            > ----


            My apologies. You are absolutely right, Stefan. That function was posted by cundo in the JB Building Blocks page for INT.

            Jack
          Your message has been successfully submitted and would be delivered to recipients shortly.