Re: INT problem
- --- In email@example.com, "paul.r.gorton@..." <paul.r.gorton@...> wrote:
>I guessed this had to do with rounding errors, and I remembered something that Stefan Pendl posted some time ago. I tried it and it seems to work, but the other methods might be simpler.
> Good Morning,
> I have a program in which the user is asked to input a number between 1.01 and 9.99. To check whether they have used extra decimals, I use the following:-
> print #main.ratio, "!contents? RAT";
> if int(RAT * 100) <> (RAT * 100) then print #main.ratio,"Ratio Error": wait
> This works fine most of the time but certain 'correct' values produce an error. For example 8.7 where int(8.7 * 100) produces 869!
> Is there an easy way round this?
> Regards, Paul
> PS. This will make a change from all the talk about LBB
Here is the rounding function:
' returns a rounded number
Round = int( number+((number>0)-(number<0))/2 )
>I haven't posted this function, so I can't receive credit for it.
> I guessed this had to do with rounding errors, and I
> remembered something that Stefan Pendl posted some time ago.
> I tried it and it seems to work, but the other methods might
> be simpler.
This is known as the floating point error and is the result of not all numbers can be represented in binary format.
The best way to avoid this problem is to change how you create the whole number from the real number.
1) retrieve a string from the text box
2) split that string by the period
3) check the length of the second string
'---code start (watch for line wraps)
num$(1) = "0.01"
num$(2) = "8.7"
num$(3) = "9.876"
num$(4) = "1"
num$(5) = "10.4"
num$(6) = "11.34"
for i = 1 to 6
select case CheckNumber(num$(i))
print " has too many leading numbers"
print " is valid"
print " has too many trailing numbers"
LeadingNumbers$ = word$(NumberAsString$, 1, ".")
TrailingNumbers$ = word$(NumberAsString$, 2, ".")
case len(LeadingNumbers$) > 1
CheckNumber = -1
case len(TrailingNumbers$) > 2
CheckNumber = 1
CheckNumber = 0
Liberty BASIC 4.04 Pro ... http://www.libertybasic.com/assist.html
Liberty BASIC 4.04 ....... http://www.libertybasic.com/lb404setup.exe
Liberty BASIC Bug Tracker ... http://libertybasicbugs.wikispaces.com/
Books at http://www.lulu.com/ and http://www.amazon.com/
Alyce Watson ... APIs for Liberty BASIC
Carl Gundel .... Beginning Programming with Liberty BASIC
Windows 7 Home Premium 64-bit SP1
AMD Turion X2 RM-70 2GHz, 4GB RAM
- --- In firstname.lastname@example.org, "Stefan Pendl" <pendl2megabit@...> wrote:
>My apologies. You are absolutely right, Stefan. That function was posted by cundo in the JB Building Blocks page for INT.
> > I guessed this had to do with rounding errors, and I
> > remembered something that Stefan Pendl posted some time ago.
> > I tried it and it seems to work, but the other methods might
> > be simpler.
> I haven't posted this function, so I can't receive credit for it.