## Re: INT problem

Expand Messages
• You might try using() isnce an error will always create a higher value. rat=8.7 print val(using( #.## ,rat)) if val(using( #.## ,rat))
Message 1 of 6 , Nov 3, 2011
• 0 Attachment
You might try using() isnce an error will always create a higher value.

rat=8.7
print val(using("#.##",rat))
if val(using("#.##",rat)) < rat then
print "Ratio Error"
else
print "Value OK"
end if

--- In libertybasic@yahoogroups.com, "paul.r.gorton@..." <paul.r.gorton@...> wrote:
>
> Good Morning,
>
> I have a program in which the user is asked to input a number between 1.01 and 9.99. To check whether they have used extra decimals, I use the following:-
>
> print #main.ratio, "!contents? RAT";
> if int(RAT * 100) <> (RAT * 100) then print #main.ratio,"Ratio Error": wait
>
> This works fine most of the time but certain 'correct' values produce an error. For example 8.7 where int(8.7 * 100) produces 869!
>
> Is there an easy way round this?
>
> Regards, Paul
>
> PS. This will make a change from all the talk about LBB
>
• ... I guessed this had to do with rounding errors, and I remembered something that Stefan Pendl posted some time ago. I tried it and it seems to work, but the
Message 2 of 6 , Nov 3, 2011
• 0 Attachment
--- In libertybasic@yahoogroups.com, "paul.r.gorton@..." <paul.r.gorton@...> wrote:
>
> Good Morning,
>
> I have a program in which the user is asked to input a number between 1.01 and 9.99. To check whether they have used extra decimals, I use the following:-
>
> print #main.ratio, "!contents? RAT";
> if int(RAT * 100) <> (RAT * 100) then print #main.ratio,"Ratio Error": wait
>
> This works fine most of the time but certain 'correct' values produce an error. For example 8.7 where int(8.7 * 100) produces 869!
>
> Is there an easy way round this?
>
> Regards, Paul
>
> PS. This will make a change from all the talk about LBB
>

I guessed this had to do with rounding errors, and I remembered something that Stefan Pendl posted some time ago. I tried it and it seems to work, but the other methods might be simpler.

Here is the rounding function:

number=8.7
print Round(number*100)
end

function Round(number)
' returns a rounded number
Round = int( number+((number>0)-(number<0))/2 )
end function
• ... I haven t posted this function, so I can t receive credit for it. ... This is known as the floating point error and is the result of not all numbers can be
Message 3 of 6 , Nov 3, 2011
• 0 Attachment
>
> I guessed this had to do with rounding errors, and I
> remembered something that Stefan Pendl posted some time ago.
> I tried it and it seems to work, but the other methods might
> be simpler.
>

I haven't posted this function, so I can't receive credit for it.

----

This is known as the floating point error and is the result of not all numbers can be represented in binary format.

The best way to avoid this problem is to change how you create the whole number from the real number.

1) retrieve a string from the text box
2) split that string by the period
3) check the length of the second string

'---code start (watch for line wraps)

num\$(1) = "0.01"
num\$(2) = "8.7"
num\$(3) = "9.876"
num\$(4) = "1"
num\$(5) = "10.4"
num\$(6) = "11.34"

for i = 1 to 6
Print num\$(i);

select case CheckNumber(num\$(i))
case -1
print " has too many leading numbers"

case 0
print " is valid"

case 1
print " has too many trailing numbers"
end select
next
print "finished..."
end

function CheckNumber(NumberAsString\$)
TrailingNumbers\$ = word\$(NumberAsString\$, 2, ".")

select case
CheckNumber = -1

case len(TrailingNumbers\$) > 2
CheckNumber = 1

case else
CheckNumber = 0
end select
end function

'---code end

---
Stefan Pendl
http://stefanpendl.runbasichosting.com/

Liberty BASIC 4.04 Pro ... http://www.libertybasic.com/assist.html
Liberty BASIC 4.04 ....... http://www.libertybasic.com/lb404setup.exe

Liberty BASIC Bug Tracker ... http://libertybasicbugs.wikispaces.com/

Books at http://www.lulu.com/ and http://www.amazon.com/
Alyce Watson ... APIs for Liberty BASIC
Carl Gundel .... Beginning Programming with Liberty BASIC

Windows 7 Home Premium 64-bit SP1
AMD Turion X2 RM-70 2GHz, 4GB RAM
• ... My apologies. You are absolutely right, Stefan. That function was posted by cundo in the JB Building Blocks page for INT. Jack
Message 4 of 6 , Nov 3, 2011
• 0 Attachment
--- In libertybasic@yahoogroups.com, "Stefan Pendl" <pendl2megabit@...> wrote:
>
> >
> > I guessed this had to do with rounding errors, and I
> > remembered something that Stefan Pendl posted some time ago.
> > I tried it and it seems to work, but the other methods might
> > be simpler.
> >
>
> I haven't posted this function, so I can't receive credit for it.
>
> ----

My apologies. You are absolutely right, Stefan. That function was posted by cundo in the JB Building Blocks page for INT.

Jack
Your message has been successfully submitted and would be delivered to recipients shortly.