## rounding issue

Expand Messages
• can anybody explain this one, i ve found another work around for now... code: x\$ = 574.42 print val( x\$ ) print val( x\$ ) * 100 print int( val( x\$ ) * 100 )
Message 1 of 6 , Jun 30, 2008
can anybody explain this one, i've found another work around for now...

code:

x\$ = "574.42"
print val( x\$ )
print val( x\$ ) * 100
print int( val( x\$ ) * 100 )

results:

574.42
57442.0
57441
• It gets even more interesting.. code: y=574.42 print y y=y*100 print y print int(y) print y print print y=y*10 print y print int(y) result: 574.42
Message 2 of 6 , Jun 30, 2008
It gets even more interesting..

code:

y=574.42
print y
y=y*100
print y
print int(y)
print y
print ""
print ""
y=y*10
print y
print int(y)

result:

574.42
57442.0
57441
57442.0

574420.0
574419

May be a small bug in the int function

--- In libertybasic@yahoogroups.com, "dlsmith67" <david.smith@...>
wrote:
>
> can anybody explain this one, i've found another work around for
now...
>
> code:
>
> x\$ = "574.42"
> print val( x\$ )
> print val( x\$ ) * 100
> print int( val( x\$ ) * 100 )
>
> results:
>
> 574.42
> 57442.0
> 57441
>
• ... This is a Frequently Asked Question! It is not a bug, or in any way strange, it just results from the fact that Liberty Basic (like the vast majority of
Message 3 of 6 , Jun 30, 2008
--- In libertybasic@yahoogroups.com, "gjbilik" <gjbilik@...> wrote:
> May be a small bug in the int function

This is a Frequently Asked Question!

It is not a bug, or in any way strange, it just results from the
fact that Liberty Basic (like the vast majority of programming
languages) works internally in BINARY, not DECIMAL.

Here is what happens internally. You enter the number 574.42 which,
in order to be meaningful to LB, has to be converted to binary.
Like the majority of non-integer decimal numbers, 574.42 *cannot be
represented exactly in binary* so LB converts it to the *closest*
binary equivalent, which happens to be this:

1000111110.0110101110000101000111101011100001010001111

Now you multiply this number by 100 (binary 1100100) giving this:

1110000001100001.1111111111111111111111111111111111111

Finally you take the integer part (INT) of this number:

1110000001100001

which is 57441 (the result LB prints).

So this behavior is to be expected, and all languages which work
internally in binary will produce the same result. A few
programming languages work internally in BCD (binary-coded decimal)
and don't suffer from this effect, but they tend to run slower
because it's harder for the processor to perform BCD calculations
than binary ones.

Richard.
• Thanks, Richard ... which,
Message 4 of 6 , Jun 30, 2008
Thanks, Richard

--- In libertybasic@yahoogroups.com, "Richard Russell" <yahoo@...>
wrote:
>
> --- In libertybasic@yahoogroups.com, "gjbilik" <gjbilik@> wrote:
> > May be a small bug in the int function
>
> This is a Frequently Asked Question!
>
> It is not a bug, or in any way strange, it just results from the
> fact that Liberty Basic (like the vast majority of programming
> languages) works internally in BINARY, not DECIMAL.
>
> Here is what happens internally. You enter the number 574.42
which,
> in order to be meaningful to LB, has to be converted to binary.
> Like the majority of non-integer decimal numbers, 574.42 *cannot be
> represented exactly in binary* so LB converts it to the *closest*
> binary equivalent, which happens to be this:
>
> 1000111110.0110101110000101000111101011100001010001111
>
> Now you multiply this number by 100 (binary 1100100) giving this:
>
> 1110000001100001.1111111111111111111111111111111111111
>
> Finally you take the integer part (INT) of this number:
>
> 1110000001100001
>
> which is 57441 (the result LB prints).
>
> So this behavior is to be expected, and all languages which work
> internally in binary will produce the same result. A few
> programming languages work internally in BCD (binary-coded decimal)
> and don't suffer from this effect, but they tend to run slower
> because it's harder for the processor to perform BCD calculations
> than binary ones.
>
> Richard.
>
• Richard has it right. In addition most programming systems don t actually do floating point math themselves, but they use the computer s built in math
Message 5 of 6 , Jun 30, 2008
Richard has it right. In addition most programming systems don't
actually do floating point math themselves, but they use the
computer's built in math coprocessor. If you did find a floating
point bug it would usually be in the coprocessor.

-Carl Gundel
Liberty BASIC for Windows - http://www.libertybasic.com
Run BASIC, easy web programming - http://www.runbasic.com

On Jun 30, 2008, at 12:30 PM, "Richard Russell"
<yahoo@...> wrote:

> --- In libertybasic@yahoogroups.com, "gjbilik" <gjbilik@...> wrote:
>> May be a small bug in the int function
>
> This is a Frequently Asked Question!
>
> It is not a bug, or in any way strange, it just results from the
> fact that Liberty Basic (like the vast majority of programming
> languages) works internally in BINARY, not DECIMAL.
>
> Here is what happens internally. You enter the number 574.42 which,
> in order to be meaningful to LB, has to be converted to binary.
> Like the majority of non-integer decimal numbers, 574.42 *cannot be
> represented exactly in binary* so LB converts it to the *closest*
> binary equivalent, which happens to be this:
>
> 1000111110.0110101110000101000111101011100001010001111
>
> Now you multiply this number by 100 (binary 1100100) giving this:
>
> 1110000001100001.1111111111111111111111111111111111111
>
> Finally you take the integer part (INT) of this number:
>
> 1110000001100001
>
> which is 57441 (the result LB prints).
>
> So this behavior is to be expected, and all languages which work
> internally in binary will produce the same result. A few
> programming languages work internally in BCD (binary-coded decimal)
> and don't suffer from this effect, but they tend to run slower
> because it's harder for the processor to perform BCD calculations
> than binary ones.
>
> Richard.
>
>
>
> ------------------------------------
>
>
>
>
• I agree. We had a rounding problem with a Basic program when we installed it on a new machine that had an AMD processor. We sent the machine back to the
Message 6 of 6 , Jul 1, 2008
I agree. We had a rounding problem with a Basic program when we
installed it on a new machine that had an AMD processor. We sent the
machine back to the manufacturer with the program and tried to explain
the problem to them.
They called and said the machine was ready to be picked up and when I
went I asked if they had tested the program. They said no, they were
hardware people. We set the machine up and tested the program and
found the same problem.
I got them to change to an Intel processor and the program worked
perfectly.
By the way I am using an Intel P4 and when I ran your code I got the

--- In libertybasic@yahoogroups.com, Carl Gundel <carlg@...> wrote:
>
> Richard has it right. In addition most programming systems don't
> actually do floating point math themselves, but they use the
> computer's built in math coprocessor. If you did find a floating
> point bug it would usually be in the coprocessor.
>
> -Carl Gundel
> Liberty BASIC for Windows - http://www.libertybasic.com
> Run BASIC, easy web programming - http://www.runbasic.com
>
> On Jun 30, 2008, at 12:30 PM, "Richard Russell"
> <yahoo@...> wrote:
>
> > --- In libertybasic@yahoogroups.com, "gjbilik" <gjbilik@> wrote:
> >> May be a small bug in the int function
> >
> > This is a Frequently Asked Question!
> >
> > It is not a bug, or in any way strange, it just results from the
> > fact that Liberty Basic (like the vast majority of programming
> > languages) works internally in BINARY, not DECIMAL.
> >
> > Here is what happens internally. You enter the number 574.42 which,
> > in order to be meaningful to LB, has to be converted to binary.
> > Like the majority of non-integer decimal numbers, 574.42 *cannot be
> > represented exactly in binary* so LB converts it to the *closest*
> > binary equivalent, which happens to be this:
> >
> > 1000111110.0110101110000101000111101011100001010001111
> >
> > Now you multiply this number by 100 (binary 1100100) giving this:
> >
> > 1110000001100001.1111111111111111111111111111111111111
> >
> > Finally you take the integer part (INT) of this number:
> >
> > 1110000001100001
> >
> > which is 57441 (the result LB prints).
> >
> > So this behavior is to be expected, and all languages which work
> > internally in binary will produce the same result. A few
> > programming languages work internally in BCD (binary-coded decimal)
> > and don't suffer from this effect, but they tend to run slower
> > because it's harder for the processor to perform BCD calculations
> > than binary ones.
> >
> > Richard.
> >
> >
> >
> > ------------------------------------
> >