## Re: [PBML] Perl floating point addition oddness

Expand Messages
• ... Not trolling. I m trying to convert node latitude/longitudes in openstreetmap.org into 63-bit INTs for sqlite3 rowid number. I m on a 32-bit machine, so %u
Message 1 of 4 , Jun 3, 2009
On 6/3/09, Randal L. Schwartz <merlyn@...> wrote:
>>>>>> "Kelly" == Kelly Jones <kelly.terry.jones@...> writes:
>
> Kelly> perl -le 'printf("%f %f %f\n", 4294967295, 2147483647*2**32,
> Kelly> 2147483647*2**32+4294967295)'
>
> Kelly> 4294967295.000000 9223372032559808512.000000
> 9223372036854775808.000000
>
> Kelly> Why? The answer is really 9223372036854775807 (one number lower), and
> Kelly> it's obvious that adding 2 and 5 in the units column should yield a 7
> Kelly> in the sum's unit column.
>
> Kelly> Roundoff error? Bug? How do I work around it?
>
> If you're not just trolling (since we just talked about floating
> point being approximate in this mailing list):
>
> use Math::BigInt;
> \$a = Math::BigInt->new('4294967295');
> \$b = Math::BigInt->new('2147483647');
>
> print "\$_\n" for \$a, \$b*2**32, \$b*2**32+\$a;
>
> ==>
>
> 4294967295
> 9223372032559808512
> 9223372036854775807
>
> And show me a C compiler that can do that correctly.

Not trolling. I'm trying to convert node latitude/longitudes in
openstreetmap.org into 63-bit INTs for sqlite3 rowid number.

I'm on a 32-bit machine, so %u doesn't help.

After interleaving, I end up w/ \$str, a 64-character long string of 0s
and 1s (the first character is always 0, so it's really a 63-bit INT)

I tried 'unpack("N",pack("B64",\$str))' but that failed.

So I ended up doing:

\$low = substr(\$str,32,32);
\$high = substr(\$str,0,32);
\$nlow = unpack("N",pack("B32",\$low));
\$nhigh = unpack("N",pack("B32",\$high));

and then calculating \$nlow+\$nhigh*2**32

I can use BigInt on \$nlow and \$nhigh as you describe (in fact, I
but is there a better approach overall?

--
We're just a Bunch Of Regular Guys, a collective group that's trying
to understand and assimilate technology. We feel that resistance to
new ideas and technology is unwise and ultimately futile.
• ... Kelly Not trolling. I m trying to convert node latitude/longitudes in Kelly openstreetmap.org into 63-bit INTs for sqlite3 rowid number. The problem is
Message 2 of 4 , Jun 3, 2009
>>>>> "Kelly" == Kelly Jones <kelly.terry.jones@...> writes:

Kelly> Not trolling. I'm trying to convert node latitude/longitudes in
Kelly> openstreetmap.org into 63-bit INTs for sqlite3 rowid number.

The problem is that Perl uses doubles inside for everything, including
integers, and the integer part of the floating point number is somewhat
limited, as you see (I think it's only 56 bits). That's why Math::BigInt is
needed here.

--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@...> <URL:http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion
Your message has been successfully submitted and would be delivered to recipients shortly.