Loading ...
Sorry, an error occurred while loading the content.

Re: [PBML] Perl floating point addition oddness

Expand Messages
  • Kelly Jones
    ... Not trolling. I m trying to convert node latitude/longitudes in openstreetmap.org into 63-bit INTs for sqlite3 rowid number. I m on a 32-bit machine, so %u
    Message 1 of 4 , Jun 3, 2009
    View Source
    • 0 Attachment
      On 6/3/09, Randal L. Schwartz <merlyn@...> wrote:
      >>>>>> "Kelly" == Kelly Jones <kelly.terry.jones@...> writes:
      >
      > Kelly> perl -le 'printf("%f %f %f\n", 4294967295, 2147483647*2**32,
      > Kelly> 2147483647*2**32+4294967295)'
      >
      > Kelly> 4294967295.000000 9223372032559808512.000000
      > 9223372036854775808.000000
      >
      > Kelly> Why? The answer is really 9223372036854775807 (one number lower), and
      > Kelly> it's obvious that adding 2 and 5 in the units column should yield a 7
      > Kelly> in the sum's unit column.
      >
      > Kelly> Roundoff error? Bug? How do I work around it?
      >
      > If you're not just trolling (since we just talked about floating
      > point being approximate in this mailing list):
      >
      > use Math::BigInt;
      > $a = Math::BigInt->new('4294967295');
      > $b = Math::BigInt->new('2147483647');
      >
      > print "$_\n" for $a, $b*2**32, $b*2**32+$a;
      >
      > ==>
      >
      > 4294967295
      > 9223372032559808512
      > 9223372036854775807
      >
      > And show me a C compiler that can do that correctly.

      Not trolling. I'm trying to convert node latitude/longitudes in
      openstreetmap.org into 63-bit INTs for sqlite3 rowid number.

      I'm on a 32-bit machine, so %u doesn't help.

      After interleaving, I end up w/ $str, a 64-character long string of 0s
      and 1s (the first character is always 0, so it's really a 63-bit INT)

      I tried 'unpack("N",pack("B64",$str))' but that failed.

      So I ended up doing:

      $low = substr($str,32,32);
      $high = substr($str,0,32);
      $nlow = unpack("N",pack("B32",$low));
      $nhigh = unpack("N",pack("B32",$high));

      and then calculating $nlow+$nhigh*2**32

      I can use BigInt on $nlow and $nhigh as you describe (in fact, I
      googled "perl unlimited precision" and was about to do this anyway),
      but is there a better approach overall?

      --
      We're just a Bunch Of Regular Guys, a collective group that's trying
      to understand and assimilate technology. We feel that resistance to
      new ideas and technology is unwise and ultimately futile.
    • merlyn@stonehenge.com
      ... Kelly Not trolling. I m trying to convert node latitude/longitudes in Kelly openstreetmap.org into 63-bit INTs for sqlite3 rowid number. The problem is
      Message 2 of 4 , Jun 3, 2009
      View Source
      • 0 Attachment
        >>>>> "Kelly" == Kelly Jones <kelly.terry.jones@...> writes:

        Kelly> Not trolling. I'm trying to convert node latitude/longitudes in
        Kelly> openstreetmap.org into 63-bit INTs for sqlite3 rowid number.

        The problem is that Perl uses doubles inside for everything, including
        integers, and the integer part of the floating point number is somewhat
        limited, as you see (I think it's only 56 bits). That's why Math::BigInt is
        needed here.

        --
        Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
        <merlyn@...> <URL:http://www.stonehenge.com/merlyn/>
        Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
        See http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion
      Your message has been successfully submitted and would be delivered to recipients shortly.