Loading ...
Sorry, an error occurred while loading the content.

A curly NT maths question

Expand Messages
  • Paul
    ^!Set %_N%=14608138974422630400 ^!Set %_T%=^$Calc(ABS(^%_N%))$ ^!Info ABS(_T)=^%_T% ^!If ^%_T%=0 NEXT ELSE SKIP_2 ^!InsertText Something funny here!! ^!Goto
    Message 1 of 31 , Dec 9, 2010
    • 0 Attachment
      ^!Set %_N%=14608138974422630400
      ^!Set %_T%=^$Calc(ABS(^%_N%))$
      ^!Info ABS(_T)=^%_T%
      ^!If ^%_T%=0 NEXT ELSE SKIP_2
      ^!InsertText Something funny here!!
      ^!Goto End
      ^!InsertText All Okay!

      Run the above code. I know _N is a big number; that's beside the point. Can anyone tell me *why* _T reports as 'zero' or at least why the test for equality asserts true?

      btw it is not important that the ABS function will have rounding errors beyond about 15 digits. I want to reliably test if _N is zero or not. So now I'm using $Calc(Zero())$ however this bug cost me an hour an a half to trace!! (um.. *is* it a bug??)

      Further, this code runs quite well for numbers up to this *magic* number. I think the only point of distinction (because no I did not sequentially test numbers up to this one - good grief!) is that it is one digit longer than a number that passes the test for equality correctly. The error continues to appear with numbers larger than the one I've set _N to in the code above. Further still, the number assigned to _T appears to be valid, aside from the accuracy of the trailing digits. Yet it fails as a parameter in an If statement???

      If this is a simple limitation of the maths engine, or a deliberate implementation intention, that's no problem but perhaps we could get the help file to note the details.

      Hope that makes sense.
      Paul
    • Paul
      ... I agree, Lotta. ... With regard to a scripting language with a math engine as an add-on, it s fair enough to not assume things work the way they do in
      Message 31 of 31 , Dec 18, 2010
      • 0 Attachment
        lotta wrote:
        > Not really. Should one also assume it's natural and expected for
        > this to change without notice between two minor versions of the
        > program, away from how it probably has been since comparison
        > operators were introduced? Don't think so.

        I agree, Lotta.

        > Paul wrote:
        > >Axel Berger wrote:
        > > > One ought not to have to, but it helps not to assume too much.
        > >
        > >Fair call!

        With regard to a scripting language with a math engine as an add-on, it's fair enough to not assume things work the way they do in other languages.

        <Ahem> BUG!!

        Bug: It's like a mozzie bite between the shoulder blades: it itches like fire but do you think you can get near it to scratch it?

        Again, "Hello Software Support???" Send a back-scratcher pronto!

        Paul
      Your message has been successfully submitted and would be delivered to recipients shortly.