Sorry, an error occurred while loading the content.

## A curly NT maths question

Expand Messages
• ^!Set %_N%=14608138974422630400 ^!Set %_T%=^\$Calc(ABS(^%_N%))\$ ^!Info ABS(_T)=^%_T% ^!If ^%_T%=0 NEXT ELSE SKIP_2 ^!InsertText Something funny here!! ^!Goto
Message 1 of 31 , Dec 9, 2010
• 0 Attachment
^!Set %_N%=14608138974422630400
^!Set %_T%=^\$Calc(ABS(^%_N%))\$
^!Info ABS(_T)=^%_T%
^!If ^%_T%=0 NEXT ELSE SKIP_2
^!InsertText Something funny here!!
^!Goto End
^!InsertText All Okay!

Run the above code. I know _N is a big number; that's beside the point. Can anyone tell me *why* _T reports as 'zero' or at least why the test for equality asserts true?

btw it is not important that the ABS function will have rounding errors beyond about 15 digits. I want to reliably test if _N is zero or not. So now I'm using \$Calc(Zero())\$ however this bug cost me an hour an a half to trace!! (um.. *is* it a bug??)

Further, this code runs quite well for numbers up to this *magic* number. I think the only point of distinction (because no I did not sequentially test numbers up to this one - good grief!) is that it is one digit longer than a number that passes the test for equality correctly. The error continues to appear with numbers larger than the one I've set _N to in the code above. Further still, the number assigned to _T appears to be valid, aside from the accuracy of the trailing digits. Yet it fails as a parameter in an If statement???

If this is a simple limitation of the maths engine, or a deliberate implementation intention, that's no problem but perhaps we could get the help file to note the details.

Hope that makes sense.
Paul
• ... I agree, Lotta. ... With regard to a scripting language with a math engine as an add-on, it s fair enough to not assume things work the way they do in
Message 31 of 31 , Dec 18, 2010
• 0 Attachment
lotta wrote:
> Not really. Should one also assume it's natural and expected for
> this to change without notice between two minor versions of the
> program, away from how it probably has been since comparison
> operators were introduced? Don't think so.

I agree, Lotta.

> Paul wrote:
> >Axel Berger wrote:
> > > One ought not to have to, but it helps not to assume too much.
> >
> >Fair call!

With regard to a scripting language with a math engine as an add-on, it's fair enough to not assume things work the way they do in other languages.

<Ahem> BUG!!

Bug: It's like a mozzie bite between the shoulder blades: it itches like fire but do you think you can get near it to scratch it?

Again, "Hello Software Support???" Send a back-scratcher pronto!

Paul
Your message has been successfully submitted and would be delivered to recipients shortly.