Loading ...
Sorry, an error occurred while loading the content.

Re: ploticus numerical instability

Expand Messages
  • Stephen C. Grubb
    Hi Phil, thanks for pointing this out.... and supplying a code fix... should be able to include it in the next release. -Steve ... Stephen C. Grubb
    Message 1 of 1 , Sep 3, 2008
    • 0 Attachment
      Hi Phil,

      thanks for pointing this out.... and supplying a code fix... should be
      able to include it in the next release.

      -Steve

      On Thu, 28 Aug 2008, Steve Grubb wrote:

      > ---------- Forwarded message ----------
      > From: Phil Carmody <thefatphil@...>
      > Date: Fri, Aug 22, 2008 at 11:53 AM
      > Subject: ploticus numerical instability
      > To: stevegrubb@...
      >
      >
      > I've been trying to plot a heatmap based on data that's exact multiples of
      > 0.1 into buckets of granularity 0.1, and it seems that there's horrible
      > rounding instabilities in GL_Numgroup. Sometimes fmod would return 0.1, and
      > the bucket selected would be wrong.
      >
      > Some inputs, and outputs:
      > 0.0 0.1
      > 0.000000->0.050000
      > 0.100000->0.150000
      > 0.2 0.3
      > 0.200000->0.250000
      > 0.300000->0.250000
      > 0.4 0.5
      > 0.400000->0.450000
      > 0.500000->0.450000
      > 0.6 0.7
      > 0.600000->0.550000
      > 0.700000->0.650000
      > 0.8 0.9
      > 0.800000->0.850000
      > 0.900000->0.850000
      > 1.0 1.1
      > 1.000000->0.950000
      > 1.100000->1.150000
      >
      > as you can see 0.35 and 0.85 (and 1.35...) are never attained.
      >
      >
      > I can't say I really understand what 'mode' is supposed to be, but the
      > following works far better in 'mid' mode, which is the only mode that I
      > could see actually in use in the code. It basically dicks around with
      > rounding, and all of the above numbers get offset in the same direction.
      >
      >
      >
      > -- 8< ---
      > /* ========================= */
      > /* NUMGROUP - convert val to a nearby multiple of h, taking mode (low, mid,
      > high) into account */
      >
      > double
      > GL_numgroup( val, h, mode )
      > double val, h;
      > char *mode;
      > {
      > double fmod(), ofs, modf;
      > ofs = 0.0;
      > if( mode[0] == 'm' ) ofs = h / 2.0;
      > else if( mode[0] == 'h' ) ofs = h;
      > #if defined(UNSTABLE_GROUPING)
      > modf = fmod( val, h );
      > return( (val - modf) + ofs );
      > #else
      > modf = h*(int)(val/h+0.5);
      > return(modf+ofs);
      > #endif
      > }
      > -- 8< ---
      >
      >
      > I hope that this idea is useful to you, as I'd very much like to use
      > ploticus for some of my website graphics, as it seems vastly more suitable
      > than anything else out there.
      >
      > Cheers,
      > Phil
      >
      > --
      > () ASCII ribbon campaign () Hopeless ribbon campaign
      > /\ against HTML mail /\ against gratuitous bloodshed
      >
      > [stolen with permission from Daniel B. Cristofani]
      >


      Stephen C. Grubb scg@... x-6633
      Scientific Software Engineer, The Jackson Laboratory
      600 Main Street Bar Harbor, Maine 04609 USA
    Your message has been successfully submitted and would be delivered to recipients shortly.