Loading ...
Sorry, an error occurred while loading the content.

datetime vs time, axis vs plotted data

Expand Messages
  • Dave
    Hi, Been playing with Ploticus for about three days now, and I like it :) I have a problem, though; I m trying to plot tens of thousands of individual points
    Message 1 of 5 , May 22, 2005
    • 0 Attachment
      Hi,

      Been playing with Ploticus for about three days now, and I like it :)

      I have a problem, though; I'm trying to plot tens of thousands of individual
      points that are bogofilter email spamicity scores, and I'm running into a
      problem once I get past a certain range of times to plot. It may be easier
      to show you all than to explain in words:

      Here's a "good" plot:

      http://www.halibut.com/~dlc/tmp/31jan05-24may05.png

      And here's a "bad" plot with just three more days of data:

      http://www.halibut.com/~dlc/tmp/28Jan05-22May05.png

      In order to keep the second plot from failing alltogether (well,
      plotting 618 inches off of the bottom of the page) I had to change
      the x-axis scale from "datetime" to "date". But then the scatterplot
      lines up all points within each day on the same vertical line, and
      it destroys the ability for the eye to detect patterns.

      Of course when I get much more data than that, it turns useful again:

      http://www.halibut.com/~dlc/tmp/8month.png

      Is there something I'm missing about axis stubs or autoranging that
      would mitigate this effect?

      Thanks.
    • Stephen C. Grubb
      Hi, this could be a bug but I can t really tell unless I can reproduce the problem here. If you could send me your script, and the well-behaving data set, and
      Message 2 of 5 , May 23, 2005
      • 0 Attachment
        Hi, this could be a bug but I can't really tell unless I can reproduce the
        problem here. If you could send me your script, and the well-behaving
        data set, and the 3 additional days to append, I'll look into it. (you
        can just send a url links if you prefer)

        I gather that you got an extremely "tall" plot in the "bad" case.. was
        this the scatterplot or the bar graph?

        Also indicate your pl version and os platform please.

        Thanks,

        Steve


        On Sun, 22 May 2005, Dave wrote:

        >
        > Hi,
        >
        > Been playing with Ploticus for about three days now, and I like it :)
        >
        > I have a problem, though; I'm trying to plot tens of thousands of individual
        > points that are bogofilter email spamicity scores, and I'm running into a
        > problem once I get past a certain range of times to plot. It may be easier
        > to show you all than to explain in words:
        >
        > Here's a "good" plot:
        >
        > http://www.halibut.com/~dlc/tmp/31jan05-24may05.png
        >
        > And here's a "bad" plot with just three more days of data:
        >
        > http://www.halibut.com/~dlc/tmp/28Jan05-22May05.png
        >
        > In order to keep the second plot from failing alltogether (well,
        > plotting 618 inches off of the bottom of the page) I had to change
        > the x-axis scale from "datetime" to "date". But then the scatterplot
        > lines up all points within each day on the same vertical line, and
        > it destroys the ability for the eye to detect patterns.
        >
        > Of course when I get much more data than that, it turns useful again:
        >
        > http://www.halibut.com/~dlc/tmp/8month.png
        >
        > Is there something I'm missing about axis stubs or autoranging that
        > would mitigate this effect?
        >
        > Thanks.
        >
        >
        >
        >
        >
        >
        > Yahoo! Groups Links
        >
        >
        >
        >
        >
        >
        >


        Stephen C. Grubb
        Scientific Software Engineer, The Jackson Laboratory
        600 Main Street Bar Harbor, Maine 04609 USA
      • Dave
        OK, it will take me a little while to get the time.
        Message 3 of 5 , May 23, 2005
        • 0 Attachment
          OK, it will take me a little while to get the time.


          On Mon, May 23, 2005 at 02:07:20PM -0400, Stephen C. Grubb wrote:
          > Hi, this could be a bug but I can't really tell unless I can reproduce the
          > problem here. If you could send me your script, and the well-behaving
          > data set, and the 3 additional days to append, I'll look into it. (you
          > can just send a url links if you prefer)
          >
          > I gather that you got an extremely "tall" plot in the "bad" case.. was
          > this the scatterplot or the bar graph?
          >
          > Also indicate your pl version and os platform please.
          >
          > Thanks,
          >
          > Steve
          >
          >
          > On Sun, 22 May 2005, Dave wrote:
          >
          > >
          > > Hi,
          > >
          > > Been playing with Ploticus for about three days now, and I like it :)
          > >
          > > I have a problem, though; I'm trying to plot tens of thousands of individual
          > > points that are bogofilter email spamicity scores, and I'm running into a
          > > problem once I get past a certain range of times to plot. It may be easier
          > > to show you all than to explain in words:
          > >
          > > Here's a "good" plot:
          > >
          > > http://www.halibut.com/~dlc/tmp/31jan05-24may05.png
          > >
          > > And here's a "bad" plot with just three more days of data:
          > >
          > > http://www.halibut.com/~dlc/tmp/28Jan05-22May05.png
          > >
          > > In order to keep the second plot from failing alltogether (well,
          > > plotting 618 inches off of the bottom of the page) I had to change
          > > the x-axis scale from "datetime" to "date". But then the scatterplot
          > > lines up all points within each day on the same vertical line, and
          > > it destroys the ability for the eye to detect patterns.
          > >
          > > Of course when I get much more data than that, it turns useful again:
          > >
          > > http://www.halibut.com/~dlc/tmp/8month.png
          > >
          > > Is there something I'm missing about axis stubs or autoranging that
          > > would mitigate this effect?
          > >
          > > Thanks.
          > >
          > >
          > >
          > >
          > >
          > >
          > > Yahoo! Groups Links
          > >
          > >
          > >
          > >
          > >
          > >
          > >
          >
          >
          > Stephen C. Grubb
          > Scientific Software Engineer, The Jackson Laboratory
          > 600 Main Street Bar Harbor, Maine 04609 USA
          >
          >
          >
          >
          > Yahoo! Groups Links
          >
          >
          >
          >
          >
        • Dave
          So the issue appears to be that datematic stubs want to switch to month intervals, but since the xscaletype is datetime, I get the error message: pl proc axis:
          Message 4 of 5 , Jun 1, 2005
          • 0 Attachment
            So the issue appears to be that datematic stubs want to switch to month
            intervals, but since the xscaletype is datetime, I get the error message:

            pl proc axis: error 2476: month increment only valid with date scale type

            I'm seeing this now on another project; the script and dataset are here:

            http://www.halibut.com/~dlc/tmp/plotcsv.plo
            http://www.halibut.com/~dlc/tmp/plotcsv.data.gz (54kb)

            You can easily replicate my problem by manipulating the x range in my
            #proc xasis, for example by commenting out the xautorange and specifying a
            smaller range.

            The other issue is the appearance of the graph when the stub increments change;
            I would hope that the horizontal resolution would be able to scale smoothly, but
            you can see in the following two images that when I have to switch from datetime to
            date, my smooth curve turns into jumps between vertical bars, because for example in
            this case all 96 15-minute intervals in a single day end up being plotted with the
            same X value:

            http://www.halibut.com/~dlc/tmp/all.png
            vs
            http://www.halibut.com/~dlc/tmp/9-10apr.png

            ploticus 2.31

            FreeBSD kma.lab.netapp.com 5.4-RELEASE FreeBSD 5.4-RELEASE


            Thanks!


            > I gather that you got an extremely "tall" plot in the "bad" case.. was
            > this the scatterplot or the bar graph?
            >
            > Also indicate your pl version and os platform please.
            >
            > Thanks,
            >
            > Steve
            >
            >
          • Stephen C. Grubb
            Hi, The original notion was that it would be crazy to plot across 6 mos or a year+ with data having 1-minute granularity.. that the user should compute daily
            Message 5 of 5 , Jun 6, 2005
            • 0 Attachment
              Hi,

              The original notion was that it would be crazy to plot across 6 mos or a
              year+ with data having 1-minute granularity.. that the user should compute
              daily averages and plot those using date scaletype. Things have evolved
              and now I agree it would be nice for it to seamlessly handle anything you
              throw at it.. but currently it doesn't work this way.

              You should be able to get something to work by doing it this way:

              1. read in your data
              2. set up your plotting area, using xscaletype: date
              3. render your axes
              4. now, change the xscaletype to datetime , like this:

              #proc defineunits
              axis: x
              units: datetime yyyymmdd.hhmmss (or whatever)

              5. plot the curves



              On Wed, 1 Jun 2005, Dave wrote:

              >
              > So the issue appears to be that datematic stubs want to switch to month
              > intervals, but since the xscaletype is datetime, I get the error message:
              >
              > pl proc axis: error 2476: month increment only valid with date scale type
              >
              > I'm seeing this now on another project; the script and dataset are here:
              >
              > http://www.halibut.com/~dlc/tmp/plotcsv.plo
              > http://www.halibut.com/~dlc/tmp/plotcsv.data.gz (54kb)
              >
              > You can easily replicate my problem by manipulating the x range in my
              > #proc xasis, for example by commenting out the xautorange and specifying a
              > smaller range.
              >
              > The other issue is the appearance of the graph when the stub increments change;
              > I would hope that the horizontal resolution would be able to scale smoothly, but
              > you can see in the following two images that when I have to switch from datetime to
              > date, my smooth curve turns into jumps between vertical bars, because for example in
              > this case all 96 15-minute intervals in a single day end up being plotted with the
              > same X value:
              >
              > http://www.halibut.com/~dlc/tmp/all.png
              > vs
              > http://www.halibut.com/~dlc/tmp/9-10apr.png
              >
              > ploticus 2.31
              >
              > FreeBSD kma.lab.netapp.com 5.4-RELEASE FreeBSD 5.4-RELEASE
              >
              >
              > Thanks!
              >
              >
              > > I gather that you got an extremely "tall" plot in the "bad" case.. was
              > > this the scatterplot or the bar graph?
              > >
              > > Also indicate your pl version and os platform please.
              > >
              > > Thanks,
              > >
              > > Steve
              > >
              > >
              >
              >
              >
              >
              > Yahoo! Groups Links
              >
              >
              >
              >
              >
              >
              >


              Stephen C. Grubb
              Scientific Software Engineer, The Jackson Laboratory
              600 Main Street Bar Harbor, Maine 04609 USA
            Your message has been successfully submitted and would be delivered to recipients shortly.