Loading ...
Sorry, an error occurred while loading the content.

GEOSTATS: RE: Model Comparison....Help anyone?

Expand Messages
  • jchristensen@seamail.nos.noaa.gov
    Dear Geostats subscribers, I am a fisheries biologist in somewhat of a dilemma. Let me first attempt to specify the problem: I have constructed an
    Message 1 of 3 , Jul 14, 1997
    • 0 Attachment
      Dear Geostats subscribers,

      I am a fisheries biologist in somewhat of a dilemma. Let me first attempt to
      specify the problem: I have constructed an "ecological" model in the arc/info
      grid module. The model is designed to predict the relative suitability of
      discrete points in space for oysters in Pensacola Bay, Florida. Subsequent to
      the development of my model, I superimposed observed oyster reef locations on
      the model output.

      My question is this: What is the most appropriate method to test whether or not
      my model results do a reasonable job of predicting or identifying prime oyster
      habitat? The model results in 37,000 grid cells (for the entire bay); however,
      I have only 113 existing oyster reef points. The model output is categorical
      (optimum, high, medium, low, unsuitable), and my oyster point data is merely
      presence/absence (0/1).

      Visually, the distribution of oyster points in my model output indicate that the
      model worked well (87% of points fell in optimum category, 13% in high, and no
      points in the remaining suitability categories). Even so, because there are so
      many resulting grid cells in the model, most of the optimum and high grid cell
      contain NO oysters. Is there a method to investigate whether or not the model
      is statistically significant?

      I appreciate your interest and help with my question in advance. Please forward
      your thoughts to me at the e-mail address given below. Again, thanks much!


      Sincerely,

      John D. Christensen, Fisheries Biologist
      jchristensen@...
      NOAA/NOS/SEA Division
      Silver Spring, MD 20910


      --
      *To post a message to the list, send it to ai-geostats@....
      *As a general service to list users, please remember to post a summary
      of any useful responses to your questions.
      *To unsubscribe, send email to majordomo@... with no subject and
      "unsubscribe ai-geostats" in the message body.
      DO NOT SEND Subscribe/Unsubscribe requests to the list!
    • Jennifer Dungan
      ... One appropriate method would be the practice used in comparison of thematic maps generated using remotely sensed data (usually used with the raster, or
      Message 2 of 3 , Jul 14, 1997
      • 0 Attachment
        jchristensen@... wrote:

        > My question is this: What is the most appropriate method to test whether or not
        > my model results do a reasonable job of predicting or identifying prime oyster
        > habitat? The model results in 37,000 grid cells (for the entire bay); however,
        > I have only 113 existing oyster reef points. The model output is categorical
        > (optimum, high, medium, low, unsuitable), and my oyster point data is merely
        > presence/absence (0/1).

        One appropriate method would be the practice used in comparison of thematic
        maps generated using remotely sensed data (usually used with the raster, or
        grid cell, data model). See pages 388-395 in Campbell (1996) Introduction to
        Remote Sensing for a clear exposition. The summary statistic is called kappa,
        and it distills the measure of agreement between the two maps adjusted for
        chance agreement. First, a contingency table or error matrix must be created.

        > Visually, the distribution of oyster points in my model output indicate that the
        > model worked well (87% of points fell in optimum category, 13% in high, and no
        > points in the remaining suitability categories). Even so, because there are so
        > many resulting grid cells in the model, most of the optimum and high grid cell
        > contain NO oysters. Is there a method to investigate whether or not the model
        > is statistically significant?

        The error matrix would then have

        Number of model cells Number of model cell
        with optimal (and/or high) oysters with low suit. of oysters
        Number of cells
        with existing oysters 98 (or 113) 0

        Number of cells
        without existing oysters x1 x2

        If your 113 existing oyster locations represent
        an exhaustive survey and are all the points you think are in the bay, x1 and x2 will be
        large numbers. If not, and you will only test the quality of your model at the 113
        survey locations, x1=15 (or 0) and x2=0 (or 15). You will need to decide what cutoff
        in terms of suitability to use to evaulate the results (the optimal only or optimal+high
        classes). See the example in Campbell for how to calculate kappa. You could also use
        alternative methods as described in his text.

        This is of course an aspatial summary of your results. Also valuable would
        be a simple error map, showing *where* the errors of commission and ommission
        are.

        Sincerely,

        Jennifer

        Jennifer Dungan | MS 242-4
        Research Scientist, JCWS, Inc. | NASA Ames Research Center
        Tel: 415-604-3618 FAX: 415-604-4680 | Moffett Field, CA 94035-1000
        email: jdungan@... | USA
        URL: http://geo.arc.nasa.gov |

        ***Jung says that dreams are the woofer and tweeter of the total
        sound system. -- Chris in the Morning ***
        --
        *To post a message to the list, send it to ai-geostats@....
        *As a general service to list users, please remember to post a summary
        of any useful responses to your questions.
        *To unsubscribe, send email to majordomo@... with no subject and
        "unsubscribe ai-geostats" in the message body.
        DO NOT SEND Subscribe/Unsubscribe requests to the list!
      • J. Felipe Costa
        Dear John, Since the problem you posed consist in checking discrete (very dense) block model interpolated from a sparsely sampled data set, I would try a cross
        Message 3 of 3 , Jul 15, 1997
        • 0 Attachment
          Dear John,

          Since the problem you posed consist in checking discrete (very dense)
          block model interpolated from a sparsely sampled data set, I would try a
          cross validation or jacknifing method.

          As far as I understood, the only information you have are the 137 sampled
          points and it is against them you can check your assumptions used in
          interpolating the entire grid.

          Unless you have further information to validate your model, the input
          sample data set seems to be the only reasonable info to be cross checked.

          Check in the geostat literature (Isaaks and Sirvastava 1989 for example)
          for the details in cross validation.

          cheers

          ------------------------------------------------------------------------
          J. Felipe Costa, PhD candidate Phone:
          University of Queensland national: (07)3365.3473
          Dept. of Earth Sciences international: (61)(7)3365-3473
          Brisbane, Qld 4072 Fax: (07) 3365-1277
          E-mail: costa@... home: (07) 3878.6475
          ------------------------------------------------------------------------

          --
          *To post a message to the list, send it to ai-geostats@....
          *As a general service to list users, please remember to post a summary
          of any useful responses to your questions.
          *To unsubscribe, send email to majordomo@... with no subject and
          "unsubscribe ai-geostats" in the message body.
          DO NOT SEND Subscribe/Unsubscribe requests to the list!
        Your message has been successfully submitted and would be delivered to recipients shortly.