Loading ...
Sorry, an error occurred while loading the content.

Re: AI-GEOSTATS: Detecting spatial autocorrelation in highly non normal data

Expand Messages
  • Volker Bahn
    The problem is not unique to testing significance of spatial correlation. Any traditional hypothesis test is non-sensical as you describe because we ALWAYS
    Message 1 of 6 , Nov 20, 2003
    View Source
    • 0 Attachment
      The problem is not unique to testing significance of spatial correlation.
      Any traditional hypothesis test is non-sensical as you describe because we
      ALWAYS know that the null hypothesis is wrong. The question of interest is
      how wrong it is and whether the detected effect is of practical consequence.
      However, a hypothesis test does not test practical relevance of an effect
      but statistical power to detect it, which depends on sample size, inherent
      variability and also on effect size. Given a large enough sample size, one
      will always reject the null hypothesis. Why then do people still hold on to
      hypothesis tests? Because it gives them a false sense of objectivity. No one
      wants to admit that the judgement of whether an effect is of practical
      consequence is a to a certain degree inherently subjective decision (as is
      the level of alpha etc).

      Cheers

      Volker


      ----- Original Message -----
      From: "Edzer J. Pebesma" <e.pebesma@...>
      To: <trevor.middel@...>
      Cc: <ai-geostats@...>
      Sent: Thursday, November 20, 2003 5:11
      Subject: Re: AI-GEOSTATS: Detecting spatial autocorrelation in highly non
      normal data


      | Trevor,
      |
      | I always wonder what the value of testing significance of spatial
      | correlation is, and never advise to do it. See, if data are spatial, it
      | is extremely unlikely that they are spatially uncorrelated. Rejecting
      | the test is usually only a matter of collecting sufficient evidence,
      | and not at all an interesting finding, because the data were spatial.
      |
      | Probably a more real problem is: to what extent does the spatial
      | correlation present (which may be very weak!) mess up an analysis
      | that assumes independence of observations. If you choose for
      | an analysis method that addresses spatial correlation, you're always
      | on safe ground.
      |
      | If your data were collected using some form of random sampling,
      | analysis based on independent observations is perfectly valid for
      | estimating areal mean values. This does not imply that data are
      | spatially uncorrelated, but just that they may be treated independent
      | because of the sampling scheme.
      | --
      | Edzer
      |
      | trevor.middel@... wrote:
      |
      | >Hi Folks,
      | >
      | >I'm hoping someone can help steer me in the right direction.
      | >
      | >I have several sets of data acquired from acoustic surveys conducted on a
      | >small lake trout lake. The data consist of sampling units aligned in
      | >transects. Each sampling unit is 50 m in length. A mean lake trout
      density
      | >is associated with each sampling unit.
      | >
      | >I'm interested in examining whether any significant spatial
      autocorrelation
      | >in the exists in the observed distribution of lake trout. The data are
      | >highly non-normal with 75-90% of the observations being zero. Log and Ln
      | >transformations do not normalize the data.
      | >
      | >I've been doing some reading and it seems that most methods of
      quantifying
      | >spatial autocorrelation require some kind of normality in the data. Any
      | >suggestions on how I might proceed with these data?
      | >
      | >Thanks in advance for all suggestions.
      | >
      | >Apologies for the simplicity of the question, but I'm just beginning my
      | >foray into spatial statistics.
      | >
      | >Trevor Middel
      | >
      | >--
      | >* To post a message to the list, send it to ai-geostats@...
      | >* As a general service to the users, please remember to post a summary of
      any useful responses to your questions.
      | >* To unsubscribe, send an email to majordomo@... with no subject and
      "unsubscribe ai-geostats" followed by "end" on the next line in the message
      body. DO NOT SEND Subscribe/Unsubscribe requests to the list
      | >* Support to the list is provided at http://www.ai-geostats.org
      | >
      | >
      |
      |
      | --
      | * To post a message to the list, send it to ai-geostats@...
      | * As a general service to the users, please remember to post a summary of
      any useful responses to your questions.
      | * To unsubscribe, send an email to majordomo@... with no subject and
      "unsubscribe ai-geostats" followed by "end" on the next line in the message
      body. DO NOT SEND Subscribe/Unsubscribe requests to the list
      | * Support to the list is provided at http://www.ai-geostats.org


      --
      * To post a message to the list, send it to ai-geostats@...
      * As a general service to the users, please remember to post a summary of any useful responses to your questions.
      * To unsubscribe, send an email to majordomo@... with no subject and "unsubscribe ai-geostats" followed by "end" on the next line in the message body. DO NOT SEND Subscribe/Unsubscribe requests to the list
      * Support to the list is provided at http://www.ai-geostats.org
    • Ruben Roa Ureta
      ... A cautionary note. The use of random-sampling estimators, i.e. those whose expectations are computed over the space of potential samples by using the
      Message 2 of 6 , Nov 20, 2003
      View Source
      • 0 Attachment
        > Trevor,
        >
        > I always wonder what the value of testing significance of spatial
        > correlation is, and never advise to do it. See, if data are spatial, it
        > is extremely unlikely that they are spatially uncorrelated. Rejecting
        > the test is usually only a matter of collecting sufficient evidence,
        > and not at all an interesting finding, because the data were spatial.
        >
        > Probably a more real problem is: to what extent does the spatial
        > correlation present (which may be very weak!) mess up an analysis
        > that assumes independence of observations. If you choose for
        > an analysis method that addresses spatial correlation, you're always
        > on safe ground.
        >
        > If your data were collected using some form of random sampling,
        > analysis based on independent observations is perfectly valid for
        > estimating areal mean values. This does not imply that data are
        > spatially uncorrelated, but just that they may be treated independent
        > because of the sampling scheme.
        > --
        > Edzer

        A cautionary note. The use of random-sampling estimators, i.e. those whose
        expectations are computed over the space of potential samples by using the
        probabilities assigned by the artificial sampling design, is highly
        questionable. Thus it is not at all clear that estimating areal mean
        values using the sampling scheme is valid, not only in the case of
        spatially correlated phenomena, but generally. See Valliant et al., 2000,
        Finite Population Sampling and Inference, A prediction Approach, Wiley;
        the discussion on the paper of Hansen et al. 1983 An evaluation of
        model-dependent and probability-sampling inferences in sample surveys.
        Journal of the American Statistical Association 78:776-793. I think
        Godambe's paradox (Godambe 1982 Ancillarity principle and a statistical
        paradox. J. Amer. Stats. Assoc. 77:931-933) proves that inference based on
        artificial independence introduced by the sampling scheme is basically
        flawed. Maybe this is too statistical for the mail list but i guess it is
        better to be aware of the problem.
        Rubén

        --
        * To post a message to the list, send it to ai-geostats@...
        * As a general service to the users, please remember to post a summary of any useful responses to your questions.
        * To unsubscribe, send an email to majordomo@... with no subject and "unsubscribe ai-geostats" followed by "end" on the next line in the message body. DO NOT SEND Subscribe/Unsubscribe requests to the list
        * Support to the list is provided at http://www.ai-geostats.org
      • Pierre Goovaerts
        Hello, I agree that in most situations testing for spatial correlation is not very informative since the null hypothesis of spatial independence is
        Message 3 of 6 , Nov 20, 2003
        View Source
        • 0 Attachment
          Hello,

          I agree that in most situations testing for spatial correlation
          is not very informative since the null hypothesis of spatial
          independence is unrealistici and its rejection is trivial.
          This is why at Biomedware we are working on tests
          of hypothesis where the null hypothesis is a particular spatial pattern.
          Stochastic simulation allows one to generate many realizations of this
          spatial pattern that are then used to derive the distribution of the
          test statistics. More information can be found in the following
          publication: http://www-personal.engin.umich.edu/~goovaert/liebisch.pdf

          Cheers,

          Pierre Goovaerts
          <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

          Dr. Pierre Goovaerts
          President of PGeostat, LLC
          Chief Scientist with Biomedware Inc.
          710 Ridgemont Lane
          Ann Arbor, Michigan, 48103-1535, U.S.A.

          E-mail: goovaert@...
          Phone: (734) 668-9900
          Fax: (734) 668-7788
          http://alumni.engin.umich.edu/~goovaert/

          <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

          On Thu, 20 Nov 2003, Volker Bahn wrote:

          > The problem is not unique to testing significance of spatial correlation.
          > Any traditional hypothesis test is non-sensical as you describe because we
          > ALWAYS know that the null hypothesis is wrong. The question of interest is
          > how wrong it is and whether the detected effect is of practical consequence.
          > However, a hypothesis test does not test practical relevance of an effect
          > but statistical power to detect it, which depends on sample size, inherent
          > variability and also on effect size. Given a large enough sample size, one
          > will always reject the null hypothesis. Why then do people still hold on to
          > hypothesis tests? Because it gives them a false sense of objectivity. No one
          > wants to admit that the judgement of whether an effect is of practical
          > consequence is a to a certain degree inherently subjective decision (as is
          > the level of alpha etc).
          >
          > Cheers
          >
          > Volker
          >
          >
          > ----- Original Message -----
          > From: "Edzer J. Pebesma" <e.pebesma@...>
          > To: <trevor.middel@...>
          > Cc: <ai-geostats@...>
          > Sent: Thursday, November 20, 2003 5:11
          > Subject: Re: AI-GEOSTATS: Detecting spatial autocorrelation in highly non
          > normal data
          >
          >
          > | Trevor,
          > |
          > | I always wonder what the value of testing significance of spatial
          > | correlation is, and never advise to do it. See, if data are spatial, it
          > | is extremely unlikely that they are spatially uncorrelated. Rejecting
          > | the test is usually only a matter of collecting sufficient evidence,
          > | and not at all an interesting finding, because the data were spatial.
          > |
          > | Probably a more real problem is: to what extent does the spatial
          > | correlation present (which may be very weak!) mess up an analysis
          > | that assumes independence of observations. If you choose for
          > | an analysis method that addresses spatial correlation, you're always
          > | on safe ground.
          > |
          > | If your data were collected using some form of random sampling,
          > | analysis based on independent observations is perfectly valid for
          > | estimating areal mean values. This does not imply that data are
          > | spatially uncorrelated, but just that they may be treated independent
          > | because of the sampling scheme.
          > | --
          > | Edzer
          > |
          > | trevor.middel@... wrote:
          > |
          > | >Hi Folks,
          > | >
          > | >I'm hoping someone can help steer me in the right direction.
          > | >
          > | >I have several sets of data acquired from acoustic surveys conducted on a
          > | >small lake trout lake. The data consist of sampling units aligned in
          > | >transects. Each sampling unit is 50 m in length. A mean lake trout
          > density
          > | >is associated with each sampling unit.
          > | >
          > | >I'm interested in examining whether any significant spatial
          > autocorrelation
          > | >in the exists in the observed distribution of lake trout. The data are
          > | >highly non-normal with 75-90% of the observations being zero. Log and Ln
          > | >transformations do not normalize the data.
          > | >
          > | >I've been doing some reading and it seems that most methods of
          > quantifying
          > | >spatial autocorrelation require some kind of normality in the data. Any
          > | >suggestions on how I might proceed with these data?
          > | >
          > | >Thanks in advance for all suggestions.
          > | >
          > | >Apologies for the simplicity of the question, but I'm just beginning my
          > | >foray into spatial statistics.
          > | >
          > | >Trevor Middel
          > | >
          > | >--
          > | >* To post a message to the list, send it to ai-geostats@...
          > | >* As a general service to the users, please remember to post a summary of
          > any useful responses to your questions.
          > | >* To unsubscribe, send an email to majordomo@... with no subject and
          > "unsubscribe ai-geostats" followed by "end" on the next line in the message
          > body. DO NOT SEND Subscribe/Unsubscribe requests to the list
          > | >* Support to the list is provided at http://www.ai-geostats.org
          > | >
          > | >
          > |
          > |
          > | --
          > | * To post a message to the list, send it to ai-geostats@...
          > | * As a general service to the users, please remember to post a summary of
          > any useful responses to your questions.
          > | * To unsubscribe, send an email to majordomo@... with no subject and
          > "unsubscribe ai-geostats" followed by "end" on the next line in the message
          > body. DO NOT SEND Subscribe/Unsubscribe requests to the list
          > | * Support to the list is provided at http://www.ai-geostats.org
          >
          >
          > --
          > * To post a message to the list, send it to ai-geostats@...
          > * As a general service to the users, please remember to post a summary of any useful responses to your questions.
          > * To unsubscribe, send an email to majordomo@... with no subject and "unsubscribe ai-geostats" followed by "end" on the next line in the message body. DO NOT SEND Subscribe/Unsubscribe requests to the list
          > * Support to the list is provided at http://www.ai-geostats.org
          >



          --
          * To post a message to the list, send it to ai-geostats@...
          * As a general service to the users, please remember to post a summary of any useful responses to your questions.
          * To unsubscribe, send an email to majordomo@... with no subject and "unsubscribe ai-geostats" followed by "end" on the next line in the message body. DO NOT SEND Subscribe/Unsubscribe requests to the list
          * Support to the list is provided at http://www.ai-geostats.org
        Your message has been successfully submitted and would be delivered to recipients shortly.