Loading ...
Sorry, an error occurred while loading the content.

AI-GEOSTATS: Detecting spatial autocorrelation in highly non normal data

Expand Messages
  • trevor.middel@mnr.gov.on.ca
    Hi Folks, I m hoping someone can help steer me in the right direction. I have several sets of data acquired from acoustic surveys conducted on a small lake
    Message 1 of 6 , Nov 18, 2003
    • 0 Attachment
      Hi Folks,

      I'm hoping someone can help steer me in the right direction.

      I have several sets of data acquired from acoustic surveys conducted on a
      small lake trout lake. The data consist of sampling units aligned in
      transects. Each sampling unit is 50 m in length. A mean lake trout density
      is associated with each sampling unit.

      I'm interested in examining whether any significant spatial autocorrelation
      in the exists in the observed distribution of lake trout. The data are
      highly non-normal with 75-90% of the observations being zero. Log and Ln
      transformations do not normalize the data.

      I've been doing some reading and it seems that most methods of quantifying
      spatial autocorrelation require some kind of normality in the data. Any
      suggestions on how I might proceed with these data?

      Thanks in advance for all suggestions.

      Apologies for the simplicity of the question, but I'm just beginning my
      foray into spatial statistics.

      Trevor Middel

      --
      * To post a message to the list, send it to ai-geostats@...
      * As a general service to the users, please remember to post a summary of any useful responses to your questions.
      * To unsubscribe, send an email to majordomo@... with no subject and "unsubscribe ai-geostats" followed by "end" on the next line in the message body. DO NOT SEND Subscribe/Unsubscribe requests to the list
      * Support to the list is provided at http://www.ai-geostats.org
    • Monica Palaseanu-Lovejoy
      Hi, I never did that myself - but i was at the last STATGIS workshop in Austria and there somebody tackled same problem. They used indicator kriging, built the
      Message 2 of 6 , Nov 19, 2003
      • 0 Attachment
        Hi,

        I never did that myself - but i was at the last STATGIS workshop in
        Austria and there somebody tackled same problem. They used
        indicator kriging, built the semi-variograms for IK and tested like
        that the spatial aotocorrelation.

        The main author is Vanessa Stelzenmuller from the dep. of Aquatic
        Ecology, ICBM, C.v.O. University, 26111 Oldenburg, Germany,
        email: vanessa.stelzenmuller@...-oldenburg.de

        I hope this will help you a little.

        Monica

        --
        * To post a message to the list, send it to ai-geostats@...
        * As a general service to the users, please remember to post a summary of any useful responses to your questions.
        * To unsubscribe, send an email to majordomo@... with no subject and "unsubscribe ai-geostats" followed by "end" on the next line in the message body. DO NOT SEND Subscribe/Unsubscribe requests to the list
        * Support to the list is provided at http://www.ai-geostats.org
      • Edzer J. Pebesma
        Trevor, I always wonder what the value of testing significance of spatial correlation is, and never advise to do it. See, if data are spatial, it is extremely
        Message 3 of 6 , Nov 20, 2003
        • 0 Attachment
          Trevor,

          I always wonder what the value of testing significance of spatial
          correlation is, and never advise to do it. See, if data are spatial, it
          is extremely unlikely that they are spatially uncorrelated. Rejecting
          the test is usually only a matter of collecting sufficient evidence,
          and not at all an interesting finding, because the data were spatial.

          Probably a more real problem is: to what extent does the spatial
          correlation present (which may be very weak!) mess up an analysis
          that assumes independence of observations. If you choose for
          an analysis method that addresses spatial correlation, you're always
          on safe ground.

          If your data were collected using some form of random sampling,
          analysis based on independent observations is perfectly valid for
          estimating areal mean values. This does not imply that data are
          spatially uncorrelated, but just that they may be treated independent
          because of the sampling scheme.
          --
          Edzer

          trevor.middel@... wrote:

          >Hi Folks,
          >
          >I'm hoping someone can help steer me in the right direction.
          >
          >I have several sets of data acquired from acoustic surveys conducted on a
          >small lake trout lake. The data consist of sampling units aligned in
          >transects. Each sampling unit is 50 m in length. A mean lake trout density
          >is associated with each sampling unit.
          >
          >I'm interested in examining whether any significant spatial autocorrelation
          >in the exists in the observed distribution of lake trout. The data are
          >highly non-normal with 75-90% of the observations being zero. Log and Ln
          >transformations do not normalize the data.
          >
          >I've been doing some reading and it seems that most methods of quantifying
          >spatial autocorrelation require some kind of normality in the data. Any
          >suggestions on how I might proceed with these data?
          >
          >Thanks in advance for all suggestions.
          >
          >Apologies for the simplicity of the question, but I'm just beginning my
          >foray into spatial statistics.
          >
          >Trevor Middel
          >
          >--
          >* To post a message to the list, send it to ai-geostats@...
          >* As a general service to the users, please remember to post a summary of any useful responses to your questions.
          >* To unsubscribe, send an email to majordomo@... with no subject and "unsubscribe ai-geostats" followed by "end" on the next line in the message body. DO NOT SEND Subscribe/Unsubscribe requests to the list
          >* Support to the list is provided at http://www.ai-geostats.org
          >
          >


          --
          * To post a message to the list, send it to ai-geostats@...
          * As a general service to the users, please remember to post a summary of any useful responses to your questions.
          * To unsubscribe, send an email to majordomo@... with no subject and "unsubscribe ai-geostats" followed by "end" on the next line in the message body. DO NOT SEND Subscribe/Unsubscribe requests to the list
          * Support to the list is provided at http://www.ai-geostats.org
        • Volker Bahn
          The problem is not unique to testing significance of spatial correlation. Any traditional hypothesis test is non-sensical as you describe because we ALWAYS
          Message 4 of 6 , Nov 20, 2003
          • 0 Attachment
            The problem is not unique to testing significance of spatial correlation.
            Any traditional hypothesis test is non-sensical as you describe because we
            ALWAYS know that the null hypothesis is wrong. The question of interest is
            how wrong it is and whether the detected effect is of practical consequence.
            However, a hypothesis test does not test practical relevance of an effect
            but statistical power to detect it, which depends on sample size, inherent
            variability and also on effect size. Given a large enough sample size, one
            will always reject the null hypothesis. Why then do people still hold on to
            hypothesis tests? Because it gives them a false sense of objectivity. No one
            wants to admit that the judgement of whether an effect is of practical
            consequence is a to a certain degree inherently subjective decision (as is
            the level of alpha etc).

            Cheers

            Volker


            ----- Original Message -----
            From: "Edzer J. Pebesma" <e.pebesma@...>
            To: <trevor.middel@...>
            Cc: <ai-geostats@...>
            Sent: Thursday, November 20, 2003 5:11
            Subject: Re: AI-GEOSTATS: Detecting spatial autocorrelation in highly non
            normal data


            | Trevor,
            |
            | I always wonder what the value of testing significance of spatial
            | correlation is, and never advise to do it. See, if data are spatial, it
            | is extremely unlikely that they are spatially uncorrelated. Rejecting
            | the test is usually only a matter of collecting sufficient evidence,
            | and not at all an interesting finding, because the data were spatial.
            |
            | Probably a more real problem is: to what extent does the spatial
            | correlation present (which may be very weak!) mess up an analysis
            | that assumes independence of observations. If you choose for
            | an analysis method that addresses spatial correlation, you're always
            | on safe ground.
            |
            | If your data were collected using some form of random sampling,
            | analysis based on independent observations is perfectly valid for
            | estimating areal mean values. This does not imply that data are
            | spatially uncorrelated, but just that they may be treated independent
            | because of the sampling scheme.
            | --
            | Edzer
            |
            | trevor.middel@... wrote:
            |
            | >Hi Folks,
            | >
            | >I'm hoping someone can help steer me in the right direction.
            | >
            | >I have several sets of data acquired from acoustic surveys conducted on a
            | >small lake trout lake. The data consist of sampling units aligned in
            | >transects. Each sampling unit is 50 m in length. A mean lake trout
            density
            | >is associated with each sampling unit.
            | >
            | >I'm interested in examining whether any significant spatial
            autocorrelation
            | >in the exists in the observed distribution of lake trout. The data are
            | >highly non-normal with 75-90% of the observations being zero. Log and Ln
            | >transformations do not normalize the data.
            | >
            | >I've been doing some reading and it seems that most methods of
            quantifying
            | >spatial autocorrelation require some kind of normality in the data. Any
            | >suggestions on how I might proceed with these data?
            | >
            | >Thanks in advance for all suggestions.
            | >
            | >Apologies for the simplicity of the question, but I'm just beginning my
            | >foray into spatial statistics.
            | >
            | >Trevor Middel
            | >
            | >--
            | >* To post a message to the list, send it to ai-geostats@...
            | >* As a general service to the users, please remember to post a summary of
            any useful responses to your questions.
            | >* To unsubscribe, send an email to majordomo@... with no subject and
            "unsubscribe ai-geostats" followed by "end" on the next line in the message
            body. DO NOT SEND Subscribe/Unsubscribe requests to the list
            | >* Support to the list is provided at http://www.ai-geostats.org
            | >
            | >
            |
            |
            | --
            | * To post a message to the list, send it to ai-geostats@...
            | * As a general service to the users, please remember to post a summary of
            any useful responses to your questions.
            | * To unsubscribe, send an email to majordomo@... with no subject and
            "unsubscribe ai-geostats" followed by "end" on the next line in the message
            body. DO NOT SEND Subscribe/Unsubscribe requests to the list
            | * Support to the list is provided at http://www.ai-geostats.org


            --
            * To post a message to the list, send it to ai-geostats@...
            * As a general service to the users, please remember to post a summary of any useful responses to your questions.
            * To unsubscribe, send an email to majordomo@... with no subject and "unsubscribe ai-geostats" followed by "end" on the next line in the message body. DO NOT SEND Subscribe/Unsubscribe requests to the list
            * Support to the list is provided at http://www.ai-geostats.org
          • Ruben Roa Ureta
            ... A cautionary note. The use of random-sampling estimators, i.e. those whose expectations are computed over the space of potential samples by using the
            Message 5 of 6 , Nov 20, 2003
            • 0 Attachment
              > Trevor,
              >
              > I always wonder what the value of testing significance of spatial
              > correlation is, and never advise to do it. See, if data are spatial, it
              > is extremely unlikely that they are spatially uncorrelated. Rejecting
              > the test is usually only a matter of collecting sufficient evidence,
              > and not at all an interesting finding, because the data were spatial.
              >
              > Probably a more real problem is: to what extent does the spatial
              > correlation present (which may be very weak!) mess up an analysis
              > that assumes independence of observations. If you choose for
              > an analysis method that addresses spatial correlation, you're always
              > on safe ground.
              >
              > If your data were collected using some form of random sampling,
              > analysis based on independent observations is perfectly valid for
              > estimating areal mean values. This does not imply that data are
              > spatially uncorrelated, but just that they may be treated independent
              > because of the sampling scheme.
              > --
              > Edzer

              A cautionary note. The use of random-sampling estimators, i.e. those whose
              expectations are computed over the space of potential samples by using the
              probabilities assigned by the artificial sampling design, is highly
              questionable. Thus it is not at all clear that estimating areal mean
              values using the sampling scheme is valid, not only in the case of
              spatially correlated phenomena, but generally. See Valliant et al., 2000,
              Finite Population Sampling and Inference, A prediction Approach, Wiley;
              the discussion on the paper of Hansen et al. 1983 An evaluation of
              model-dependent and probability-sampling inferences in sample surveys.
              Journal of the American Statistical Association 78:776-793. I think
              Godambe's paradox (Godambe 1982 Ancillarity principle and a statistical
              paradox. J. Amer. Stats. Assoc. 77:931-933) proves that inference based on
              artificial independence introduced by the sampling scheme is basically
              flawed. Maybe this is too statistical for the mail list but i guess it is
              better to be aware of the problem.
              Rubén

              --
              * To post a message to the list, send it to ai-geostats@...
              * As a general service to the users, please remember to post a summary of any useful responses to your questions.
              * To unsubscribe, send an email to majordomo@... with no subject and "unsubscribe ai-geostats" followed by "end" on the next line in the message body. DO NOT SEND Subscribe/Unsubscribe requests to the list
              * Support to the list is provided at http://www.ai-geostats.org
            • Pierre Goovaerts
              Hello, I agree that in most situations testing for spatial correlation is not very informative since the null hypothesis of spatial independence is
              Message 6 of 6 , Nov 20, 2003
              • 0 Attachment
                Hello,

                I agree that in most situations testing for spatial correlation
                is not very informative since the null hypothesis of spatial
                independence is unrealistici and its rejection is trivial.
                This is why at Biomedware we are working on tests
                of hypothesis where the null hypothesis is a particular spatial pattern.
                Stochastic simulation allows one to generate many realizations of this
                spatial pattern that are then used to derive the distribution of the
                test statistics. More information can be found in the following
                publication: http://www-personal.engin.umich.edu/~goovaert/liebisch.pdf

                Cheers,

                Pierre Goovaerts
                <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

                Dr. Pierre Goovaerts
                President of PGeostat, LLC
                Chief Scientist with Biomedware Inc.
                710 Ridgemont Lane
                Ann Arbor, Michigan, 48103-1535, U.S.A.

                E-mail: goovaert@...
                Phone: (734) 668-9900
                Fax: (734) 668-7788
                http://alumni.engin.umich.edu/~goovaert/

                <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

                On Thu, 20 Nov 2003, Volker Bahn wrote:

                > The problem is not unique to testing significance of spatial correlation.
                > Any traditional hypothesis test is non-sensical as you describe because we
                > ALWAYS know that the null hypothesis is wrong. The question of interest is
                > how wrong it is and whether the detected effect is of practical consequence.
                > However, a hypothesis test does not test practical relevance of an effect
                > but statistical power to detect it, which depends on sample size, inherent
                > variability and also on effect size. Given a large enough sample size, one
                > will always reject the null hypothesis. Why then do people still hold on to
                > hypothesis tests? Because it gives them a false sense of objectivity. No one
                > wants to admit that the judgement of whether an effect is of practical
                > consequence is a to a certain degree inherently subjective decision (as is
                > the level of alpha etc).
                >
                > Cheers
                >
                > Volker
                >
                >
                > ----- Original Message -----
                > From: "Edzer J. Pebesma" <e.pebesma@...>
                > To: <trevor.middel@...>
                > Cc: <ai-geostats@...>
                > Sent: Thursday, November 20, 2003 5:11
                > Subject: Re: AI-GEOSTATS: Detecting spatial autocorrelation in highly non
                > normal data
                >
                >
                > | Trevor,
                > |
                > | I always wonder what the value of testing significance of spatial
                > | correlation is, and never advise to do it. See, if data are spatial, it
                > | is extremely unlikely that they are spatially uncorrelated. Rejecting
                > | the test is usually only a matter of collecting sufficient evidence,
                > | and not at all an interesting finding, because the data were spatial.
                > |
                > | Probably a more real problem is: to what extent does the spatial
                > | correlation present (which may be very weak!) mess up an analysis
                > | that assumes independence of observations. If you choose for
                > | an analysis method that addresses spatial correlation, you're always
                > | on safe ground.
                > |
                > | If your data were collected using some form of random sampling,
                > | analysis based on independent observations is perfectly valid for
                > | estimating areal mean values. This does not imply that data are
                > | spatially uncorrelated, but just that they may be treated independent
                > | because of the sampling scheme.
                > | --
                > | Edzer
                > |
                > | trevor.middel@... wrote:
                > |
                > | >Hi Folks,
                > | >
                > | >I'm hoping someone can help steer me in the right direction.
                > | >
                > | >I have several sets of data acquired from acoustic surveys conducted on a
                > | >small lake trout lake. The data consist of sampling units aligned in
                > | >transects. Each sampling unit is 50 m in length. A mean lake trout
                > density
                > | >is associated with each sampling unit.
                > | >
                > | >I'm interested in examining whether any significant spatial
                > autocorrelation
                > | >in the exists in the observed distribution of lake trout. The data are
                > | >highly non-normal with 75-90% of the observations being zero. Log and Ln
                > | >transformations do not normalize the data.
                > | >
                > | >I've been doing some reading and it seems that most methods of
                > quantifying
                > | >spatial autocorrelation require some kind of normality in the data. Any
                > | >suggestions on how I might proceed with these data?
                > | >
                > | >Thanks in advance for all suggestions.
                > | >
                > | >Apologies for the simplicity of the question, but I'm just beginning my
                > | >foray into spatial statistics.
                > | >
                > | >Trevor Middel
                > | >
                > | >--
                > | >* To post a message to the list, send it to ai-geostats@...
                > | >* As a general service to the users, please remember to post a summary of
                > any useful responses to your questions.
                > | >* To unsubscribe, send an email to majordomo@... with no subject and
                > "unsubscribe ai-geostats" followed by "end" on the next line in the message
                > body. DO NOT SEND Subscribe/Unsubscribe requests to the list
                > | >* Support to the list is provided at http://www.ai-geostats.org
                > | >
                > | >
                > |
                > |
                > | --
                > | * To post a message to the list, send it to ai-geostats@...
                > | * As a general service to the users, please remember to post a summary of
                > any useful responses to your questions.
                > | * To unsubscribe, send an email to majordomo@... with no subject and
                > "unsubscribe ai-geostats" followed by "end" on the next line in the message
                > body. DO NOT SEND Subscribe/Unsubscribe requests to the list
                > | * Support to the list is provided at http://www.ai-geostats.org
                >
                >
                > --
                > * To post a message to the list, send it to ai-geostats@...
                > * As a general service to the users, please remember to post a summary of any useful responses to your questions.
                > * To unsubscribe, send an email to majordomo@... with no subject and "unsubscribe ai-geostats" followed by "end" on the next line in the message body. DO NOT SEND Subscribe/Unsubscribe requests to the list
                > * Support to the list is provided at http://www.ai-geostats.org
                >



                --
                * To post a message to the list, send it to ai-geostats@...
                * As a general service to the users, please remember to post a summary of any useful responses to your questions.
                * To unsubscribe, send an email to majordomo@... with no subject and "unsubscribe ai-geostats" followed by "end" on the next line in the message body. DO NOT SEND Subscribe/Unsubscribe requests to the list
                * Support to the list is provided at http://www.ai-geostats.org
              Your message has been successfully submitted and would be delivered to recipients shortly.