Loading ...
Sorry, an error occurred while loading the content.

GEOSTATS: large datasets

Expand Messages
  • Marcia Gumpertz
    I m interested in doing a regression with spatially correlated errors. There are several regressor variables. Usually I would use SAS PROC MIXED to estimated
    Message 1 of 3 , Jun 2, 1998
    • 0 Attachment
      I'm interested in doing a regression with spatially correlated errors. There
      are several regressor variables. Usually I would use SAS PROC MIXED to
      estimated all the parameters at once using maximum likelihood, but this dataset
      has about 4000 observations. I'd be happy if I could just do estimated
      generalized least squares given a known (assumed) covariance matrix, but I have
      a strong feeling any program is going to balk at inverting such a large
      covariance matrix. What do you all do in a situation like this?

      Thanks,

      Marcia

      --
      -----------------------------------------------------------------
      Marcia Gumpertz phone: (919)515-1923
      Box 8203 fax: (919)515-1169
      Statistics Department e-mail: gumpertz@...
      NCSU
      Raleigh, NC 27695-8203
      -----------------------------------------------------------------
      --
      *To post a message to the list, send it to ai-geostats@....
      *As a general service to list users, please remember to post a summary
      of any useful responses to your questions.
      *To unsubscribe, send email to majordomo@... with no subject and
      "unsubscribe ai-geostats" in the message body.
      DO NOT SEND Subscribe/Unsubscribe requests to the list!
    • Yoshiro Nagao
      I am sorry that I cannot be helpful, but I am interested in maximum likelihood regression analysis by SAS applied to spatial dataset. Could you elaborate it
      Message 2 of 3 , Jun 3, 1998
      • 0 Attachment
        I am sorry that I cannot be helpful, but
        I am interested in maximum likelihood regression analysis
        by SAS applied to spatial dataset.

        Could you elaborate it farther, I mean, are there
        any geostatistical module for SAS?

        "Marcia Gumpertz" <gumpertz@...> sama said:
        >I'm interested in doing a regression with spatially correlated errors. There
        >are several regressor variables. Usually I would use SAS PROC MIXED to
        >estimated all the parameters at once using maximum likelihood, but this dataset
        >has about 4000 observations. I'd be happy if I could just do estimated
        >generalized least squares given a known (assumed) covariance matrix, but I have
        >a strong feeling any program is going to balk at inverting such a large
        >covariance matrix. What do you all do in a situation like this?

        Yoshiro Nagao ( Y-Nagao )
        International Centre for Medical Research
        Kobe University School of Medicine
        --
        *To post a message to the list, send it to ai-geostats@....
        *As a general service to list users, please remember to post a summary
        of any useful responses to your questions.
        *To unsubscribe, send email to majordomo@... with no subject and
        "unsubscribe ai-geostats" in the message body.
        DO NOT SEND Subscribe/Unsubscribe requests to the list!
      • Robert K. Pace
        With regard to large datasets, I cannot comment on how to estimate spatial autoregressions in SAS. I have estimated a lattice model in S-PLUS (SAR) using 3,107
        Message 3 of 3 , Jun 4, 1998
        • 0 Attachment
          With regard to large datasets, I cannot comment on how to estimate spatial
          autoregressions in SAS. I have estimated a lattice model in S-PLUS (SAR)
          using 3,107 observations. This took around 20 minutes on a 200 Mhz Pentium
          Pro.

          I have published some papers about how to estimate large lattice models - I
          have dealt with as many as 200K+ observations using some of these
          techniques. I currently have two software packages I have written which I
          am distributing for free (with correspondingly little emphasis on GUI).

          I have a Fortran 90 based package with PC executable code. This relies on
          nearest neighbors for specifying the spatial relations. One can choose the
          number of neighbors and the weight given to them. To conduct maximum
          likelihood you need the log of the determinant of the variance-covariance
          matrix or its inverse. This package uses an approximation (which yields
          confidence intervals) for the log-determinant. We have handled matrices as
          large as 1M by 1M using this technique (see Barry and Pace, Linear Algebra
          and its Applications, forthcoming).

          Given the log of the determinant, we estimate via ML the model:
          Y=X*B1+S*X*B2+alpha*S*Y+e, where S is the n by n spatial weight matrix (we
          take the log-determinant of (I-alpha*S)). This corresponds to having
          separately spatially lagged independent and dependent variables. It takes
          under 10 seconds on a Pentium 233 MMX to find the neighbors, estimate the
          log-determinant, and compute the maximum likelihood estimates for the 3,107
          observation dataset. It yields profile likelihoods for the overall model
          and many submodels. There are separate profile likelihoods for the lower
          and upper bounds of the log-determinant and hence the model accounts for
          this source of uncertainty. Hence, one can conduct likelihood ratio tests
          easily.

          Compressed this package with an example and some documentation takes under
          1MB compressed and hence I can email it to whoever wishes to use it.

          I have a second more comprehensive (at least for lattice models) package
          written in Matlab. This does SAR, CAR, the model above, has Delaunay and
          nearest neighbor weight matrices, and simulation routines. This also has
          several example datasets and so forth. This package takes around 30MB and
          cannot be so easily sent. If anyone truly wants it right away, I can burn a
          CDROM and send it. However, I am planning on duplicating this on CDROM and
          sending out some copies at the end-of-the month for those without a
          pressing need.

          Anyone who is interested in receiving these please send a message to
          kpace@... and put "spatial package" as the first two words
          of the subject. I will send you either or both of these toolboxes.

          Kelley Pace
          Real Estate Research Institute
          E.J. Ourso College of Business
          Louisiana State University
          Baton Rouge, LA 70803

          email: kpace@...

          also

          email kelleypace@...

          phone: 504-388-6257
          --
          *To post a message to the list, send it to ai-geostats@....
          *As a general service to list users, please remember to post a summary
          of any useful responses to your questions.
          *To unsubscribe, send email to majordomo@... with no subject and
          "unsubscribe ai-geostats" in the message body.
          DO NOT SEND Subscribe/Unsubscribe requests to the list!
        Your message has been successfully submitted and would be delivered to recipients shortly.