Loading ...
Sorry, an error occurred while loading the content.

Re: AI-GEOSTATS: geoR and memory

Expand Messages
  • Ernesto Jardim
    ... Hi Marcelo, It s commom to run out of memory with R, in particular if you are using a version older than 1.4 (see man pages in that case). Also in versions
    Message 1 of 11 , Oct 22, 2003
    • 0 Attachment
      On Wed, 2003-10-22 at 12:03, Marcelo Alexandre Bruno wrote:
      > Dear list-members
      >
      > I try to run geoR for generate interpolation maps.
      > My data is: x(longitud, decimal degree),y(latitud,
      > decimal degree),z(Sa, acoustic parameter, limits range
      > is 0 to 1000000, extremely skewed, represent
      > backscatering echoes signals of fish schools). The
      > dataset is 3007 nautic miles of sampling!
      >
      > The analysis is run according to steps of Dr. Paulo
      > Justiano's homepage
      > http://www.est.ufpr.br/~paulojus/geoR/geoRdoc/geoRintro.html#starting
      > .
      > When kriging run, consume all memory of system,
      > (R.bin with only geoR consumes 86.8mb!! in Athlon
      > xp1.6GHz, Linux redhat 9.0, 250mb + swapp 541mb).
      > Is there something wrong (possible newbie question)
      > or its possible vectorizing computations to
      > reduce memory consume?
      >
      >
      > "The necessary reliance on a n by n matrix, where n
      > represents the number of observations, greatly impedes
      > traditional spatial statistics from handling larger
      > sample sizes." published from:
      > Pace, R. Kelley, and Ronald Barry, "Fast CARs,"
      > Journal of Statistical Computation and Simulation,
      > Volume 59, Number 2, 1997, p. 123-147.
      >
      > This is the case??
      > (Apologies for my long msg.)
      > Thanks for all.
      >
      > =====
      > ## ~~~~~~~ Oceanólogo ~~~~~~~ ##
      > # Marcelo Alexandre Bruno
      > # Linux User: 124592
      > # Pós-graduação Oceanografia Biológica
      > # FUNDACAO UNIV. FEDERAL do RIO GRANDE
      > # Departamento de Oceanografia
      > # Lab. de Tecnologia Pesqueira e Hidroacústica
      > # AV. ITÁLIA km 8 s/n - CARREIROS
      > # 96201-900 (0xx53) 2336528
      > # Rio Grande - RS - BRAZIL
      > ## ---------------------------------------- ##
      >
      > Yahoo! Mail - o melhor webmail do Brasil
      > http://mail.yahoo.com.br
      >
      > --
      > * To post a message to the list, send it to ai-geostats@...
      > * As a general service to the users, please remember to post a summary of any useful responses to your questions.
      > * To unsubscribe, send an email to majordomo@... with no subject and "unsubscribe ai-geostats" followed by "end" on the next line in the message body. DO NOT SEND Subscribe/Unsubscribe requests to the list
      > * Support to the list is provided at http://www.ai-geostats.org

      Hi Marcelo,

      It's commom to run out of memory with R, in particular if you are using
      a version older than 1.4 (see man pages in that case). Also in versions
      newer than 1.7 R loads a lot of libraries that may not be needed. You
      can detach those libraries to get more memory.

      My R.bin with geoR uses 16252k, the values you're getting includes the
      memory occupied by your data.

      When I have that kind of problems I try to reduce the data to the
      minimum necessary and run R directly on the terminal, without X running.

      Maybe Paulo Ribeiro can give you some hints about the way geoR processes
      the data and try to optimize your computer resources.

      Best regards

      EJ

      --
      Ernesto Jardim <ernesto@...>
      Biólogo Marinho/Marine Biologist
      IPIMAR - Instituto Nacional de Investigação Agrária e das Pescas
      IPIMAR - National Research Institute for Agriculture and Fisheries
      Av. Brasilia, 1400-006
      Lisboa, Portugal
      Tel: +351 213 027 000
      Fax: +351 213 015 948
      http://ernesto.freezope.org


      --
      * To post a message to the list, send it to ai-geostats@...
      * As a general service to the users, please remember to post a summary of any useful responses to your questions.
      * To unsubscribe, send an email to majordomo@... with no subject and "unsubscribe ai-geostats" followed by "end" on the next line in the message body. DO NOT SEND Subscribe/Unsubscribe requests to the list
      * Support to the list is provided at http://www.ai-geostats.org
    • Martin Roseveare
      Just to reinforce what Edzer said about projections - convert your longitude and latitude data into a planar projection first - UTM should be OK though check
      Message 2 of 11 , Oct 24, 2003
      • 0 Attachment
        Just to reinforce what Edzer said about projections - convert your longitude
        and latitude data into a planar projection first - UTM should be OK though
        check your data doesn't cross zones. If you don't do this the stats will not
        produce meaningful results as degrees are not a correct spatial measure.
        Even for very small areas the shape and hence the interrelationship between
        data is very different for long lat and metres.

        Martin Roseveare
        ______________________________________

        ArchaeoPhysica Ltd.
        Reconnaissance & Geophysics for Archaeology

        Telephone: +44 (0) 7050 369789
        E-mail: mail@...
        Website: http://www.archaeophysica.co.uk
        ______________________________________

        This e-mail is intended only for the addressee
        named above and may contain confidential or
        privileged information. If you receive this e-mail
        by mistake please advise the sender and destroy
        it without further disclosure of its content.

        Unless otherwise stated no opinions expressed in
        this e-mail should be regarded as representative of
        any policy of ArchaeoPhysica Ltd.
        ----- Original Message -----
        From: "Marcelo Alexandre Bruno" <marcelo2lei@...>
        To: <Paulo.Ribeiro@...>
        Cc: <ai-geostats@...>
        Sent: Wednesday, October 22, 2003 12:03 PM
        Subject: AI-GEOSTATS: geoR and memory


        > Dear list-members
        >
        > I try to run geoR for generate interpolation maps.
        > My data is: x(longitud, decimal degree),y(latitud,
        > decimal degree),z(Sa, acoustic parameter, limits range
        > is 0 to 1000000, extremely skewed, represent
        > backscatering echoes signals of fish schools). The
        > dataset is 3007 nautic miles of sampling!
        >
        > The analysis is run according to steps of Dr. Paulo
        > Justiano's homepage
        > http://www.est.ufpr.br/~paulojus/geoR/geoRdoc/geoRintro.html#starting
        > .
        > When kriging run, consume all memory of system,
        > (R.bin with only geoR consumes 86.8mb!! in Athlon
        > xp1.6GHz, Linux redhat 9.0, 250mb + swapp 541mb).
        > Is there something wrong (possible newbie question)
        > or its possible vectorizing computations to
        > reduce memory consume?
        >
        >
        > "The necessary reliance on a n by n matrix, where n
        > represents the number of observations, greatly impedes
        > traditional spatial statistics from handling larger
        > sample sizes." published from:
        > Pace, R. Kelley, and Ronald Barry, "Fast CARs,"
        > Journal of Statistical Computation and Simulation,
        > Volume 59, Number 2, 1997, p. 123-147.
        >
        > This is the case??
        > (Apologies for my long msg.)
        > Thanks for all.
        >
        > =====
        > ## ~~~~~~~ Oceanslogo ~~~~~~~ ##
        > # Marcelo Alexandre Bruno
        > # Linux User: 124592
        > # Pss-graduagco Oceanografia Biolsgica
        > # FUNDACAO UNIV. FEDERAL do RIO GRANDE
        > # Departamento de Oceanografia
        > # Lab. de Tecnologia Pesqueira e Hidroaczstica
        > # AV. ITALIA km 8 s/n - CARREIROS
        > # 96201-900 (0xx53) 2336528
        > # Rio Grande - RS - BRAZIL
        > ## ---------------------------------------- ##
        >
        > Yahoo! Mail - o melhor webmail do Brasil
        > http://mail.yahoo.com.br
        >
        > --
        > * To post a message to the list, send it to ai-geostats@...
        > * As a general service to the users, please remember to post a summary of
        any useful responses to your questions.
        > * To unsubscribe, send an email to majordomo@... with no subject and
        "unsubscribe ai-geostats" followed by "end" on the next line in the message
        body. DO NOT SEND Subscribe/Unsubscribe requests to the list
        > * Support to the list is provided at http://www.ai-geostats.org
        >


        --
        * To post a message to the list, send it to ai-geostats@...
        * As a general service to the users, please remember to post a summary of any useful responses to your questions.
        * To unsubscribe, send an email to majordomo@... with no subject and "unsubscribe ai-geostats" followed by "end" on the next line in the message body. DO NOT SEND Subscribe/Unsubscribe requests to the list
        * Support to the list is provided at http://www.ai-geostats.org
      • Isobel Clark
        Martin ... Do you have an algorithm that works everywhere in the world? Isobel http://geoecosse.bizland.com/whatsnew.htm
        Message 3 of 11 , Oct 24, 2003
        • 0 Attachment
          Martin

          > - convert your longitude
          > and latitude data into a planar projection first
          Do you have an algorithm that works everywhere in the
          world?

          Isobel
          http://geoecosse.bizland.com/whatsnew.htm

          ________________________________________________________________________
          Want to chat instantly with your online friends? Get the FREE Yahoo!
          Messenger http://mail.messenger.yahoo.co.uk

          --
          * To post a message to the list, send it to ai-geostats@...
          * As a general service to the users, please remember to post a summary of any useful responses to your questions.
          * To unsubscribe, send an email to majordomo@... with no subject and "unsubscribe ai-geostats" followed by "end" on the next line in the message body. DO NOT SEND Subscribe/Unsubscribe requests to the list
          * Support to the list is provided at http://www.ai-geostats.org
        • Susan King
          I would like to have algorithms for converting lat/long into different projections (albers, utm). Susan L. King Operations Research Analyst USDA Forest
          Message 4 of 11 , Oct 24, 2003
          • 0 Attachment
            I would like to have algorithms for converting lat/long into different
            projections (albers, utm).




            Susan L. King
            Operations Research Analyst
            USDA Forest Service
            11 Campus Blvd., Suite 200
            Newtown Square, PA 19073
            Phone: (610) 557-4048
            Email: sking01@...
            FAX: (610) 557-4250



            Isobel Clark
            <drisobelclark@y To: Martin Roseveare <martin_roseveare@...>
            ahoo.co.uk> cc: ai-geostats@...
            Sent by: Subject: Re: AI-GEOSTATS: geoR and memory
            ai-geostats-list
            @...


            10/24/2003 11:06
            AM
            Please respond
            to Isobel Clark






            Martin

            > - convert your longitude
            > and latitude data into a planar projection first
            Do you have an algorithm that works everywhere in the
            world?

            Isobel
            http://geoecosse.bizland.com/whatsnew.htm

            ________________________________________________________________________
            Want to chat instantly with your online friends? Get the FREE Yahoo!
            Messenger http://mail.messenger.yahoo.co.uk

            --
            * To post a message to the list, send it to ai-geostats@...
            * As a general service to the users, please remember to post a summary of
            any useful responses to your questions.
            * To unsubscribe, send an email to majordomo@... with no subject and
            "unsubscribe ai-geostats" followed by "end" on the next line in the message
            body. DO NOT SEND Subscribe/Unsubscribe requests to the list
            * Support to the list is provided at http://www.ai-geostats.org




            --
            * To post a message to the list, send it to ai-geostats@...
            * As a general service to the users, please remember to post a summary of any useful responses to your questions.
            * To unsubscribe, send an email to majordomo@... with no subject and "unsubscribe ai-geostats" followed by "end" on the next line in the message body. DO NOT SEND Subscribe/Unsubscribe requests to the list
            * Support to the list is provided at http://www.ai-geostats.org
          • Martin Roseveare
            Isobel, There are published algorithms that will do this - one of the advantages of the UTM projection is that it can be generated anywhere, with the zoning of
            Message 5 of 11 , Oct 24, 2003
            • 0 Attachment
              Isobel,

              There are published algorithms that will do this - one of the advantages of
              the UTM projection is that it can be generated anywhere, with the zoning of
              the projection accounting for the necessary differences around the globe.
              Good software should be able to deal with UTM zones and hence work anywhere.
              The disadvantage of UTM is that other commerical mapping (including State)
              is more likely to use a local grid for the area concerned.

              It may be possible to find free or shareware that will do the projection -
              one issue that needs to be addressed though is precision as a lot of the
              free code is limited in this respect and of course the desired spatial
              accuracy of subsequent statistical processes needs to be incorporated within
              this. Otherwise there is often software tailor made for specific countries,
              e.g., for the UK and Ireland that usually offers a higher precision. It
              depends on the source data and what you want to do with it - we work to
              sub-metre precision and this can be a problem.

              If any of the list need to pursue this further I have a contact who is
              renowned for his work on co-ordinate transformations in the aerial
              photography world and who would probably be able to advise.

              Martin
              ______________________________________

              ArchaeoPhysica Ltd.
              Reconnaissance & Geophysics for Archaeology

              Telephone: +44 (0) 7050 369789
              E-mail: mail@...
              Website: http://www.archaeophysica.co.uk
              ______________________________________

              This e-mail is intended only for the addressee
              named above and may contain confidential or
              privileged information. If you receive this e-mail
              by mistake please advise the sender and destroy
              it without further disclosure of its content.

              Unless otherwise stated no opinions expressed in
              this e-mail should be regarded as representative of
              any policy of ArchaeoPhysica Ltd.
              ----- Original Message -----
              From: "Isobel Clark" <drisobelclark@...>
              To: "Martin Roseveare" <martin_roseveare@...>
              Cc: <ai-geostats@...>
              Sent: Friday, October 24, 2003 4:06 PM
              Subject: Re: AI-GEOSTATS: geoR and memory


              > Martin
              >
              > > - convert your longitude
              > > and latitude data into a planar projection first
              > Do you have an algorithm that works everywhere in the
              > world?
              >
              > Isobel
              > http://geoecosse.bizland.com/whatsnew.htm
              >
              > ________________________________________________________________________
              > Want to chat instantly with your online friends? Get the FREE Yahoo!
              > Messenger http://mail.messenger.yahoo.co.uk


              --
              * To post a message to the list, send it to ai-geostats@...
              * As a general service to the users, please remember to post a summary of any useful responses to your questions.
              * To unsubscribe, send an email to majordomo@... with no subject and "unsubscribe ai-geostats" followed by "end" on the next line in the message body. DO NOT SEND Subscribe/Unsubscribe requests to the list
              * Support to the list is provided at http://www.ai-geostats.org
            • Jim Bash
              ... There is no single projection that would be appropriate for everywhere in the world at once, just as there is no 2 dimensional (planar) projection that can
              Message 6 of 11 , Oct 24, 2003
              • 0 Attachment
                On Fri, 24 Oct 2003, [iso-8859-1] Isobel Clark wrote:
                > > - convert your longitude
                > > and latitude data into a planar projection first
                > Do you have an algorithm that works everywhere in the
                > world?

                There is no single projection that would be appropriate for everywhere in
                the world at once, just as there is no 2 dimensional (planar) projection
                that can accurately preserve angles, distance, direction and area for an
                entire map:-) Is that what you are asking for?

                jim b
                -----
                Jim Bash, Associate Director Internet : JBash@...
                CAGIS (m/c 092) Phone (W): 312/996-6367
                Program in Geography, Dept of Anthropology Fax : 312/996-6343
                University of Illinois at Chicago
                1007 W. Harrison St, Rm 2102 BSB URL : http://www.cagis.uic.edu/
                Chicago, IL 60607-7138

                --
                * To post a message to the list, send it to ai-geostats@...
                * As a general service to the users, please remember to post a summary of any useful responses to your questions.
                * To unsubscribe, send an email to majordomo@... with no subject and "unsubscribe ai-geostats" followed by "end" on the next line in the message body. DO NOT SEND Subscribe/Unsubscribe requests to the list
                * Support to the list is provided at http://www.ai-geostats.org
              • Jim Bash
                ... You might check out http://www.remotesensing.org/proj/ which has source and documentation for PROJ.4, map projection software. I think the source is in C
                Message 7 of 11 , Oct 24, 2003
                • 0 Attachment
                  On Fri, 24 Oct 2003, Susan King wrote:
                  > I would like to have algorithms for converting lat/long into different
                  > projections (albers, utm).

                  You might check out http://www.remotesensing.org/proj/ which has source
                  and documentation for PROJ.4, map projection software. I think the source
                  is in C and you would probably have to dig for the algorithms, unless they
                  are in the documentation.

                  Sorry but I haven't looked at PROJ.4 in a long time. I normally refer to a
                  book by John Snyder, "Map Projections-- A Working Manual" which was
                  published by USGS as "USGS Professional Paper 1395" in 1987. But that
                  isn't online and I'm not sure it is available anymore.

                  HTH a little,

                  jim b
                  ------
                  Jim Bash, Associate Director Internet : JBash@...
                  CAGIS (m/c 092) Phone (W): 312/996-6367
                  Program in Geography, Dept of Anthropology Fax : 312/996-6343
                  University of Illinois at Chicago
                  1007 W. Harrison St, Rm 2102 BSB URL : http://www.cagis.uic.edu/
                  Chicago, IL 60607-7138

                  --
                  * To post a message to the list, send it to ai-geostats@...
                  * As a general service to the users, please remember to post a summary of any useful responses to your questions.
                  * To unsubscribe, send an email to majordomo@... with no subject and "unsubscribe ai-geostats" followed by "end" on the next line in the message body. DO NOT SEND Subscribe/Unsubscribe requests to the list
                  * Support to the list is provided at http://www.ai-geostats.org
                • Julian Burgos
                  Hello everyone, Here is a link to Geotrans, a program developed by the US National Imagery and Mapping Agency to convert locations between different coordinate
                  Message 8 of 11 , Oct 24, 2003
                  • 0 Attachment
                    Hello everyone,

                    Here is a link to Geotrans, a program developed by the US National Imagery
                    and Mapping Agency to convert locations between different coordinate
                    systems.
                    http://earth-info.nima.mil/GandG/geotrans/geotrans.html
                    It can work in batch mode.

                    Julian M. Burgos

                    University of Washington
                    School of Aquatic and Fishery Sciences
                    1122 NE Boat St. (Box 355020)
                    Seattle, WA 98195-5020

                    ----- Original Message -----
                    From: "Susan King" <sking01@...>
                    To: <ai-geostats@...>
                    Sent: Friday, October 24, 2003 12:27 PM
                    Subject: Re: AI-GEOSTATS: geoR and memory


                    >
                    >
                    >
                    >
                    > I would like to have algorithms for converting lat/long into different
                    > projections (albers, utm).
                    >
                    >
                    >
                    >
                    > Susan L. King
                    > Operations Research Analyst
                    > USDA Forest Service
                    > 11 Campus Blvd., Suite 200
                    > Newtown Square, PA 19073
                    > Phone: (610) 557-4048
                    > Email: sking01@...
                    > FAX: (610) 557-4250
                    >
                    >
                    >
                    > Isobel Clark
                    > <drisobelclark@y To: Martin Roseveare
                    <martin_roseveare@...>
                    > ahoo.co.uk> cc:
                    ai-geostats@...
                    > Sent by: Subject: Re: AI-GEOSTATS:
                    geoR and memory
                    > ai-geostats-list
                    > @...
                    >

                    >
                    > 10/24/2003 11:06
                    > AM
                    > Please respond
                    > to Isobel Clark
                    >
                    >
                    >
                    >
                    >
                    >
                    > Martin
                    >
                    > > - convert your longitude
                    > > and latitude data into a planar projection first
                    > Do you have an algorithm that works everywhere in the
                    > world?
                    >
                    > Isobel
                    > http://geoecosse.bizland.com/whatsnew.htm
                    >
                    > ________________________________________________________________________
                    > Want to chat instantly with your online friends? Get the FREE Yahoo!
                    > Messenger http://mail.messenger.yahoo.co.uk
                    >
                    > --
                    > * To post a message to the list, send it to ai-geostats@...
                    > * As a general service to the users, please remember to post a summary of
                    > any useful responses to your questions.
                    > * To unsubscribe, send an email to majordomo@... with no subject and
                    > "unsubscribe ai-geostats" followed by "end" on the next line in the
                    message
                    > body. DO NOT SEND Subscribe/Unsubscribe requests to the list
                    > * Support to the list is provided at http://www.ai-geostats.org
                    >
                    >
                    >
                    >
                    > --
                    > * To post a message to the list, send it to ai-geostats@...
                    > * As a general service to the users, please remember to post a summary of
                    any useful responses to your questions.
                    > * To unsubscribe, send an email to majordomo@... with no subject and
                    "unsubscribe ai-geostats" followed by "end" on the next line in the message
                    body. DO NOT SEND Subscribe/Unsubscribe requests to the list
                    > * Support to the list is provided at http://www.ai-geostats.org
                    >


                    --
                    * To post a message to the list, send it to ai-geostats@...
                    * As a general service to the users, please remember to post a summary of any useful responses to your questions.
                    * To unsubscribe, send an email to majordomo@... with no subject and "unsubscribe ai-geostats" followed by "end" on the next line in the message body. DO NOT SEND Subscribe/Unsubscribe requests to the list
                    * Support to the list is provided at http://www.ai-geostats.org
                  • Martin Roseveare
                    I ve been passed this link regarding co-ordinate conversion - I haven t tried it so can t comment on it s suitability.
                    Message 9 of 11 , Oct 25, 2003
                    • 0 Attachment
                      I've been passed this link regarding co-ordinate conversion - I haven't
                      tried it so can't comment on it's suitability.

                      http://www.connect.net/jbanta/Downloads.html#cat01

                      Martin
                      ______________________________________

                      ArchaeoPhysica Ltd.
                      Reconnaissance & Geophysics for Archaeology

                      Telephone: +44 (0) 7050 369789
                      E-mail: mail@...
                      Website: http://www.archaeophysica.co.uk
                      ______________________________________

                      This e-mail is intended only for the addressee
                      named above and may contain confidential or
                      privileged information. If you receive this e-mail
                      by mistake please advise the sender and destroy
                      it without further disclosure of its content.

                      Unless otherwise stated no opinions expressed in
                      this e-mail should be regarded as representative of
                      any policy of ArchaeoPhysica Ltd.
                      ----- Original Message -----
                      From: "Jim Bash" <jbash@...>
                      To: "Susan King" <sking01@...>
                      Cc: <ai-geostats@...>; "Cliff Tiedemann" <clifft@...>
                      Sent: Friday, October 24, 2003 10:29 PM
                      Subject: Re: AI-GEOSTATS: geoR and memory


                      >
                      > On Fri, 24 Oct 2003, Susan King wrote:
                      > > I would like to have algorithms for converting lat/long into different
                      > > projections (albers, utm).
                      >
                      > You might check out http://www.remotesensing.org/proj/ which has source
                      > and documentation for PROJ.4, map projection software. I think the source
                      > is in C and you would probably have to dig for the algorithms, unless they
                      > are in the documentation.
                      >
                      > Sorry but I haven't looked at PROJ.4 in a long time. I normally refer to a
                      > book by John Snyder, "Map Projections-- A Working Manual" which was
                      > published by USGS as "USGS Professional Paper 1395" in 1987. But that
                      > isn't online and I'm not sure it is available anymore.
                      >
                      > HTH a little,
                      >
                      > jim b
                      > ------
                      > Jim Bash, Associate Director Internet : JBash@...
                      > CAGIS (m/c 092) Phone (W): 312/996-6367
                      > Program in Geography, Dept of Anthropology Fax : 312/996-6343
                      > University of Illinois at Chicago
                      > 1007 W. Harrison St, Rm 2102 BSB URL :
                      http://www.cagis.uic.edu/
                      > Chicago, IL 60607-7138
                      >
                      > --
                      > * To post a message to the list, send it to ai-geostats@...
                      > * As a general service to the users, please remember to post a summary of
                      any useful responses to your questions.
                      > * To unsubscribe, send an email to majordomo@... with no subject and
                      "unsubscribe ai-geostats" followed by "end" on the next line in the message
                      body. DO NOT SEND Subscribe/Unsubscribe requests to the list
                      > * Support to the list is provided at http://www.ai-geostats.org


                      --
                      * To post a message to the list, send it to ai-geostats@...
                      * As a general service to the users, please remember to post a summary of any useful responses to your questions.
                      * To unsubscribe, send an email to majordomo@... with no subject and "unsubscribe ai-geostats" followed by "end" on the next line in the message body. DO NOT SEND Subscribe/Unsubscribe requests to the list
                      * Support to the list is provided at http://www.ai-geostats.org
                    Your message has been successfully submitted and would be delivered to recipients shortly.