Loading ...
Sorry, an error occurred while loading the content.
 

Re: [AI-GEOSTATS: MSE to compare different methods]

Expand Messages
  • Pierre Goovaerts
    Hi Mercedes, I fully agree with Gregoire s suggestions of performing a series of jackknifes over a range of sampling densities. In this way, you account for
    Message 1 of 2 , Jan 7 7:45 AM
      Hi Mercedes,

      I fully agree with Gregoire's suggestions of performing
      a series of jackknifes over a range of sampling densities.
      In this way, you account for both the impact of sampling
      density and sampling fluctuations in the comparison.
      An example of this approach can be found in:
      Saito, H. and P. Goovaerts. 2000.
      Geostatistical interpolation of positively skewed and censored data
      in a dioxin contaminated site.
      Environmental Science & Technology, vol.34, No.19: 4228-4235.

      I can e-mail you a PDF copy of the paper if you like.

      Cheers,

      Pierre
      <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

      ________ ________
      | \ / | Pierre Goovaerts
      |_ \ / _| Assistant professor
      __|________\/________|__ Dept of Civil & Environmental Engineering
      | | The University of Michigan
      | M I C H I G A N | EWRE Building, Room 117
      |________________________| Ann Arbor, Michigan, 48109-2125, U.S.A
      _| |_\ /_| |_
      | |\ /| | E-mail: goovaert@...
      |________| \/ |________| Phone: (734) 936-0141
      Fax: (734) 763-2275
      http://www-personal.engin.umich.edu/~goovaert/

      <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>


      On 7 Jan 2001, Gregoire Dubois wrote:

      > Dear Mercedes,
      >
      > doing k fold cross validation (taking out X % of the samples) will not give
      > you any reliable results unless you repeat the operation several times. Taking
      > out 15% of the samples one time only will give you an MSE that will depend
      > strongly on the data you have removed. Has the selection of the 15% been made
      > randomly? You may get a strong bias if the 15% of the samples have been taken
      > in one region in particular or if you have taken out extreme values only. At
      > this stage, I would trust more the results obtained by standard cross
      > validation (leave one out method).
      >
      > I didn’t check your previous mail but if you have few samples only,
      > k-fold cross validation won’t help you much.
      >
      > If you have many samples, then you should repeat the procedure at least 10
      > times to be sure that the way you have extracted the data has not influenced
      > too much the results.
      > Also, if you have a phenomenon that fluctuates at different scales, you may
      > have removed the short scale effect by taking out only few samples (15% is not
      > much).
      >
      > My suggestion is the following: it is time consuming but might be worth the
      > effort. The idea is to take out an increasing number of samples (10, 20, 30,
      > 40, 50, 60, ...,X%) of samples, this 10 times, and see how the average MSE
      > evolves. You may find out that methods A & B work better than C & D when only
      > few samples are removed and that C & D give better results than A & B when
      > more than 40% of the samples have been removed. This would mean that C & D
      > describe better the general trend of the phenomenon while A & B are more
      > sensitive to the local structures (since you have more dense data).
      >
      > If you don’t have the time to proceed in such a way, you should use standard
      > cross validation only and investigate the regions/samples where you have the
      > highest errors.
      >
      > Just few thoughts.
      >
      > Gregoire
      >
      > "Berterretche, Mercedes" <Mercedes.Berterretche@...> wrote:
      > >
      > > I would like to thank Benjamin Warr for his siggestion about doing
      > > difference images instead of global measures as MSE.
      > >
      > > I'm confused because crossvalidation MSE (taking one sample out and
      > > recalculating) and validation MSE (taking 15 percent of the samples out and
      > > recalculating) are giving me opposite results. The validation method would
      > > allows me to compare kriging vs cokriging vs Kriging with an external drift
      > > vs regression , but I don't know if I can trust the results at this point.
      > >
      > > Does anybody have any input about this?
      > > Thanks in advance,
      > > Mercedes Berterretche
      > >
      > > --
      > > * To post a message to the list, send it to ai-geostats@...
      > > * As a general service to the users, please remember to post a summary of
      > any useful responses to your questions.
      > > * To unsubscribe, send an email to majordomo@... with no subject and
      > "unsubscribe ai-geostats" followed by "end" on the next line in the message
      > body. DO NOT SEND Subscribe/Unsubscribe requests to the list
      > > * Support to the list is provided at http://www.ai-geostats.org
      >
      >
      > Gregoire Dubois (Ph.D.)
      > Institute of Mineralogy and Petrography
      > Dept. of Earth Sciences
      > University of Lausanne
      > Switzerland
      >
      > http://www.ai-geostats.org
      >
      > ____________________________________________________________________
      > Get free email and a permanent address at http://www.netaddress.com/?N=1
      >
      > --
      > * To post a message to the list, send it to ai-geostats@...
      > * As a general service to the users, please remember to post a summary of any useful responses to your questions.
      > * To unsubscribe, send an email to majordomo@... with no subject and "unsubscribe ai-geostats" followed by "end" on the next line in the message body. DO NOT SEND Subscribe/Unsubscribe requests to the list
      > * Support to the list is provided at http://www.ai-geostats.org
      >


      --
      * To post a message to the list, send it to ai-geostats@...
      * As a general service to the users, please remember to post a summary of any useful responses to your questions.
      * To unsubscribe, send an email to majordomo@... with no subject and "unsubscribe ai-geostats" followed by "end" on the next line in the message body. DO NOT SEND Subscribe/Unsubscribe requests to the list
      * Support to the list is provided at http://www.ai-geostats.org
    Your message has been successfully submitted and would be delivered to recipients shortly.