Loading ...
Sorry, an error occurred while loading the content.
 

Predicting balloon flights

Expand Messages
  • L. Paul Verhage
    Resending the Power Point
    Message 1 of 2 , Jul 14, 2013

    Resending the Power Point

  • Anthony Stirk
    Hi there, This was interesting reading. I though it may be worth adding to this. Daniel Richman has recently rewritten the CUSF Predictor. We have been having
    Message 2 of 2 , Jul 14, 2013
      Hi there,

      This was interesting reading. I though it may be worth adding to this. Daniel Richman has recently rewritten the CUSF Predictor. We have been having lots of issues requesting data sets from NOAA so we've decided just to download all of the GFS model four times daily. As a result the predictor is now pretty much instantaneous and it does work worldwide : http://predict.habhub.org/

      Daniels mail to UKHAS here :

      Hi everyone,

      TL;DR: the predictor now works, and works very quickly at that!
      http://predict.habhub.org (the old address redirects). While I've been
      quite careful to make sure it behaves exactly the same as the old
      predictor, it's a mix of old and relatively new software so should
      perhaps be regarded as beta.
      (It's also maybe worth reminding you that no guarantee is given for
      the accuracy of any data included or produced by this program, use it
      at your own risk.)


      (not TL;DR - What follows is a brief tale of the problems of prediction:)

      As you've no doubt aware, the predictor has been a little bit broken
      over the last week.
      This was, as almost every problem with the predictor is, due to the
      NOAA wind data servers. (Though, we are of course very grateful for
      them providing wind data for free :-).)

      The predictor used to use their OPeNDAP servers. It's basically a Java
      (and therefore slow) web application; we would ask it "can I have wind
      data in this latitude/longitude range please". We've had some minor
      issues with them (e.g., "blah is not an available dataset" due to its
      domain resolving to two IP addresses with a very low TTL), but it was
      generally OK. However, over the last week they've become too slow for
      the predictor to get any wind data at all.

      It turns out you can just wget the the entire forecast off a NOAA
      HTTP/FTP server - so that's what we're doing now. Every 6 hours a
      daemon downloads ~6-7GB - the entire forecast - and decompresses it
      for the predictor to use. As a result, predictions complete
      near-instantly; sometimes taking a second or two to bring the data
      from disk into RAM, or if you're lucky and it's cached, about 20
      milliseconds.

      I've upgraded the predictor and spacenearus' live predictions. I will
      upgrade the hourly predictor soon (though am going on holiday
      shortly...), however, the 8GB download may be prohibitive to people
      running their own hourly predictors. Luckily, since we now have
      worldwide data on the habhub servers, we can start running hourly
      predictions for anyone that wants them ourselves. I'll email when this
      is ready.

      It's worth noting that if you want a copy of the dataset yourself, it
      might be better if we mirror it for you. Several benefits:

       - the habhub.org servers (perhaps owing to us having gigabit, maybe
      due to less load) are faster than the NOAA HTTP server
       - only 25% of the 8GB download is actually useful, so we can mirror a
      compressed 1.8G file.
       - reduce load on the NOAA servers, because we're nice people

      I'll have to double check that this is legal first. Let us know
      (#habhub FreeNode, ukhas-tools@...) if you're interested.

      This upgrade was actually part of larger CUSpaceflight work on
      rewriting the predictor (replacing bits one at a time), which was
      going to happen later; replacement of the wind downloading bit was
      brought forward since the current one wasn't working.

      Thanks,
      Daniel


      On Sun, Jul 14, 2013 at 3:49 PM, L. Paul Verhage <nearsys@...> wrote:
       
      [Attachment(s) from L. Paul Verhage included below]

      Resending the Power Point


    Your message has been successfully submitted and would be delivered to recipients shortly.