Loading ...
Sorry, an error occurred while loading the content.

LotW Backlog continues to grow ...

Expand Messages
  • Joe Subich, W4TV
    LotW backlog: Upload Processed Backlog 2012-11-28 22:32 2012-12-05 02:08 6 days 05 Hr 36 min 2012-11-29 18:38 2012-12-06 05:10 6 days 10 hr 32 min
    Message 1 of 53 , Dec 7, 2012
    • 0 Attachment
      LotW backlog:

      Upload Processed Backlog
      2012-11-28 22:32 2012-12-05 02:08 6 days 05 Hr 36 min
      2012-11-29 18:38 2012-12-06 05:10 6 days 10 hr 32 min
      2012-11-30 00:58 2012-12-06 18:24 6 days 17 hr 26 min
      2012-11-30 17:04 2012-12-07 17:28 7 days 0 hr 24 min

      LotW has now broken *one week* in delay. As of last Friday LotW was
      receiving 50% more uploads than it was processing each day.

      Although spot checks of the LotW statistics (increase in the number of
      QSO records in a timed 10 or 30 second interval) seem to indicate that
      LotW is processing QSOs at a faster rate - perhaps as many as 3600 Q/hr
      - the system still appears to be critically overloaded to a degree not
      resolvable by the simple storage system upgrade ARRL have announced is
      "coming soon."

      I do not know what hardware is being used for Logbook but it appears
      that the system is compute bound and should be completely redesigned
      for operation on a multi-processor ("blade") server where uploads
      could be processed in parallel to enter the data into the database
      as quickly as possible without concern for "matching." Once the input
      load dropped to a sustainable level, some processors could then switch
      to QSL matching functions working down the list of "new" entries.

      --

      73,

      ... Joe, W4TV
    • Kenneth Grimm
      ... Thanks for the link, Bob. Enterprise decisions concerning SSDs are the same as I went through when deciding to use them. 8*) Speaking of enterprise...do
      Message 53 of 53 , Dec 11, 2012
      • 0 Attachment
        On Tue, Dec 11, 2012 at 2:33 AM, Robert Chudek - K0RC <k0rc@...> wrote:


        It is certainly a sign of competence and design skills when you realize the LoTW system was deployed on SAN technology 10+ years ago. That was leading edge architecture at the time, and unlikely to be found outside of corporate server farms or large data centers.

        For anyone interested in considerations the regarding the marriage of SANs and Database architecture, here's an article describing the issues facing system designers. It's a pretty decent primer on some basic database design too.

        http://tinyurl.com/7bpmt64

        73 de Bob - KØRC in MN



        Thanks for the link, Bob.  Enterprise decisions concerning SSDs are the same as I went through when deciding to use them.  8*)  

        Speaking of enterprise...do you know of any studies comparing MTBF for consumer vs enterprise SSDs?

        73,

        --
        Ken - K4XL
        BoatAnchor Manual Archive
        BAMA - http://bama.edebris.com

      Your message has been successfully submitted and would be delivered to recipients shortly.