Loading ...
Sorry, an error occurred while loading the content.

also, offsite cache?

Expand Messages
  • Nathan Hawks
    Another thought, what about a cache? - Script that uploads cached pages offsite after the thomas grab - Instead of the Site Offline message, do a JS pull from
    Message 1 of 4 , Jul 27, 2007
    • 0 Attachment
      Another thought, what about a cache?
      - Script that uploads cached pages offsite after the thomas grab
      - Instead of the Site Offline message, do a JS pull from the offsite
      cache
      - Would have to insert a big message div at top saying that this was
      cached at date X time Y and dynamic content will be available again
      ASAP.
    • Joshua Tauberer / GovTrack.us
      ... You re talking about caching GovTrack pages, right? Mostly, I hope to just fix the problem... Hopefully some improvements will be made in the next few
      Message 2 of 4 , Jul 27, 2007
      • 0 Attachment
        Nathan Hawks wrote:
        > Another thought, what about a cache?
        > - Script that uploads cached pages offsite after the thomas grab
        > - Instead of the Site Offline message, do a JS pull from the offsite
        > cache
        > - Would have to insert a big message div at top saying that this was
        > cached at date X time Y and dynamic content will be available again

        You're talking about caching GovTrack pages, right?

        Mostly, I hope to just fix the problem... Hopefully some improvements
        will be made in the next few days.

        But, for caching, it's very problematic because there are thousands of
        potential pages GovTrack could serve. Most don't ever get requested.
        5,000 subject terms, some 50,000 bills, the thousands of people that
        have ever served in Congress, etc. (It wouldn't be impossible, but it's
        not trivial.)

        - Josh
      • Nathan Hawks
        A rolling cache, perhaps? Everything that ever gets clicked on, each time it s updated, gets submitted to the morning upload queue? Sans wikis and
        Message 3 of 4 , Jul 27, 2007
        • 0 Attachment
          A rolling cache, perhaps? Everything that ever gets clicked on, each
          time it's updated, gets submitted to the morning upload queue? Sans
          wikis and purely-user-generated content, of course.

          Even once you fix the underlying problem, it'd be nice to imagine and
          work for a day when tens of millions of people are hitting the page 100+
          times a day :)

          I agree, probably not trivial.

          In any event good luck with the bughunting :)

          Nathan

          On Fri, 2007-07-27 at 12:52 -0400, Joshua Tauberer / GovTrack.us wrote:
          > Nathan Hawks wrote:
          > > Another thought, what about a cache?
          > > - Script that uploads cached pages offsite after the thomas grab
          > > - Instead of the Site Offline message, do a JS pull from the offsite
          > > cache
          > > - Would have to insert a big message div at top saying that this was
          > > cached at date X time Y and dynamic content will be available again
          >
          > You're talking about caching GovTrack pages, right?
          >
          > Mostly, I hope to just fix the problem... Hopefully some improvements
          > will be made in the next few days.
          >
          > But, for caching, it's very problematic because there are thousands
          > of
          > potential pages GovTrack could serve. Most don't ever get requested.
          > 5,000 subject terms, some 50,000 bills, the thousands of people that
          > have ever served in Congress, etc. (It wouldn't be impossible, but
          > it's
          > not trivial.)
          >
          > - Josh
          >
          >
          >
          >
        • Josh Tauberer
          ... Actually, I already cache pages that have no user-specific content (if the user comes with no GovTrack cookies set), and I wonder if I could just spew
          Message 4 of 4 , Aug 3, 2007
          • 0 Attachment
            Nathan Hawks wrote:
            > Another thought, what about a cache?
            > - Script that uploads cached pages offsite after the thomas grab
            > - Instead of the Site Offline message, do a JS pull from the offsite
            > cache
            > - Would have to insert a big message div at top saying that this was
            > cached at date X time Y and dynamic content will be available again
            > ASAP.

            Actually, I already cache pages that have no user-specific content (if
            the user comes with no GovTrack cookies set), and I wonder if I could
            just spew those back during down time. That's an interesting idea.

            OTOH, either today was a light day, or upgrading to the latest Mono
            fixed something. There haven't been many server issues today.
            Nevertheless, next week I plan to buy a better server and will probably
            have the site moved over in a few weeks. (These days the advertising on
            the site generates enough money to pay a normal price for a good
            dedicated server.)

            --
            - Josh Tauberer

            http://razor.occams.info

            "Yields falsehood when preceded by its quotation! Yields
            falsehood when preceded by its quotation!" Achilles to
            Tortoise (in "Gödel, Escher, Bach" by Douglas Hofstadter)
          Your message has been successfully submitted and would be delivered to recipients shortly.