Loading ...
Sorry, an error occurred while loading the content.

Re: [govtrack] Re: What is the best way to fetch votes?

Expand Messages
  • Neil Drumm
    ... Yes, I had been using this previously, but have removed dependence on it. Right now everything is run over HTTP. The main advantage is less to set up and
    Message 1 of 5 , Oct 24, 2007
    • 0 Attachment
      On 10/24/07, Josh Tauberer <tauberer@...> wrote:
      >
      > Kevin Henry wrote:
      > > I think the easiest way to keep up to date is just to use rsync for
      > > both the bill and vote directories - it will automatically check the
      > > (file) dates and only download the files that have changed. See
      > > http://www.govtrack.us/source.xpd.
      >
      > Right.
      >
      > For reference, the file data/us/110/votes.all.index.xml is in danger of
      > being dropped. I dropped it in January, brought it back, and now I don't
      > use it anymore so I will probably get rid of it again.

      Yes, I had been using this previously, but have removed dependence on it.

      Right now everything is run over HTTP. The main advantage is less to
      set up and less that might fail. Moving to keeping a local copy on
      rsync is possible, but a bit of effort and a different set of
      advantages and disadvantages.

      I suppose an alternative might be parsing
      http://www.govtrack.us/data/us/110/rolls/ periodically.

      > > There are some files that don't seem to be managed by the rsync server
      > > (like bills.technorati.xml), so for those I just use wget, which also
      > > checks the date before downloading.
      >
      > That's weird. Everything in /data should work under rsync. Are you sure
      > something is wrong there?
      >
      > > And if you're doing anything with the people data, make sure you're
      > > also wgetting people.xml in your daily update. I forgot about this at
      > > first and slowly got out of sync.
      >
      > Very true.

      We have this automated, but not scheduled. I was thinking of doing it weekly.

      --
      Neil Drumm
      http://delocalizedham.com
    • Kevin Henry
      ... You re probably right, it was many months ago that I ran into a problem and switched to wget, and I haven t tried it since. Kevin
      Message 2 of 5 , Oct 24, 2007
      • 0 Attachment
        > > There are some files that don't seem to be managed by the rsync server
        > > (like bills.technorati.xml), so for those I just use wget, which also
        > > checks the date before downloading.
        >
        > That's weird. Everything in /data should work under rsync. Are you sure
        > something is wrong there?

        You're probably right, it was many months ago that I ran into a
        problem and switched to wget, and I haven't tried it since.


        Kevin
        http://www.whereabill.org/
      Your message has been successfully submitted and would be delivered to recipients shortly.