Loading ...
Sorry, an error occurred while loading the content.

Re: [govtrack] discrepancies w/ Thomas on #s of co-sponsors on bills

Expand Messages
  • Derek Willis
    FWIW, here s how we handle this issue: We use THOMAS browse by type feature, which displays pages like this (this one contains HR 1207)
    Message 1 of 4 , Apr 28 7:23 PM
    View Source
    • 0 Attachment
      FWIW, here's how we handle this issue:

      We use THOMAS' browse by type feature, which displays pages like this
      (this one contains HR 1207)

      http://thomas.loc.gov/cgi-bin/bdquery/d?d111:1200:./list/bss/d111HR.lst:[[o]]&items=100&|TOM:/bss/111search.html|

      And grab our basic bill information, including number of cosponsors,
      for those bills. We do this twice a day, and it doesn't take very long
      since we're grabbing 100 bills' details at a time rather than loading
      each bill.

      Derek

      On Tue, Apr 28, 2009 at 5:27 PM, Josh Tauberer <tauberer@...> wrote:
      >
      >
      > There's no direct way to determine when a bill has a cosponsorship
      > change, and there are too many bills to scan all of them. (Fetching
      > pages at the maximum limit around the clock might work and would hit
      > every bill once every day or so, but I prefer not to hammer THOMAS.)
      >
      > Someone suggested to me that it might be possible to figure out what to
      > update by scanning the congressional record.
      >
      > So if it's important, that's where you can look to start hacking!
      >
      > (And, yes, any issues like this should be posted here.)
      >
      > Josh
      >
      > David Breakdance wrote:
      >> Hi Josh,
      >>
      >> David here. I believe this is the place where you requested we post
      >> issues, and if not, please correct me, thanks in advance.
      >>
      >> I believe this is a known issue, something I understand has been examined
      >> before and which likely occurs as a result of changes on Thomas' end: the #s
      >> of co-sponsors of certain bills differs b/w Thomas & GovTrack (and hence
      >> OpenCongress, of course). At a quick-fast first glance it occurs in 7 of our
      >> 24 most-viewed bills, so it's not every bill, just some, and there's no
      >> immediately determinate reason as to why that I see.
      >>
      >> Will paste examples below for reference-- I'm pasting a number of examples
      >> in a simple effort to help pinpoint the problem, trying to share what we've
      >> found, you know how it goes. I realize Thomas permalinks are problematic and
      >> generally don't work, but I'll post them here anyway for reference.
      >>
      >> H.R. 1207, Thomas shows 84 ::
      >>
      >> http://thomas.loc.gov/cgi-bin/bdquery/z?d111:h.r.01207
      >>
      >> ... GovTrack shows 55 ::
      >>
      >> http://www.govtrack.us/congress/bill.xpd?bill=h111-1207
      >>
      >> ###
      >>
      >> S.482, Thomas shows 37 ::
      >>
      >> http://thomas.loc.gov/cgi-bin/bdquery/z?d111:s482:
      >>
      >> ... GovTrack shows 33 ::
      >>
      >> http://www.govtrack.us/congress/bill.xpd?bill=s111-482
      >>
      >> ###
      >>
      >> ... quicker format for the rest ::
      >>
      >> H.R.1409 GT is -1
      >> H.R.875 GT is +1
      >> H.R.1960 GT is -5
      >> H.R.1841 GT is -2
      >> S.21 GT is -1
      >>
      >> ###
      >>
      >> ... hope this info helps kick off a process towards a fix, let me know if
      >> we can help in any specific way take a look at anything, and as I mentioned
      >> before, realize it's likely due to changes on Thomas' end, not yours. Call
      >> or write anytime for help, happy to kick in.
      >>
      >> Thanks very much!
      >> -David
      >>
      >> drm@...
      >>
      >>
      >>
      >>
      >>
      >> ------------------------------------
      >>
      >> Yahoo! Groups Links
      >>
      >>
      >>
      >



      --
      --
      Derek Willis
      dwillis@...
    • Josh Tauberer
      Thanks, Derek. That s great. I m implementing that now. Still, if a cosponsor is added while another is removed so the total doesn t change, we still can t
      Message 2 of 4 , Apr 29 4:13 AM
      View Source
      • 0 Attachment
        Thanks, Derek. That's great. I'm implementing that now.

        Still, if a cosponsor is added while another is removed so the total
        doesn't change, we still can't pick it up, but this is much better.

        There is also other bill metadata that isn't reflected in the summary on
        the list pages that won't necessarily get updated when it changes- when
        subject terms are assigned, new summaries, etc.

        - Josh Tauberer
        - GovTrack.us

        http://razor.occams.info

        "Yields falsehood when preceded by its quotation! Yields
        falsehood when preceded by its quotation!" Achilles to
        Tortoise (in "Godel, Escher, Bach" by Douglas Hofstadter)

        On 04/28/2009 10:23 PM, Derek Willis wrote:
        > FWIW, here's how we handle this issue:
        >
        > We use THOMAS' browse by type feature, which displays pages like this
        > (this one contains HR 1207)
        >
        > http://thomas.loc.gov/cgi-bin/bdquery/d?d111:1200:./list/bss/d111HR.lst:[[o]]&items=100&|TOM:/bss/111search.html|
        >
        > And grab our basic bill information, including number of cosponsors,
        > for those bills. We do this twice a day, and it doesn't take very long
        > since we're grabbing 100 bills' details at a time rather than loading
        > each bill.
        >
        > Derek
        >
        > On Tue, Apr 28, 2009 at 5:27 PM, Josh Tauberer<tauberer@...> wrote:
        >>
        >> There's no direct way to determine when a bill has a cosponsorship
        >> change, and there are too many bills to scan all of them. (Fetching
        >> pages at the maximum limit around the clock might work and would hit
        >> every bill once every day or so, but I prefer not to hammer THOMAS.)
        >>
        >> Someone suggested to me that it might be possible to figure out what to
        >> update by scanning the congressional record.
        >>
        >> So if it's important, that's where you can look to start hacking!
        >>
        >> (And, yes, any issues like this should be posted here.)
        >>
        >> Josh
        >>
        >> David Breakdance wrote:
        >>> Hi Josh,
        >>>
        >>> David here. I believe this is the place where you requested we post
        >>> issues, and if not, please correct me, thanks in advance.
        >>>
        >>> I believe this is a known issue, something I understand has been examined
        >>> before and which likely occurs as a result of changes on Thomas' end: the #s
        >>> of co-sponsors of certain bills differs b/w Thomas& GovTrack (and hence
        >>> OpenCongress, of course). At a quick-fast first glance it occurs in 7 of our
        >>> 24 most-viewed bills, so it's not every bill, just some, and there's no
        >>> immediately determinate reason as to why that I see.
        >>>
        >>> Will paste examples below for reference-- I'm pasting a number of examples
        >>> in a simple effort to help pinpoint the problem, trying to share what we've
        >>> found, you know how it goes. I realize Thomas permalinks are problematic and
        >>> generally don't work, but I'll post them here anyway for reference.
        >>>
        >>> H.R. 1207, Thomas shows 84 ::
        >>>
        >>> http://thomas.loc.gov/cgi-bin/bdquery/z?d111:h.r.01207
        >>>
        >>> ... GovTrack shows 55 ::
        >>>
        >>> http://www.govtrack.us/congress/bill.xpd?bill=h111-1207
        >>>
        >>> ###
        >>>
        >>> S.482, Thomas shows 37 ::
        >>>
        >>> http://thomas.loc.gov/cgi-bin/bdquery/z?d111:s482:
        >>>
        >>> ... GovTrack shows 33 ::
        >>>
        >>> http://www.govtrack.us/congress/bill.xpd?bill=s111-482
        >>>
        >>> ###
        >>>
        >>> ... quicker format for the rest ::
        >>>
        >>> H.R.1409 GT is -1
        >>> H.R.875 GT is +1
        >>> H.R.1960 GT is -5
        >>> H.R.1841 GT is -2
        >>> S.21 GT is -1
        >>>
        >>> ###
        >>>
        >>> ... hope this info helps kick off a process towards a fix, let me know if
        >>> we can help in any specific way take a look at anything, and as I mentioned
        >>> before, realize it's likely due to changes on Thomas' end, not yours. Call
        >>> or write anytime for help, happy to kick in.
        >>>
        >>> Thanks very much!
        >>> -David
        >>>
        >>> drm@...
        >>>
        >>>
        >>>
        >>>
        >>>
        >>> ------------------------------------
        >>>
        >>> Yahoo! Groups Links
        >>>
        >>>
        >>>
        >
        >
        >
      Your message has been successfully submitted and would be delivered to recipients shortly.