Loading ...
Sorry, an error occurred while loading the content.

GovTrack API (experimental)

Expand Messages
  • Josh Tauberer
    I ve put this off just about long enough- GovTrack now has an API! Currently bills and people are included in the API. (Votes and committees will come later if
    Message 1 of 9 , May 16, 2012
    • 0 Attachment
      I've put this off just about long enough-

      GovTrack now has an API!

      Currently bills and people are included in the API. (Votes and
      committees will come later if there's interest.)

      More here:

      http://www.govtrack.us/developers/api

      It's experimental at the moment. I don't have any caching on it, no API
      keys, etc. Go easy!

      --
      - Josh Tauberer (@JoshData)

      http://razor.occams.info
    • Eric Mill
      Super awesome, really glad to see this happen. Nice job with the docs, too (and a nice reminder to redo and update my own). Are you using this in anything
      Message 2 of 9 , May 16, 2012
      • 0 Attachment
        Super awesome, really glad to see this happen. Nice job with the docs, too (and a nice reminder to redo and update my own).

        Are you using this in anything yourself now?

        -- Eric

        On Wed, May 16, 2012 at 9:07 PM, Josh Tauberer <tauberer@...> wrote:
        I've put this off just about long enough-

        GovTrack now has an API!

        Currently bills and people are included in the API. (Votes and
        committees will come later if there's interest.)

        More here:

        http://www.govtrack.us/developers/api

        It's experimental at the moment. I don't have any caching on it, no API
        keys, etc. Go easy!

        --
        - Josh Tauberer (@JoshData)

        http://razor.occams.info


        ------------------------------------

        Yahoo! Groups Links

        <*> To visit your group on the web, go to:
           http://groups.yahoo.com/group/govtrack/

        <*> Your email settings:
           Individual Email | Traditional

        <*> To change settings online go to:
           http://groups.yahoo.com/group/govtrack/join
           (Yahoo! ID required)

        <*> To change settings via email:
           govtrack-digest@yahoogroups.com
           govtrack-fullfeatured@yahoogroups.com

        <*> To unsubscribe from this group, send an email to:
           govtrack-unsubscribe@yahoogroups.com

        <*> Your use of Yahoo! Groups is subject to:
           http://docs.yahoo.com/info/terms/




        --

      • Josh Tauberer
        ... Thanks. I drew on a few ideas from RTC, e.g. I made sure the infrastructure I chose (Tasypie) supported __[operator] filtering. ... Not yet. The main
        Message 3 of 9 , May 17, 2012
        • 0 Attachment
          On 05/16/2012 11:39 PM, Eric Mill wrote:
          > Super awesome, really glad to see this happen. Nice job with the docs,
          > too (and a nice reminder to redo and update my own).

          Thanks. I drew on a few ideas from RTC, e.g. I made sure the
          infrastructure I chose (Tasypie) supported __[operator] filtering.

          > Are you using this in anything yourself now?

          Not yet. The main reason I made it is so when people come to me asking
          for CSV exports I have a way to pull them quickly. If people want to
          build actual apps, even better. (Only half joking.)

          Except that the API is tightly coupled to the site's models, so I use it
          at that level. (Makes it practically self-documenting too.)


          - Josh Tauberer (@JoshData)

          http://razor.occams.info

          On 05/16/2012 11:39 PM, Eric Mill wrote:
          >
          >
          > Super awesome, really glad to see this happen. Nice job with the docs,
          > too (and a nice reminder to redo and update my own).
          >
          > Are you using this in anything yourself now?
          >
          > -- Eric
          >
          > On Wed, May 16, 2012 at 9:07 PM, Josh Tauberer <tauberer@...
          > <mailto:tauberer@...>> wrote:
          >
          > I've put this off just about long enough-
          >
          > GovTrack now has an API!
          >
          > Currently bills and people are included in the API. (Votes and
          > committees will come later if there's interest.)
          >
          > More here:
          >
          > http://www.govtrack.us/developers/api
          >
          > It's experimental at the moment. I don't have any caching on it, no API
          > keys, etc. Go easy!
          >
          > --
          > - Josh Tauberer (@JoshData)
          >
          > http://razor.occams.info
          >
          >
          > ------------------------------------
          >
          > Yahoo! Groups Links
          >
          >
          > <mailto:govtrack-fullfeatured@yahoogroups.com>
          >
          >
          >
          >
          >
          > --
          > Developer | sunlightfoundation.com <http://sunlightfoundation.com>
          >
          >
          >
          >
        • Eric Mill
          Sweet. Are you going to dive into text search at all? This has become the most useful thing to come out of RTC, though I haven t documented the search methods
          Message 4 of 9 , May 17, 2012
          • 0 Attachment
            Sweet. Are you going to dive into text search at all? This has become the most useful thing to come out of RTC, though I haven't documented the search methods publicly yet. It's a hard problem, but you're developing all sorts of nifty ways of calculating relevance, so you'd be in a good position to do it well.

            -- Eric

            On Thu, May 17, 2012 at 6:58 AM, Josh Tauberer <tauberer@...> wrote:
            On 05/16/2012 11:39 PM, Eric Mill wrote:
            > Super awesome, really glad to see this happen. Nice job with the docs,
            > too (and a nice reminder to redo and update my own).

            Thanks. I drew on a few ideas from RTC, e.g. I made sure the infrastructure I chose (Tasypie) supported __[operator] filtering.


            > Are you using this in anything yourself now?

            Not yet. The main reason I made it is so when people come to me asking for CSV exports I have a way to pull them quickly. If people want to build actual apps, even better. (Only half joking.)

            Except that the API is tightly coupled to the site's models, so I use it at that level. (Makes it practically self-documenting too.)



            - Josh Tauberer (@JoshData)

            http://razor.occams.info

            On 05/16/2012 11:39 PM, Eric Mill wrote:


            Super awesome, really glad to see this happen. Nice job with the docs,
            too (and a nice reminder to redo and update my own).

            Are you using this in anything yourself now?

            -- Eric

            On Wed, May 16, 2012 at 9:07 PM, Josh Tauberer <tauberer@...
            <mailto:tauberer@...>> wrote:

               I've put this off just about long enough-

               GovTrack now has an API!

               Currently bills and people are included in the API. (Votes and
               committees will come later if there's interest.)

               More here:

               http://www.govtrack.us/developers/api

               It's experimental at the moment. I don't have any caching on it, no API
               keys, etc. Go easy!

               --
               - Josh Tauberer (@JoshData)

               http://razor.occams.info


               ------------------------------------

               Yahoo! Groups Links

               <*> To visit your group on the web, go to:
               http://groups.yahoo.com/group/govtrack/

               <*> Your email settings:
                   Individual Email | Traditional

               <*> To change settings online go to:
               http://groups.yahoo.com/group/govtrack/join
                   (Yahoo! ID required)

               <*> To change settings via email:
               govtrack-digest@yahoogroups.com <mailto:govtrack-digest@yahoogroups.com>
               govtrack-fullfeatured@yahoogroups.com
               <mailto:govtrack-fullfeatured@yahoogroups.com>


               <*> To unsubscribe from this group, send an email to:
               govtrack-unsubscribe@yahoogroups.com
               <mailto:govtrack-unsubscribe@yahoogroups.com>


               <*> Your use of Yahoo! Groups is subject to:
               http://docs.yahoo.com/info/terms/




            --
            Developer | sunlightfoundation.com <http://sunlightfoundation.com>







            --

          • Josh Tauberer
            Probably, yeah. Right now the API is based on the Django ORM, which hits MySQL. That s not great for search so I have to see if I can switch it transparently
            Message 5 of 9 , May 17, 2012
            • 0 Attachment
              Probably, yeah. Right now the API is based on the Django ORM, which hits
              MySQL. That's not great for search so I have to see if I can switch it
              transparently to using Haystack, or else build a separate API for search.

              But document relevance is.... ugh. Two fun facts: The highest-prognosis
              bills are the dumb ones like post office namings, so that's not good for
              relevance. And the top visited bills are almost never the ones people
              should really be looking at, so popularity is bad too. Fun fun.

              - Josh Tauberer (@JoshData)

              http://razor.occams.info

              On 05/17/2012 11:15 AM, Eric Mill wrote:
              >
              >
              > Sweet. Are you going to dive into text search at all? This has become
              > the most useful thing to come out of RTC, though I haven't documented
              > the search methods publicly yet. It's a hard problem, but you're
              > developing all sorts of nifty ways of calculating relevance, so you'd be
              > in a good position to do it well.
              >
              > -- Eric
              >
              > On Thu, May 17, 2012 at 6:58 AM, Josh Tauberer <tauberer@...
              > <mailto:tauberer@...>> wrote:
              >
              > On 05/16/2012 11:39 PM, Eric Mill wrote:
              > > Super awesome, really glad to see this happen. Nice job with the
              > docs,
              > > too (and a nice reminder to redo and update my own).
              >
              > Thanks. I drew on a few ideas from RTC, e.g. I made sure the
              > infrastructure I chose (Tasypie) supported __[operator] filtering.
              >
              >
              > > Are you using this in anything yourself now?
              >
              > Not yet. The main reason I made it is so when people come to me
              > asking for CSV exports I have a way to pull them quickly. If people
              > want to build actual apps, even better. (Only half joking.)
              >
              > Except that the API is tightly coupled to the site's models, so I
              > use it at that level. (Makes it practically self-documenting too.)
              >
              >
              >
              > - Josh Tauberer (@JoshData)
              >
              > http://razor.occams.info
              >
              > On 05/16/2012 11:39 PM, Eric Mill wrote:
              >
              >
              >
              > Super awesome, really glad to see this happen. Nice job with the
              > docs,
              > too (and a nice reminder to redo and update my own).
              >
              > Are you using this in anything yourself now?
              >
              > -- Eric
              >
              > On Wed, May 16, 2012 at 9:07 PM, Josh Tauberer
              > <tauberer@... <mailto:tauberer@...>
              > <mailto:tauberer@... <mailto:tauberer@...>>> wrote:
              >
              > I've put this off just about long enough-
              >
              > GovTrack now has an API!
              >
              > Currently bills and people are included in the API. (Votes and
              > committees will come later if there's interest.)
              >
              > More here:
              >
              > http://www.govtrack.us/__developers/api
              > <http://www.govtrack.us/developers/api>
              >
              > It's experimental at the moment. I don't have any caching on
              > it, no API
              > keys, etc. Go easy!
              >
              > --
              > - Josh Tauberer (@JoshData)
              >
              > http://razor.occams.info
              >
              >
              > ------------------------------__------
              >
              > Yahoo! Groups Links
              >
              >
              > (Yahoo! ID required)
              >
              > <mailto:govtrack-digest@__yahoogroups.com
              > <mailto:govtrack-digest@yahoogroups.com>>
              > govtrack-fullfeatured@__yahoogroups.com
              > <mailto:govtrack-fullfeatured@yahoogroups.com>
              > <mailto:govtrack-fullfeatured@__yahoogroups.com
              > <mailto:govtrack-fullfeatured@yahoogroups.com>>
              >
              >
              > <mailto:govtrack-unsubscribe@__yahoogroups.com
              > <mailto:govtrack-unsubscribe@yahoogroups.com>>
              >
              >
              >
              >
              >
              >
              > --
              > Developer | sunlightfoundation.com
              > <http://sunlightfoundation.com> <http://sunlightfoundation.com__>
              >
              >
              >
              >
              >
              >
              >
              > --
              > Developer | sunlightfoundation.com <http://sunlightfoundation.com>
              >
              >
              >
              >
            • Eric Mill
              So fun. We had a nice (potential) success story at Sunlight the other day here, where we
              Message 6 of 9 , May 17, 2012
              • 0 Attachment
                So fun. 

                We had a nice (potential) success story at Sunlight the other day here, where we caught a nasty proposed FOIA exemption in a bill being actively crafted inside the House through our search engine (second item down on this list, right now: http://scout.sunlightfoundation.com/search/federal_bills/section%20552%20of%20title%205). It has no official summary, at the time it had no official "short title" yet and was a big paragraph of text...and as you can see my highlighter is failing to even provide an excerpt for that result. John had to open up the official text on GPO and dig through the text to find out why it even came up. This was a very relevant bill, but it was only spotted because it was new and because John is a diligent guy.

                It had been announced for floor activity, and was introduced by the committee chair, so maybe those would help in relevance - if you also tuned out the post office renamings. Maybe that's the only way, is just to filter out specific classes of bills and resolutions.

                -- Eric

                On Thu, May 17, 2012 at 11:23 AM, Josh Tauberer <tauberer@...> wrote:
                Probably, yeah. Right now the API is based on the Django ORM, which hits MySQL. That's not great for search so I have to see if I can switch it transparently to using Haystack, or else build a separate API for search.

                But document relevance is.... ugh. Two fun facts: The highest-prognosis bills are the dumb ones like post office namings, so that's not good for relevance. And the top visited bills are almost never the ones people should really be looking at, so popularity is bad too. Fun fun.


                - Josh Tauberer (@JoshData)

                http://razor.occams.info

                On 05/17/2012 11:15 AM, Eric Mill wrote:


                Sweet. Are you going to dive into text search at all? This has become
                the most useful thing to come out of RTC, though I haven't documented
                the search methods publicly yet. It's a hard problem, but you're
                developing all sorts of nifty ways of calculating relevance, so you'd be
                in a good position to do it well.

                -- Eric

                On Thu, May 17, 2012 at 6:58 AM, Josh Tauberer <tauberer@...
                <mailto:tauberer@...>> wrote:

                   On 05/16/2012 11:39 PM, Eric Mill wrote:
                    > Super awesome, really glad to see this happen. Nice job with the
                   docs,
                    > too (and a nice reminder to redo and update my own).

                   Thanks. I drew on a few ideas from RTC, e.g. I made sure the
                   infrastructure I chose (Tasypie) supported __[operator] filtering.


                    > Are you using this in anything yourself now?

                   Not yet. The main reason I made it is so when people come to me
                   asking for CSV exports I have a way to pull them quickly. If people
                   want to build actual apps, even better. (Only half joking.)

                   Except that the API is tightly coupled to the site's models, so I
                   use it at that level. (Makes it practically self-documenting too.)



                   - Josh Tauberer (@JoshData)

                   http://razor.occams.info

                   On 05/16/2012 11:39 PM, Eric Mill wrote:



                       Super awesome, really glad to see this happen. Nice job with the
                       docs,
                       too (and a nice reminder to redo and update my own).

                       Are you using this in anything yourself now?

                       -- Eric

                       On Wed, May 16, 2012 at 9:07 PM, Josh Tauberer
                       <tauberer@... <mailto:tauberer@...>
                       <mailto:tauberer@... <mailto:tauberer@...>>> wrote:

                           I've put this off just about long enough-

                           GovTrack now has an API!

                           Currently bills and people are included in the API. (Votes and
                           committees will come later if there's interest.)

                           More here:

                       http://www.govtrack.us/__developers/api

                       <http://www.govtrack.us/developers/api>

                           It's experimental at the moment. I don't have any caching on
                       it, no API
                           keys, etc. Go easy!

                           --
                           - Josh Tauberer (@JoshData)

                       http://razor.occams.info


                           ------------------------------__------


                           Yahoo! Groups Links

                       <*> To visit your group on the web, go to:
                       http://groups.yahoo.com/group/__govtrack/

                       <http://groups.yahoo.com/group/govtrack/>

                       <*> Your email settings:
                               Individual Email | Traditional

                       <*> To change settings online go to:
                       http://groups.yahoo.com/group/__govtrack/join

                       <http://groups.yahoo.com/group/govtrack/join>
                               (Yahoo! ID required)

                       <*> To change settings via email:
                       govtrack-digest@yahoogroups.__com
                       <mailto:govtrack-digest@yahoogroups.com>
                       <mailto:govtrack-digest@__yahoogroups.com
                       <mailto:govtrack-digest@yahoogroups.com>>
                       govtrack-fullfeatured@__yahoogroups.com
                       <mailto:govtrack-fullfeatured@yahoogroups.com>
                       <mailto:govtrack-fullfeatured@__yahoogroups.com

                       <mailto:govtrack-fullfeatured@yahoogroups.com>>


                       <*> To unsubscribe from this group, send an email to:
                       govtrack-unsubscribe@__yahoogroups.com
                       <mailto:govtrack-unsubscribe@yahoogroups.com>
                       <mailto:govtrack-unsubscribe@__yahoogroups.com

                       <mailto:govtrack-unsubscribe@yahoogroups.com>>


                       <*> Your use of Yahoo! Groups is subject to:
                       http://docs.yahoo.com/info/__terms/

                       <http://docs.yahoo.com/info/terms/>




                       --
                       Developer | sunlightfoundation.com
                       <http://sunlightfoundation.com> <http://sunlightfoundation.com__>



                --

              • Josh Tauberer
                ... Yeah, that is worth a try. I m now sorting bills on the advanced search by a modified-prognosis score that just omits any boost that post office bills and
                Message 7 of 9 , May 23, 2012
                • 0 Attachment
                  On 05/17/2012 11:36 AM, Eric Mill wrote:
                  > if you also
                  > tuned out the post office renamings. Maybe that's the only way, is just
                  > to filter out specific classes of bills and resolutions.

                  Yeah, that is worth a try. I'm now sorting bills on the advanced search
                  by a modified-prognosis score that just omits any boost that post office
                  bills and some other bills get, and also arbitrarily de-weights resolutions.

                  http://www.govtrack.us/congress/bills/browse

                  Think it's more useful than e.g. last action date? Looking at the
                  results I think so but I'm not sure.

                  (I guess I could do A/B testing with users but it's so much trouble to
                  be quantitative.)

                  - Josh Tauberer (@JoshData)

                  http://razor.occams.info

                  On 05/17/2012 11:36 AM, Eric Mill wrote:
                  > So fun.
                  >
                  > We had a nice (potential) success story at Sunlight the other day here,
                  > where we caught
                  > <https://twitter.com/#!/JohnWonderlich/status/202385148161953793> a
                  > nasty proposed FOIA exemption in a bill being actively crafted inside
                  > the House through our search engine (second item down on this list,
                  > right now:
                  > http://scout.sunlightfoundation.com/search/federal_bills/section%20552%20of%20title%205).
                  > It has no official summary, at the time it had no official "short title"
                  > yet and was a big paragraph of text...and as you can see my highlighter
                  > is failing to even provide an excerpt for that result. John had to open
                  > up the official text on GPO and dig through the text to find out why it
                  > even came up. This was a very relevant bill, but it was only spotted
                  > because it was new and because John is a diligent guy.
                  >
                  > It had been announced for floor activity, and was introduced by the
                  > committee chair, so maybe those would help in relevance - if you also
                  > tuned out the post office renamings. Maybe that's the only way, is just
                  > to filter out specific classes of bills and resolutions.
                  >
                  > -- Eric
                  >
                  > On Thu, May 17, 2012 at 11:23 AM, Josh Tauberer <tauberer@...
                  > <mailto:tauberer@...>> wrote:
                  >
                  > Probably, yeah. Right now the API is based on the Django ORM, which
                  > hits MySQL. That's not great for search so I have to see if I can
                  > switch it transparently to using Haystack, or else build a separate
                  > API for search.
                  >
                  > But document relevance is.... ugh. Two fun facts: The
                  > highest-prognosis bills are the dumb ones like post office namings,
                  > so that's not good for relevance. And the top visited bills are
                  > almost never the ones people should really be looking at, so
                  > popularity is bad too. Fun fun.
                  >
                  >
                  > - Josh Tauberer (@JoshData)
                  >
                  > http://razor.occams.info
                  >
                  > On 05/17/2012 11:15 AM, Eric Mill wrote:
                  >
                  >
                  >
                  > Sweet. Are you going to dive into text search at all? This has
                  > become
                  > the most useful thing to come out of RTC, though I haven't
                  > documented
                  > the search methods publicly yet. It's a hard problem, but you're
                  > developing all sorts of nifty ways of calculating relevance, so
                  > you'd be
                  > in a good position to do it well.
                  >
                  > -- Eric
                  >
                  > On Thu, May 17, 2012 at 6:58 AM, Josh Tauberer
                  > <tauberer@... <mailto:tauberer@...>
                  > <mailto:tauberer@... <mailto:tauberer@...>>> wrote:
                  >
                  > On 05/16/2012 11:39 PM, Eric Mill wrote:
                  > > Super awesome, really glad to see this happen. Nice job with the
                  > docs,
                  > > too (and a nice reminder to redo and update my own).
                  >
                  > Thanks. I drew on a few ideas from RTC, e.g. I made sure the
                  > infrastructure I chose (Tasypie) supported __[operator]
                  > filtering.
                  >
                  >
                  > > Are you using this in anything yourself now?
                  >
                  > Not yet. The main reason I made it is so when people come to me
                  > asking for CSV exports I have a way to pull them quickly. If
                  > people
                  > want to build actual apps, even better. (Only half joking.)
                  >
                  > Except that the API is tightly coupled to the site's models,
                  > so I
                  > use it at that level. (Makes it practically self-documenting
                  > too.)
                  >
                  >
                  >
                  > - Josh Tauberer (@JoshData)
                  >
                  > http://razor.occams.info
                  >
                  > On 05/16/2012 11:39 PM, Eric Mill wrote:
                  >
                  >
                  >
                  > Super awesome, really glad to see this happen. Nice job
                  > with the
                  > docs,
                  > too (and a nice reminder to redo and update my own).
                  >
                  > Are you using this in anything yourself now?
                  >
                  > -- Eric
                  >
                  > On Wed, May 16, 2012 at 9:07 PM, Josh Tauberer
                  > <tauberer@... <mailto:tauberer@...>
                  > <mailto:tauberer@... <mailto:tauberer@...>>
                  > <mailto:tauberer@... <mailto:tauberer@...>
                  > <mailto:tauberer@... <mailto:tauberer@...>>>__>
                  > wrote:
                  >
                  > I've put this off just about long enough-
                  >
                  > GovTrack now has an API!
                  >
                  > Currently bills and people are included in the API.
                  > (Votes and
                  > committees will come later if there's interest.)
                  >
                  > More here:
                  >
                  > http://www.govtrack.us/____developers/api
                  > <http://www.govtrack.us/__developers/api>
                  >
                  > <http://www.govtrack.us/__developers/api
                  > <http://www.govtrack.us/developers/api>>
                  >
                  > It's experimental at the moment. I don't have any
                  > caching on
                  > it, no API
                  > keys, etc. Go easy!
                  >
                  > --
                  > - Josh Tauberer (@JoshData)
                  >
                  > http://razor.occams.info
                  >
                  >
                  > ------------------------------____------
                  >
                  >
                  > Yahoo! Groups Links
                  >
                  >
                  > <http://groups.yahoo.com/__group/govtrack/
                  > <http://groups.yahoo.com/group/govtrack/>>
                  >
                  >
                  > <http://groups.yahoo.com/__group/govtrack/join
                  > <http://groups.yahoo.com/group/govtrack/join>>
                  > (Yahoo! ID required)
                  >
                  > <mailto:govtrack-digest@yahoogroups.com>>
                  > <mailto:govtrack-digest@
                  > <mailto:govtrack-digest@>__yaho__ogroups.com
                  > <http://yahoogroups.com>
                  > <mailto:govtrack-digest@__yahoogroups.com
                  > <mailto:govtrack-digest@yahoogroups.com>>>
                  > govtrack-fullfeatured@__yahoog__roups.com
                  > <http://yahoogroups.com>
                  > <mailto:govtrack-fullfeatured@__yahoogroups.com
                  > <mailto:govtrack-fullfeatured@yahoogroups.com>>
                  > <mailto:govtrack-fullfeatured@
                  > <mailto:govtrack-fullfeatured@>____yahoogroups.com
                  > <http://yahoogroups.com>
                  >
                  > <mailto:govtrack-fullfeatured@__yahoogroups.com
                  > <mailto:govtrack-fullfeatured@yahoogroups.com>>>
                  >
                  >
                  > <mailto:govtrack-unsubscribe@__yahoogroups.com
                  > <mailto:govtrack-unsubscribe@yahoogroups.com>>
                  > <mailto:govtrack-unsubscribe@
                  > <mailto:govtrack-unsubscribe@>____yahoogroups.com
                  > <http://yahoogroups.com>
                  >
                  > <mailto:govtrack-unsubscribe@__yahoogroups.com
                  > <mailto:govtrack-unsubscribe@yahoogroups.com>>>
                  >
                  >
                  >
                  > <http://docs.yahoo.com/info/__terms/
                  > <http://docs.yahoo.com/info/terms/>>
                  >
                  >
                  >
                  >
                  > --
                  > Developer | sunlightfoundation.com
                  > <http://sunlightfoundation.com>
                  > <http://sunlightfoundation.com__>
                  > <http://sunlightfoundation.__com__
                  > <http://sunlightfoundation.com__>>
                  >
                  >
                  >
                  >
                  >
                  >
                  >
                  >
                  > --
                  > Developer | sunlightfoundation.com
                  > <http://sunlightfoundation.com> <http://sunlightfoundation.com__>
                  >
                  >
                  >
                  >
                  >
                  >
                  >
                  >
                  > --
                  > Developer | sunlightfoundation.com <http://sunlightfoundation.com>
                  >
                • Eric Mill
                  Sorting on prognosis is a cool idea. Does that mean that bills already passed into law will always occur at the top, or the bottom, or...? You are the last
                  Message 8 of 9 , May 23, 2012
                  • 0 Attachment
                    Sorting on prognosis is a cool idea. Does that mean that bills already passed into law will always occur at the top, or the bottom, or...?

                    You are the last person who needs to hear the definition of resolution types, but just to make sure, you'd probably want to only de-weight plain resolutions and concurrent resolutions, and not joint resolutions (which are bills with the force of law and are signed by the President).

                    On Wed, May 23, 2012 at 6:25 PM, Josh Tauberer <tauberer@...> wrote:
                    On 05/17/2012 11:36 AM, Eric Mill wrote:
                    if you also
                    tuned out the post office renamings. Maybe that's the only way, is just
                    to filter out specific classes of bills and resolutions.

                    Yeah, that is worth a try. I'm now sorting bills on the advanced search by a modified-prognosis score that just omits any boost that post office bills and some other bills get, and also arbitrarily de-weights resolutions.

                    http://www.govtrack.us/congress/bills/browse

                    Think it's more useful than e.g. last action date? Looking at the results I think so but I'm not sure.

                    (I guess I could do A/B testing with users but it's so much trouble to be quantitative.)


                    - Josh Tauberer (@JoshData)

                    http://razor.occams.info

                    On 05/17/2012 11:36 AM, Eric Mill wrote:
                    So fun.

                    We had a nice (potential) success story at Sunlight the other day here,
                    where we caught
                    <https://twitter.com/#!/JohnWonderlich/status/202385148161953793> a

                    nasty proposed FOIA exemption in a bill being actively crafted inside
                    the House through our search engine (second item down on this list,
                    right now:
                    http://scout.sunlightfoundation.com/search/federal_bills/section%20552%20of%20title%205).
                    It has no official summary, at the time it had no official "short title"
                    yet and was a big paragraph of text...and as you can see my highlighter
                    is failing to even provide an excerpt for that result. John had to open
                    up the official text on GPO and dig through the text to find out why it
                    even came up. This was a very relevant bill, but it was only spotted
                    because it was new and because John is a diligent guy.

                    It had been announced for floor activity, and was introduced by the
                    committee chair, so maybe those would help in relevance - if you also
                    tuned out the post office renamings. Maybe that's the only way, is just
                    to filter out specific classes of bills and resolutions.

                    -- Eric

                    On Thu, May 17, 2012 at 11:23 AM, Josh Tauberer <tauberer@...
                    <mailto:tauberer@...>> wrote:

                       Probably, yeah. Right now the API is based on the Django ORM, which
                       hits MySQL. That's not great for search so I have to see if I can
                       switch it transparently to using Haystack, or else build a separate
                       API for search.

                       But document relevance is.... ugh. Two fun facts: The
                       highest-prognosis bills are the dumb ones like post office namings,
                       so that's not good for relevance. And the top visited bills are
                       almost never the ones people should really be looking at, so
                       popularity is bad too. Fun fun.


                       - Josh Tauberer (@JoshData)

                       http://razor.occams.info

                       On 05/17/2012 11:15 AM, Eric Mill wrote:



                           Sweet. Are you going to dive into text search at all? This has
                           become
                           the most useful thing to come out of RTC, though I haven't
                           documented
                           the search methods publicly yet. It's a hard problem, but you're
                           developing all sorts of nifty ways of calculating relevance, so
                           you'd be
                           in a good position to do it well.

                           -- Eric

                           On Thu, May 17, 2012 at 6:58 AM, Josh Tauberer
                           <tauberer@... <mailto:tauberer@...>
                           <mailto:tauberer@... <mailto:tauberer@...>>> wrote:

                               On 05/16/2012 11:39 PM, Eric Mill wrote:
                            > Super awesome, really glad to see this happen. Nice job with the
                               docs,
                            > too (and a nice reminder to redo and update my own).

                               Thanks. I drew on a few ideas from RTC, e.g. I made sure the
                               infrastructure I chose (Tasypie) supported __[operator]
                           filtering.


                            > Are you using this in anything yourself now?

                               Not yet. The main reason I made it is so when people come to me
                               asking for CSV exports I have a way to pull them quickly. If
                           people
                               want to build actual apps, even better. (Only half joking.)

                               Except that the API is tightly coupled to the site's models,
                           so I
                               use it at that level. (Makes it practically self-documenting
                           too.)



                               - Josh Tauberer (@JoshData)

                           http://razor.occams.info

                               On 05/16/2012 11:39 PM, Eric Mill wrote:



                                   Super awesome, really glad to see this happen. Nice job
                           with the
                                   docs,
                                   too (and a nice reminder to redo and update my own).

                                   Are you using this in anything yourself now?

                                   -- Eric

                                   On Wed, May 16, 2012 at 9:07 PM, Josh Tauberer
                           <tauberer@... <mailto:tauberer@...>
                           <mailto:tauberer@... <mailto:tauberer@...>>
                           <mailto:tauberer@... <mailto:tauberer@...>
                           <mailto:tauberer@... <mailto:tauberer@...>>>__>

                           wrote:

                                       I've put this off just about long enough-

                                       GovTrack now has an API!

                                       Currently bills and people are included in the API.
                           (Votes and
                                       committees will come later if there's interest.)

                                       More here:

                           http://www.govtrack.us/____developers/api
                           <http://www.govtrack.us/__developers/api>


                           <http://www.govtrack.us/__developers/api
                           <http://www.govtrack.us/developers/api>>

                                       It's experimental at the moment. I don't have any
                           caching on
                                   it, no API
                                       keys, etc. Go easy!

                                       --
                                       - Josh Tauberer (@JoshData)

                           http://razor.occams.info


                                       ------------------------------____------



                                       Yahoo! Groups Links

                           <*> To visit your group on the web, go to:
                           http://groups.yahoo.com/group/____govtrack/
                           <http://groups.yahoo.com/group/__govtrack/>

                           <http://groups.yahoo.com/__group/govtrack/

                           <http://groups.yahoo.com/group/govtrack/>>

                           <*> Your email settings:
                                           Individual Email | Traditional

                           <*> To change settings online go to:
                           http://groups.yahoo.com/group/____govtrack/join
                           <http://groups.yahoo.com/group/__govtrack/join>

                           <http://groups.yahoo.com/__group/govtrack/join

                           <http://groups.yahoo.com/group/govtrack/join>>
                                           (Yahoo! ID required)

                           <*> To change settings via email:
                                   govtrack-digest@yahoogroups.____com
                           <mailto:govtrack-digest@__yahoogroups.com
                           <mailto:govtrack-digest@yahoogroups.com>>
                           <mailto:govtrack-digest@
                           <mailto:govtrack-digest@>__yaho__ogroups.com
                           <http://yahoogroups.com>
                           <mailto:govtrack-digest@__yahoogroups.com
                           <mailto:govtrack-digest@yahoogroups.com>>>
                                   govtrack-fullfeatured@__yahoog__roups.com
                           <http://yahoogroups.com>
                           <mailto:govtrack-fullfeatured@__yahoogroups.com
                           <mailto:govtrack-fullfeatured@yahoogroups.com>>
                           <mailto:govtrack-fullfeatured@
                           <mailto:govtrack-fullfeatured@>____yahoogroups.com
                           <http://yahoogroups.com>


                           <mailto:govtrack-fullfeatured@__yahoogroups.com
                           <mailto:govtrack-fullfeatured@yahoogroups.com>>>


                           <*> To unsubscribe from this group, send an email to:
                                   govtrack-unsubscribe@__yahoogr__oups.com
                           <http://yahoogroups.com>
                           <mailto:govtrack-unsubscribe@__yahoogroups.com
                           <mailto:govtrack-unsubscribe@yahoogroups.com>>
                           <mailto:govtrack-unsubscribe@
                           <mailto:govtrack-unsubscribe@>____yahoogroups.com
                           <http://yahoogroups.com>


                           <mailto:govtrack-unsubscribe@__yahoogroups.com
                           <mailto:govtrack-unsubscribe@yahoogroups.com>>>


                           <*> Your use of Yahoo! Groups is subject to:
                           http://docs.yahoo.com/info/____terms/
                           <http://docs.yahoo.com/info/__terms/>


                           <http://docs.yahoo.com/info/__terms/
                           <http://docs.yahoo.com/info/terms/>>




                                   --
                                   Developer | sunlightfoundation.com
                           <http://sunlightfoundation.com>
                           <http://sunlightfoundation.com__>
                           <http://sunlightfoundation.__com__

                           <http://sunlightfoundation.com__>>








                           --
                           Developer | sunlightfoundation.com
                           <http://sunlightfoundation.com> <http://sunlightfoundation.com__>



                    --

                  • Josh Tauberer
                    ... Yeah this part is weird. The only status taken into account is whether the bill has been reported by committee. So reported bills get a boost. Enacted
                    Message 9 of 9 , May 23, 2012
                    • 0 Attachment
                      On 05/23/2012 06:37 PM, Eric Mill wrote:
                      > Sorting on prognosis is a cool idea. Does that mean that bills already
                      > passed into law will always occur at the top, or the bottom, or...?

                      Yeah this part is weird. The only status taken into account is whether
                      the bill has been reported by committee. So reported bills get a boost.
                      Enacted bills get the same boost, but no additional boost for being
                      passed/enacted/etc. So everything besides introduced bills get a boost.

                      > you'd probably want to only de-weight
                      > plain resolutions and concurrent resolutions

                      The actual weights are:

                      simple resolution .2
                      concurrent resolution: .3
                      joint resolution: .35
                      bill: 1.0

                      So I probably went too far with the joint resolutions.

                      About half of joint resolutions are proposing an amendment to the
                      constitution (=no force of law). Rather than weighting them all 0.3, I
                      should probably just see why the prognosis doesn't include a factor for
                      constitutional amendment resolutions and weight those accordingly (i.e.
                      none of those ever pass) and, overall, weight joint resolutions the same
                      as bills.

                      Thanks for the reminder about that.

                      - Josh Tauberer (@JoshData)

                      http://razor.occams.info

                      On 05/23/2012 06:37 PM, Eric Mill wrote:
                      > Sorting on prognosis is a cool idea. Does that mean that bills already
                      > passed into law will always occur at the top, or the bottom, or...?
                      >
                      > You are the last person who needs to hear the definition of resolution
                      > types, but just to make sure, you'd probably want to only de-weight
                      > plain resolutions and concurrent resolutions, and not joint resolutions
                      > (which are bills with the force of law and are signed by the President).
                      >
                      > On Wed, May 23, 2012 at 6:25 PM, Josh Tauberer <tauberer@...
                      > <mailto:tauberer@...>> wrote:
                      >
                      > On 05/17/2012 11:36 AM, Eric Mill wrote:
                      >
                      > if you also
                      > tuned out the post office renamings. Maybe that's the only way,
                      > is just
                      > to filter out specific classes of bills and resolutions.
                      >
                      >
                      > Yeah, that is worth a try. I'm now sorting bills on the advanced
                      > search by a modified-prognosis score that just omits any boost that
                      > post office bills and some other bills get, and also arbitrarily
                      > de-weights resolutions.
                      >
                      > http://www.govtrack.us/__congress/bills/browse
                      > <http://www.govtrack.us/congress/bills/browse>
                      >
                      > Think it's more useful than e.g. last action date? Looking at the
                      > results I think so but I'm not sure.
                      >
                      > (I guess I could do A/B testing with users but it's so much trouble
                      > to be quantitative.)
                      >
                      >
                      > - Josh Tauberer (@JoshData)
                      >
                      > http://razor.occams.info
                      >
                      > On 05/17/2012 11:36 AM, Eric Mill wrote:
                      >
                      > So fun.
                      >
                      > We had a nice (potential) success story at Sunlight the other
                      > day here,
                      > where we caught
                      > <https://twitter.com/#!/__JohnWonderlich/status/__202385148161953793
                      > <https://twitter.com/#!/JohnWonderlich/status/202385148161953793>> a
                      >
                      > nasty proposed FOIA exemption in a bill being actively crafted
                      > inside
                      > the House through our search engine (second item down on this list,
                      > right now:
                      > http://scout.__sunlightfoundation.com/search/__federal_bills/section%20552%__20of%20title%205
                      > <http://scout.sunlightfoundation.com/search/federal_bills/section%20552%20of%20title%205>).
                      > It has no official summary, at the time it had no official
                      > "short title"
                      > yet and was a big paragraph of text...and as you can see my
                      > highlighter
                      > is failing to even provide an excerpt for that result. John had
                      > to open
                      > up the official text on GPO and dig through the text to find out
                      > why it
                      > even came up. This was a very relevant bill, but it was only spotted
                      > because it was new and because John is a diligent guy.
                      >
                      > It had been announced for floor activity, and was introduced by the
                      > committee chair, so maybe those would help in relevance - if you
                      > also
                      > tuned out the post office renamings. Maybe that's the only way,
                      > is just
                      > to filter out specific classes of bills and resolutions.
                      >
                      > -- Eric
                      >
                      > On Thu, May 17, 2012 at 11:23 AM, Josh Tauberer
                      > <tauberer@... <mailto:tauberer@...>
                      > <mailto:tauberer@... <mailto:tauberer@...>>> wrote:
                      >
                      > Probably, yeah. Right now the API is based on the Django
                      > ORM, which
                      > hits MySQL. That's not great for search so I have to see if
                      > I can
                      > switch it transparently to using Haystack, or else build a
                      > separate
                      > API for search.
                      >
                      > But document relevance is.... ugh. Two fun facts: The
                      > highest-prognosis bills are the dumb ones like post office
                      > namings,
                      > so that's not good for relevance. And the top visited bills are
                      > almost never the ones people should really be looking at, so
                      > popularity is bad too. Fun fun.
                      >
                      >
                      > - Josh Tauberer (@JoshData)
                      >
                      > http://razor.occams.info
                      >
                      > On 05/17/2012 11:15 AM, Eric Mill wrote:
                      >
                      >
                      >
                      > Sweet. Are you going to dive into text search at all?
                      > This has
                      > become
                      > the most useful thing to come out of RTC, though I haven't
                      > documented
                      > the search methods publicly yet. It's a hard problem,
                      > but you're
                      > developing all sorts of nifty ways of calculating
                      > relevance, so
                      > you'd be
                      > in a good position to do it well.
                      >
                      > -- Eric
                      >
                      > On Thu, May 17, 2012 at 6:58 AM, Josh Tauberer
                      > <tauberer@... <mailto:tauberer@...>
                      > <mailto:tauberer@... <mailto:tauberer@...>>
                      > <mailto:tauberer@... <mailto:tauberer@...>
                      > <mailto:tauberer@... <mailto:tauberer@...>>>__>
                      > wrote:
                      >
                      > On 05/16/2012 11:39 PM, Eric Mill wrote:
                      > > Super awesome, really glad to see this happen. Nice job with the
                      > docs,
                      > > too (and a nice reminder to redo and update my own).
                      >
                      > Thanks. I drew on a few ideas from RTC, e.g. I made
                      > sure the
                      > infrastructure I chose (Tasypie) supported __[operator]
                      > filtering.
                      >
                      >
                      > > Are you using this in anything yourself now?
                      >
                      > Not yet. The main reason I made it is so when people
                      > come to me
                      > asking for CSV exports I have a way to pull them
                      > quickly. If
                      > people
                      > want to build actual apps, even better. (Only half
                      > joking.)
                      >
                      > Except that the API is tightly coupled to the site's
                      > models,
                      > so I
                      > use it at that level. (Makes it practically
                      > self-documenting
                      > too.)
                      >
                      >
                      >
                      > - Josh Tauberer (@JoshData)
                      >
                      > http://razor.occams.info
                      >
                      > On 05/16/2012 11:39 PM, Eric Mill wrote:
                      >
                      >
                      >
                      > Super awesome, really glad to see this happen.
                      > Nice job
                      > with the
                      > docs,
                      > too (and a nice reminder to redo and update my own).
                      >
                      > Are you using this in anything yourself now?
                      >
                      > -- Eric
                      >
                      > On Wed, May 16, 2012 at 9:07 PM, Josh Tauberer
                      > <tauberer@... <mailto:tauberer@...>
                      > <mailto:tauberer@... <mailto:tauberer@...>>
                      > <mailto:tauberer@... <mailto:tauberer@...>
                      > <mailto:tauberer@... <mailto:tauberer@...>>>
                      > <mailto:tauberer@... <mailto:tauberer@...>
                      > <mailto:tauberer@... <mailto:tauberer@...>>
                      > <mailto:tauberer@... <mailto:tauberer@...>
                      > <mailto:tauberer@... <mailto:tauberer@...>>>__>__>
                      >
                      > wrote:
                      >
                      > I've put this off just about long enough-
                      >
                      > GovTrack now has an API!
                      >
                      > Currently bills and people are included in
                      > the API.
                      > (Votes and
                      > committees will come later if there's interest.)
                      >
                      > More here:
                      >
                      > http://www.govtrack.us/______developers/api
                      > <http://www.govtrack.us/____developers/api>
                      > <http://www.govtrack.us/____developers/api
                      > <http://www.govtrack.us/__developers/api>>
                      >
                      >
                      > <http://www.govtrack.us/____developers/api
                      > <http://www.govtrack.us/__developers/api>
                      > <http://www.govtrack.us/__developers/api
                      > <http://www.govtrack.us/developers/api>>>
                      >
                      > It's experimental at the moment. I don't
                      > have any
                      > caching on
                      > it, no API
                      > keys, etc. Go easy!
                      >
                      > --
                      > - Josh Tauberer (@JoshData)
                      >
                      > http://razor.occams.info
                      >
                      >
                      > ------------------------------______------
                      >
                      >
                      >
                      > Yahoo! Groups Links
                      >
                      > <http://groups.yahoo.com/__group/__govtrack/
                      > <http://groups.yahoo.com/group/__govtrack/>>
                      >
                      > <http://groups.yahoo.com/____group/govtrack/
                      > <http://groups.yahoo.com/__group/govtrack/>
                      >
                      > <http://groups.yahoo.com/__group/govtrack/
                      > <http://groups.yahoo.com/group/govtrack/>>>
                      >
                      > <http://groups.yahoo.com/__group/__govtrack/join
                      > <http://groups.yahoo.com/group/__govtrack/join>>
                      >
                      > <http://groups.yahoo.com/____group/govtrack/join
                      > <http://groups.yahoo.com/__group/govtrack/join>
                      >
                      > <http://groups.yahoo.com/__group/govtrack/join
                      > <http://groups.yahoo.com/group/govtrack/join>>>
                      > (Yahoo! ID required)
                      >
                      > <mailto:govtrack-digest@>__yaho__ogroups.com
                      > <http://yahoogroups.com>
                      > <mailto:govtrack-digest@__yahoogroups.com
                      > <mailto:govtrack-digest@yahoogroups.com>>>
                      > <mailto:govtrack-digest@ <mailto:govtrack-digest@>
                      > <mailto:govtrack-digest@
                      > <mailto:govtrack-digest@>>__yah__o__ogroups.com
                      > <http://yaho__ogroups.com>
                      > <http://yahoogroups.com>
                      > <mailto:govtrack-digest@
                      > <mailto:govtrack-digest@>__yaho__ogroups.com
                      > <http://yahoogroups.com>
                      > <mailto:govtrack-digest@__yahoogroups.com
                      > <mailto:govtrack-digest@yahoogroups.com>>>>
                      > govtrack-fullfeatured@__yahoog____roups.com
                      > <http://yahoog__roups.com>
                      > <http://yahoogroups.com>
                      > <mailto:govtrack-fullfeatured@
                      > <mailto:govtrack-fullfeatured@>____yahoogroups.com
                      > <http://yahoogroups.com>
                      > <mailto:govtrack-fullfeatured@__yahoogroups.com
                      > <mailto:govtrack-fullfeatured@yahoogroups.com>>>
                      > <mailto:govtrack-fullfeatured@ <mailto:govtrack-fullfeatured@>
                      > <mailto:govtrack-fullfeatured@
                      > <mailto:govtrack-fullfeatured@>__>____yahoogroups.com
                      > <http://yahoogroups.com>
                      > <http://yahoogroups.com>
                      >
                      >
                      > <mailto:govtrack-fullfeatured@
                      > <mailto:govtrack-fullfeatured@>____yahoogroups.com
                      > <http://yahoogroups.com>
                      > <mailto:govtrack-fullfeatured@__yahoogroups.com
                      > <mailto:govtrack-fullfeatured@yahoogroups.com>>>>
                      >
                      >
                      > <http://yahoogroups.com>
                      > <mailto:govtrack-unsubscribe@
                      > <mailto:govtrack-unsubscribe@>____yahoogroups.com
                      > <http://yahoogroups.com>
                      > <mailto:govtrack-unsubscribe@__yahoogroups.com
                      > <mailto:govtrack-unsubscribe@yahoogroups.com>>>
                      > <mailto:govtrack-unsubscribe@ <mailto:govtrack-unsubscribe@>
                      > <mailto:govtrack-unsubscribe@
                      > <mailto:govtrack-unsubscribe@>>______yahoogroups.com
                      > <http://yahoogroups.com>
                      > <http://yahoogroups.com>
                      >
                      >
                      > <mailto:govtrack-unsubscribe@
                      > <mailto:govtrack-unsubscribe@>____yahoogroups.com
                      > <http://yahoogroups.com>
                      > <mailto:govtrack-unsubscribe@__yahoogroups.com
                      > <mailto:govtrack-unsubscribe@yahoogroups.com>>>>
                      >
                      >
                      > <http://docs.yahoo.com/info/____terms/
                      > <http://docs.yahoo.com/info/__terms/>>
                      >
                      >
                      > <http://docs.yahoo.com/info/____terms/
                      > <http://docs.yahoo.com/info/__terms/>
                      > <http://docs.yahoo.com/info/__terms/
                      > <http://docs.yahoo.com/info/terms/>>>
                      >
                      >
                      >
                      >
                      > --
                      > Developer | sunlightfoundation.com
                      > <http://sunlightfoundation.com>
                      > <http://sunlightfoundation.com__>
                      > <http://sunlightfoundation.__com__
                      > <http://sunlightfoundation.com__>>
                      > <http://sunlightfoundation.____com__
                      >
                      > <http://sunlightfoundation.__com__
                      > <http://sunlightfoundation.com__>>>
                      >
                      >
                      >
                      >
                      >
                      >
                      >
                      >
                      > --
                      > Developer | sunlightfoundation.com
                      > <http://sunlightfoundation.com>
                      > <http://sunlightfoundation.com__>
                      > <http://sunlightfoundation.__com__
                      > <http://sunlightfoundation.com__>>
                      >
                      >
                      >
                      >
                      >
                      >
                      >
                      >
                      >
                      > --
                      > Developer | sunlightfoundation.com
                      > <http://sunlightfoundation.com> <http://sunlightfoundation.com__>
                      >
                      >
                      >
                      >
                      > --
                      > Developer | sunlightfoundation.com <http://sunlightfoundation.com>
                      >
                      >
                      >
                      >
                    Your message has been successfully submitted and would be delivered to recipients shortly.