Loading ...
Sorry, an error occurred while loading the content.

Re: [govtrack] GovTrack API (experimental)

Expand Messages
  • Eric Mill
    Sweet. Are you going to dive into text search at all? This has become the most useful thing to come out of RTC, though I haven t documented the search methods
    Message 1 of 9 , May 17, 2012
    • 0 Attachment
      Sweet. Are you going to dive into text search at all? This has become the most useful thing to come out of RTC, though I haven't documented the search methods publicly yet. It's a hard problem, but you're developing all sorts of nifty ways of calculating relevance, so you'd be in a good position to do it well.

      -- Eric

      On Thu, May 17, 2012 at 6:58 AM, Josh Tauberer <tauberer@...> wrote:
      On 05/16/2012 11:39 PM, Eric Mill wrote:
      > Super awesome, really glad to see this happen. Nice job with the docs,
      > too (and a nice reminder to redo and update my own).

      Thanks. I drew on a few ideas from RTC, e.g. I made sure the infrastructure I chose (Tasypie) supported __[operator] filtering.


      > Are you using this in anything yourself now?

      Not yet. The main reason I made it is so when people come to me asking for CSV exports I have a way to pull them quickly. If people want to build actual apps, even better. (Only half joking.)

      Except that the API is tightly coupled to the site's models, so I use it at that level. (Makes it practically self-documenting too.)



      - Josh Tauberer (@JoshData)

      http://razor.occams.info

      On 05/16/2012 11:39 PM, Eric Mill wrote:


      Super awesome, really glad to see this happen. Nice job with the docs,
      too (and a nice reminder to redo and update my own).

      Are you using this in anything yourself now?

      -- Eric

      On Wed, May 16, 2012 at 9:07 PM, Josh Tauberer <tauberer@...
      <mailto:tauberer@...>> wrote:

         I've put this off just about long enough-

         GovTrack now has an API!

         Currently bills and people are included in the API. (Votes and
         committees will come later if there's interest.)

         More here:

         http://www.govtrack.us/developers/api

         It's experimental at the moment. I don't have any caching on it, no API
         keys, etc. Go easy!

         --
         - Josh Tauberer (@JoshData)

         http://razor.occams.info


         ------------------------------------

         Yahoo! Groups Links

         <*> To visit your group on the web, go to:
         http://groups.yahoo.com/group/govtrack/

         <*> Your email settings:
             Individual Email | Traditional

         <*> To change settings online go to:
         http://groups.yahoo.com/group/govtrack/join
             (Yahoo! ID required)

         <*> To change settings via email:
         govtrack-digest@yahoogroups.com <mailto:govtrack-digest@yahoogroups.com>
         govtrack-fullfeatured@yahoogroups.com
         <mailto:govtrack-fullfeatured@yahoogroups.com>


         <*> To unsubscribe from this group, send an email to:
         govtrack-unsubscribe@yahoogroups.com
         <mailto:govtrack-unsubscribe@yahoogroups.com>


         <*> Your use of Yahoo! Groups is subject to:
         http://docs.yahoo.com/info/terms/




      --
      Developer | sunlightfoundation.com <http://sunlightfoundation.com>







      --

    • Josh Tauberer
      Probably, yeah. Right now the API is based on the Django ORM, which hits MySQL. That s not great for search so I have to see if I can switch it transparently
      Message 2 of 9 , May 17, 2012
      • 0 Attachment
        Probably, yeah. Right now the API is based on the Django ORM, which hits
        MySQL. That's not great for search so I have to see if I can switch it
        transparently to using Haystack, or else build a separate API for search.

        But document relevance is.... ugh. Two fun facts: The highest-prognosis
        bills are the dumb ones like post office namings, so that's not good for
        relevance. And the top visited bills are almost never the ones people
        should really be looking at, so popularity is bad too. Fun fun.

        - Josh Tauberer (@JoshData)

        http://razor.occams.info

        On 05/17/2012 11:15 AM, Eric Mill wrote:
        >
        >
        > Sweet. Are you going to dive into text search at all? This has become
        > the most useful thing to come out of RTC, though I haven't documented
        > the search methods publicly yet. It's a hard problem, but you're
        > developing all sorts of nifty ways of calculating relevance, so you'd be
        > in a good position to do it well.
        >
        > -- Eric
        >
        > On Thu, May 17, 2012 at 6:58 AM, Josh Tauberer <tauberer@...
        > <mailto:tauberer@...>> wrote:
        >
        > On 05/16/2012 11:39 PM, Eric Mill wrote:
        > > Super awesome, really glad to see this happen. Nice job with the
        > docs,
        > > too (and a nice reminder to redo and update my own).
        >
        > Thanks. I drew on a few ideas from RTC, e.g. I made sure the
        > infrastructure I chose (Tasypie) supported __[operator] filtering.
        >
        >
        > > Are you using this in anything yourself now?
        >
        > Not yet. The main reason I made it is so when people come to me
        > asking for CSV exports I have a way to pull them quickly. If people
        > want to build actual apps, even better. (Only half joking.)
        >
        > Except that the API is tightly coupled to the site's models, so I
        > use it at that level. (Makes it practically self-documenting too.)
        >
        >
        >
        > - Josh Tauberer (@JoshData)
        >
        > http://razor.occams.info
        >
        > On 05/16/2012 11:39 PM, Eric Mill wrote:
        >
        >
        >
        > Super awesome, really glad to see this happen. Nice job with the
        > docs,
        > too (and a nice reminder to redo and update my own).
        >
        > Are you using this in anything yourself now?
        >
        > -- Eric
        >
        > On Wed, May 16, 2012 at 9:07 PM, Josh Tauberer
        > <tauberer@... <mailto:tauberer@...>
        > <mailto:tauberer@... <mailto:tauberer@...>>> wrote:
        >
        > I've put this off just about long enough-
        >
        > GovTrack now has an API!
        >
        > Currently bills and people are included in the API. (Votes and
        > committees will come later if there's interest.)
        >
        > More here:
        >
        > http://www.govtrack.us/__developers/api
        > <http://www.govtrack.us/developers/api>
        >
        > It's experimental at the moment. I don't have any caching on
        > it, no API
        > keys, etc. Go easy!
        >
        > --
        > - Josh Tauberer (@JoshData)
        >
        > http://razor.occams.info
        >
        >
        > ------------------------------__------
        >
        > Yahoo! Groups Links
        >
        >
        > (Yahoo! ID required)
        >
        > <mailto:govtrack-digest@__yahoogroups.com
        > <mailto:govtrack-digest@yahoogroups.com>>
        > govtrack-fullfeatured@__yahoogroups.com
        > <mailto:govtrack-fullfeatured@yahoogroups.com>
        > <mailto:govtrack-fullfeatured@__yahoogroups.com
        > <mailto:govtrack-fullfeatured@yahoogroups.com>>
        >
        >
        > <mailto:govtrack-unsubscribe@__yahoogroups.com
        > <mailto:govtrack-unsubscribe@yahoogroups.com>>
        >
        >
        >
        >
        >
        >
        > --
        > Developer | sunlightfoundation.com
        > <http://sunlightfoundation.com> <http://sunlightfoundation.com__>
        >
        >
        >
        >
        >
        >
        >
        > --
        > Developer | sunlightfoundation.com <http://sunlightfoundation.com>
        >
        >
        >
        >
      • Eric Mill
        So fun. We had a nice (potential) success story at Sunlight the other day here, where we
        Message 3 of 9 , May 17, 2012
        • 0 Attachment
          So fun. 

          We had a nice (potential) success story at Sunlight the other day here, where we caught a nasty proposed FOIA exemption in a bill being actively crafted inside the House through our search engine (second item down on this list, right now: http://scout.sunlightfoundation.com/search/federal_bills/section%20552%20of%20title%205). It has no official summary, at the time it had no official "short title" yet and was a big paragraph of text...and as you can see my highlighter is failing to even provide an excerpt for that result. John had to open up the official text on GPO and dig through the text to find out why it even came up. This was a very relevant bill, but it was only spotted because it was new and because John is a diligent guy.

          It had been announced for floor activity, and was introduced by the committee chair, so maybe those would help in relevance - if you also tuned out the post office renamings. Maybe that's the only way, is just to filter out specific classes of bills and resolutions.

          -- Eric

          On Thu, May 17, 2012 at 11:23 AM, Josh Tauberer <tauberer@...> wrote:
          Probably, yeah. Right now the API is based on the Django ORM, which hits MySQL. That's not great for search so I have to see if I can switch it transparently to using Haystack, or else build a separate API for search.

          But document relevance is.... ugh. Two fun facts: The highest-prognosis bills are the dumb ones like post office namings, so that's not good for relevance. And the top visited bills are almost never the ones people should really be looking at, so popularity is bad too. Fun fun.


          - Josh Tauberer (@JoshData)

          http://razor.occams.info

          On 05/17/2012 11:15 AM, Eric Mill wrote:


          Sweet. Are you going to dive into text search at all? This has become
          the most useful thing to come out of RTC, though I haven't documented
          the search methods publicly yet. It's a hard problem, but you're
          developing all sorts of nifty ways of calculating relevance, so you'd be
          in a good position to do it well.

          -- Eric

          On Thu, May 17, 2012 at 6:58 AM, Josh Tauberer <tauberer@...
          <mailto:tauberer@...>> wrote:

             On 05/16/2012 11:39 PM, Eric Mill wrote:
              > Super awesome, really glad to see this happen. Nice job with the
             docs,
              > too (and a nice reminder to redo and update my own).

             Thanks. I drew on a few ideas from RTC, e.g. I made sure the
             infrastructure I chose (Tasypie) supported __[operator] filtering.


              > Are you using this in anything yourself now?

             Not yet. The main reason I made it is so when people come to me
             asking for CSV exports I have a way to pull them quickly. If people
             want to build actual apps, even better. (Only half joking.)

             Except that the API is tightly coupled to the site's models, so I
             use it at that level. (Makes it practically self-documenting too.)



             - Josh Tauberer (@JoshData)

             http://razor.occams.info

             On 05/16/2012 11:39 PM, Eric Mill wrote:



                 Super awesome, really glad to see this happen. Nice job with the
                 docs,
                 too (and a nice reminder to redo and update my own).

                 Are you using this in anything yourself now?

                 -- Eric

                 On Wed, May 16, 2012 at 9:07 PM, Josh Tauberer
                 <tauberer@... <mailto:tauberer@...>
                 <mailto:tauberer@... <mailto:tauberer@...>>> wrote:

                     I've put this off just about long enough-

                     GovTrack now has an API!

                     Currently bills and people are included in the API. (Votes and
                     committees will come later if there's interest.)

                     More here:

                 http://www.govtrack.us/__developers/api

                 <http://www.govtrack.us/developers/api>

                     It's experimental at the moment. I don't have any caching on
                 it, no API
                     keys, etc. Go easy!

                     --
                     - Josh Tauberer (@JoshData)

                 http://razor.occams.info


                     ------------------------------__------


                     Yahoo! Groups Links

                 <*> To visit your group on the web, go to:
                 http://groups.yahoo.com/group/__govtrack/

                 <http://groups.yahoo.com/group/govtrack/>

                 <*> Your email settings:
                         Individual Email | Traditional

                 <*> To change settings online go to:
                 http://groups.yahoo.com/group/__govtrack/join

                 <http://groups.yahoo.com/group/govtrack/join>
                         (Yahoo! ID required)

                 <*> To change settings via email:
                 govtrack-digest@yahoogroups.__com
                 <mailto:govtrack-digest@yahoogroups.com>
                 <mailto:govtrack-digest@__yahoogroups.com
                 <mailto:govtrack-digest@yahoogroups.com>>
                 govtrack-fullfeatured@__yahoogroups.com
                 <mailto:govtrack-fullfeatured@yahoogroups.com>
                 <mailto:govtrack-fullfeatured@__yahoogroups.com

                 <mailto:govtrack-fullfeatured@yahoogroups.com>>


                 <*> To unsubscribe from this group, send an email to:
                 govtrack-unsubscribe@__yahoogroups.com
                 <mailto:govtrack-unsubscribe@yahoogroups.com>
                 <mailto:govtrack-unsubscribe@__yahoogroups.com

                 <mailto:govtrack-unsubscribe@yahoogroups.com>>


                 <*> Your use of Yahoo! Groups is subject to:
                 http://docs.yahoo.com/info/__terms/

                 <http://docs.yahoo.com/info/terms/>




                 --
                 Developer | sunlightfoundation.com
                 <http://sunlightfoundation.com> <http://sunlightfoundation.com__>



          --

        • Josh Tauberer
          ... Yeah, that is worth a try. I m now sorting bills on the advanced search by a modified-prognosis score that just omits any boost that post office bills and
          Message 4 of 9 , May 23, 2012
          • 0 Attachment
            On 05/17/2012 11:36 AM, Eric Mill wrote:
            > if you also
            > tuned out the post office renamings. Maybe that's the only way, is just
            > to filter out specific classes of bills and resolutions.

            Yeah, that is worth a try. I'm now sorting bills on the advanced search
            by a modified-prognosis score that just omits any boost that post office
            bills and some other bills get, and also arbitrarily de-weights resolutions.

            http://www.govtrack.us/congress/bills/browse

            Think it's more useful than e.g. last action date? Looking at the
            results I think so but I'm not sure.

            (I guess I could do A/B testing with users but it's so much trouble to
            be quantitative.)

            - Josh Tauberer (@JoshData)

            http://razor.occams.info

            On 05/17/2012 11:36 AM, Eric Mill wrote:
            > So fun.
            >
            > We had a nice (potential) success story at Sunlight the other day here,
            > where we caught
            > <https://twitter.com/#!/JohnWonderlich/status/202385148161953793> a
            > nasty proposed FOIA exemption in a bill being actively crafted inside
            > the House through our search engine (second item down on this list,
            > right now:
            > http://scout.sunlightfoundation.com/search/federal_bills/section%20552%20of%20title%205).
            > It has no official summary, at the time it had no official "short title"
            > yet and was a big paragraph of text...and as you can see my highlighter
            > is failing to even provide an excerpt for that result. John had to open
            > up the official text on GPO and dig through the text to find out why it
            > even came up. This was a very relevant bill, but it was only spotted
            > because it was new and because John is a diligent guy.
            >
            > It had been announced for floor activity, and was introduced by the
            > committee chair, so maybe those would help in relevance - if you also
            > tuned out the post office renamings. Maybe that's the only way, is just
            > to filter out specific classes of bills and resolutions.
            >
            > -- Eric
            >
            > On Thu, May 17, 2012 at 11:23 AM, Josh Tauberer <tauberer@...
            > <mailto:tauberer@...>> wrote:
            >
            > Probably, yeah. Right now the API is based on the Django ORM, which
            > hits MySQL. That's not great for search so I have to see if I can
            > switch it transparently to using Haystack, or else build a separate
            > API for search.
            >
            > But document relevance is.... ugh. Two fun facts: The
            > highest-prognosis bills are the dumb ones like post office namings,
            > so that's not good for relevance. And the top visited bills are
            > almost never the ones people should really be looking at, so
            > popularity is bad too. Fun fun.
            >
            >
            > - Josh Tauberer (@JoshData)
            >
            > http://razor.occams.info
            >
            > On 05/17/2012 11:15 AM, Eric Mill wrote:
            >
            >
            >
            > Sweet. Are you going to dive into text search at all? This has
            > become
            > the most useful thing to come out of RTC, though I haven't
            > documented
            > the search methods publicly yet. It's a hard problem, but you're
            > developing all sorts of nifty ways of calculating relevance, so
            > you'd be
            > in a good position to do it well.
            >
            > -- Eric
            >
            > On Thu, May 17, 2012 at 6:58 AM, Josh Tauberer
            > <tauberer@... <mailto:tauberer@...>
            > <mailto:tauberer@... <mailto:tauberer@...>>> wrote:
            >
            > On 05/16/2012 11:39 PM, Eric Mill wrote:
            > > Super awesome, really glad to see this happen. Nice job with the
            > docs,
            > > too (and a nice reminder to redo and update my own).
            >
            > Thanks. I drew on a few ideas from RTC, e.g. I made sure the
            > infrastructure I chose (Tasypie) supported __[operator]
            > filtering.
            >
            >
            > > Are you using this in anything yourself now?
            >
            > Not yet. The main reason I made it is so when people come to me
            > asking for CSV exports I have a way to pull them quickly. If
            > people
            > want to build actual apps, even better. (Only half joking.)
            >
            > Except that the API is tightly coupled to the site's models,
            > so I
            > use it at that level. (Makes it practically self-documenting
            > too.)
            >
            >
            >
            > - Josh Tauberer (@JoshData)
            >
            > http://razor.occams.info
            >
            > On 05/16/2012 11:39 PM, Eric Mill wrote:
            >
            >
            >
            > Super awesome, really glad to see this happen. Nice job
            > with the
            > docs,
            > too (and a nice reminder to redo and update my own).
            >
            > Are you using this in anything yourself now?
            >
            > -- Eric
            >
            > On Wed, May 16, 2012 at 9:07 PM, Josh Tauberer
            > <tauberer@... <mailto:tauberer@...>
            > <mailto:tauberer@... <mailto:tauberer@...>>
            > <mailto:tauberer@... <mailto:tauberer@...>
            > <mailto:tauberer@... <mailto:tauberer@...>>>__>
            > wrote:
            >
            > I've put this off just about long enough-
            >
            > GovTrack now has an API!
            >
            > Currently bills and people are included in the API.
            > (Votes and
            > committees will come later if there's interest.)
            >
            > More here:
            >
            > http://www.govtrack.us/____developers/api
            > <http://www.govtrack.us/__developers/api>
            >
            > <http://www.govtrack.us/__developers/api
            > <http://www.govtrack.us/developers/api>>
            >
            > It's experimental at the moment. I don't have any
            > caching on
            > it, no API
            > keys, etc. Go easy!
            >
            > --
            > - Josh Tauberer (@JoshData)
            >
            > http://razor.occams.info
            >
            >
            > ------------------------------____------
            >
            >
            > Yahoo! Groups Links
            >
            >
            > <http://groups.yahoo.com/__group/govtrack/
            > <http://groups.yahoo.com/group/govtrack/>>
            >
            >
            > <http://groups.yahoo.com/__group/govtrack/join
            > <http://groups.yahoo.com/group/govtrack/join>>
            > (Yahoo! ID required)
            >
            > <mailto:govtrack-digest@yahoogroups.com>>
            > <mailto:govtrack-digest@
            > <mailto:govtrack-digest@>__yaho__ogroups.com
            > <http://yahoogroups.com>
            > <mailto:govtrack-digest@__yahoogroups.com
            > <mailto:govtrack-digest@yahoogroups.com>>>
            > govtrack-fullfeatured@__yahoog__roups.com
            > <http://yahoogroups.com>
            > <mailto:govtrack-fullfeatured@__yahoogroups.com
            > <mailto:govtrack-fullfeatured@yahoogroups.com>>
            > <mailto:govtrack-fullfeatured@
            > <mailto:govtrack-fullfeatured@>____yahoogroups.com
            > <http://yahoogroups.com>
            >
            > <mailto:govtrack-fullfeatured@__yahoogroups.com
            > <mailto:govtrack-fullfeatured@yahoogroups.com>>>
            >
            >
            > <mailto:govtrack-unsubscribe@__yahoogroups.com
            > <mailto:govtrack-unsubscribe@yahoogroups.com>>
            > <mailto:govtrack-unsubscribe@
            > <mailto:govtrack-unsubscribe@>____yahoogroups.com
            > <http://yahoogroups.com>
            >
            > <mailto:govtrack-unsubscribe@__yahoogroups.com
            > <mailto:govtrack-unsubscribe@yahoogroups.com>>>
            >
            >
            >
            > <http://docs.yahoo.com/info/__terms/
            > <http://docs.yahoo.com/info/terms/>>
            >
            >
            >
            >
            > --
            > Developer | sunlightfoundation.com
            > <http://sunlightfoundation.com>
            > <http://sunlightfoundation.com__>
            > <http://sunlightfoundation.__com__
            > <http://sunlightfoundation.com__>>
            >
            >
            >
            >
            >
            >
            >
            >
            > --
            > Developer | sunlightfoundation.com
            > <http://sunlightfoundation.com> <http://sunlightfoundation.com__>
            >
            >
            >
            >
            >
            >
            >
            >
            > --
            > Developer | sunlightfoundation.com <http://sunlightfoundation.com>
            >
          • Eric Mill
            Sorting on prognosis is a cool idea. Does that mean that bills already passed into law will always occur at the top, or the bottom, or...? You are the last
            Message 5 of 9 , May 23, 2012
            • 0 Attachment
              Sorting on prognosis is a cool idea. Does that mean that bills already passed into law will always occur at the top, or the bottom, or...?

              You are the last person who needs to hear the definition of resolution types, but just to make sure, you'd probably want to only de-weight plain resolutions and concurrent resolutions, and not joint resolutions (which are bills with the force of law and are signed by the President).

              On Wed, May 23, 2012 at 6:25 PM, Josh Tauberer <tauberer@...> wrote:
              On 05/17/2012 11:36 AM, Eric Mill wrote:
              if you also
              tuned out the post office renamings. Maybe that's the only way, is just
              to filter out specific classes of bills and resolutions.

              Yeah, that is worth a try. I'm now sorting bills on the advanced search by a modified-prognosis score that just omits any boost that post office bills and some other bills get, and also arbitrarily de-weights resolutions.

              http://www.govtrack.us/congress/bills/browse

              Think it's more useful than e.g. last action date? Looking at the results I think so but I'm not sure.

              (I guess I could do A/B testing with users but it's so much trouble to be quantitative.)


              - Josh Tauberer (@JoshData)

              http://razor.occams.info

              On 05/17/2012 11:36 AM, Eric Mill wrote:
              So fun.

              We had a nice (potential) success story at Sunlight the other day here,
              where we caught
              <https://twitter.com/#!/JohnWonderlich/status/202385148161953793> a

              nasty proposed FOIA exemption in a bill being actively crafted inside
              the House through our search engine (second item down on this list,
              right now:
              http://scout.sunlightfoundation.com/search/federal_bills/section%20552%20of%20title%205).
              It has no official summary, at the time it had no official "short title"
              yet and was a big paragraph of text...and as you can see my highlighter
              is failing to even provide an excerpt for that result. John had to open
              up the official text on GPO and dig through the text to find out why it
              even came up. This was a very relevant bill, but it was only spotted
              because it was new and because John is a diligent guy.

              It had been announced for floor activity, and was introduced by the
              committee chair, so maybe those would help in relevance - if you also
              tuned out the post office renamings. Maybe that's the only way, is just
              to filter out specific classes of bills and resolutions.

              -- Eric

              On Thu, May 17, 2012 at 11:23 AM, Josh Tauberer <tauberer@...
              <mailto:tauberer@...>> wrote:

                 Probably, yeah. Right now the API is based on the Django ORM, which
                 hits MySQL. That's not great for search so I have to see if I can
                 switch it transparently to using Haystack, or else build a separate
                 API for search.

                 But document relevance is.... ugh. Two fun facts: The
                 highest-prognosis bills are the dumb ones like post office namings,
                 so that's not good for relevance. And the top visited bills are
                 almost never the ones people should really be looking at, so
                 popularity is bad too. Fun fun.


                 - Josh Tauberer (@JoshData)

                 http://razor.occams.info

                 On 05/17/2012 11:15 AM, Eric Mill wrote:



                     Sweet. Are you going to dive into text search at all? This has
                     become
                     the most useful thing to come out of RTC, though I haven't
                     documented
                     the search methods publicly yet. It's a hard problem, but you're
                     developing all sorts of nifty ways of calculating relevance, so
                     you'd be
                     in a good position to do it well.

                     -- Eric

                     On Thu, May 17, 2012 at 6:58 AM, Josh Tauberer
                     <tauberer@... <mailto:tauberer@...>
                     <mailto:tauberer@... <mailto:tauberer@...>>> wrote:

                         On 05/16/2012 11:39 PM, Eric Mill wrote:
                      > Super awesome, really glad to see this happen. Nice job with the
                         docs,
                      > too (and a nice reminder to redo and update my own).

                         Thanks. I drew on a few ideas from RTC, e.g. I made sure the
                         infrastructure I chose (Tasypie) supported __[operator]
                     filtering.


                      > Are you using this in anything yourself now?

                         Not yet. The main reason I made it is so when people come to me
                         asking for CSV exports I have a way to pull them quickly. If
                     people
                         want to build actual apps, even better. (Only half joking.)

                         Except that the API is tightly coupled to the site's models,
                     so I
                         use it at that level. (Makes it practically self-documenting
                     too.)



                         - Josh Tauberer (@JoshData)

                     http://razor.occams.info

                         On 05/16/2012 11:39 PM, Eric Mill wrote:



                             Super awesome, really glad to see this happen. Nice job
                     with the
                             docs,
                             too (and a nice reminder to redo and update my own).

                             Are you using this in anything yourself now?

                             -- Eric

                             On Wed, May 16, 2012 at 9:07 PM, Josh Tauberer
                     <tauberer@... <mailto:tauberer@...>
                     <mailto:tauberer@... <mailto:tauberer@...>>
                     <mailto:tauberer@... <mailto:tauberer@...>
                     <mailto:tauberer@... <mailto:tauberer@...>>>__>

                     wrote:

                                 I've put this off just about long enough-

                                 GovTrack now has an API!

                                 Currently bills and people are included in the API.
                     (Votes and
                                 committees will come later if there's interest.)

                                 More here:

                     http://www.govtrack.us/____developers/api
                     <http://www.govtrack.us/__developers/api>


                     <http://www.govtrack.us/__developers/api
                     <http://www.govtrack.us/developers/api>>

                                 It's experimental at the moment. I don't have any
                     caching on
                             it, no API
                                 keys, etc. Go easy!

                                 --
                                 - Josh Tauberer (@JoshData)

                     http://razor.occams.info


                                 ------------------------------____------



                                 Yahoo! Groups Links

                     <*> To visit your group on the web, go to:
                     http://groups.yahoo.com/group/____govtrack/
                     <http://groups.yahoo.com/group/__govtrack/>

                     <http://groups.yahoo.com/__group/govtrack/

                     <http://groups.yahoo.com/group/govtrack/>>

                     <*> Your email settings:
                                     Individual Email | Traditional

                     <*> To change settings online go to:
                     http://groups.yahoo.com/group/____govtrack/join
                     <http://groups.yahoo.com/group/__govtrack/join>

                     <http://groups.yahoo.com/__group/govtrack/join

                     <http://groups.yahoo.com/group/govtrack/join>>
                                     (Yahoo! ID required)

                     <*> To change settings via email:
                             govtrack-digest@yahoogroups.____com
                     <mailto:govtrack-digest@__yahoogroups.com
                     <mailto:govtrack-digest@yahoogroups.com>>
                     <mailto:govtrack-digest@
                     <mailto:govtrack-digest@>__yaho__ogroups.com
                     <http://yahoogroups.com>
                     <mailto:govtrack-digest@__yahoogroups.com
                     <mailto:govtrack-digest@yahoogroups.com>>>
                             govtrack-fullfeatured@__yahoog__roups.com
                     <http://yahoogroups.com>
                     <mailto:govtrack-fullfeatured@__yahoogroups.com
                     <mailto:govtrack-fullfeatured@yahoogroups.com>>
                     <mailto:govtrack-fullfeatured@
                     <mailto:govtrack-fullfeatured@>____yahoogroups.com
                     <http://yahoogroups.com>


                     <mailto:govtrack-fullfeatured@__yahoogroups.com
                     <mailto:govtrack-fullfeatured@yahoogroups.com>>>


                     <*> To unsubscribe from this group, send an email to:
                             govtrack-unsubscribe@__yahoogr__oups.com
                     <http://yahoogroups.com>
                     <mailto:govtrack-unsubscribe@__yahoogroups.com
                     <mailto:govtrack-unsubscribe@yahoogroups.com>>
                     <mailto:govtrack-unsubscribe@
                     <mailto:govtrack-unsubscribe@>____yahoogroups.com
                     <http://yahoogroups.com>


                     <mailto:govtrack-unsubscribe@__yahoogroups.com
                     <mailto:govtrack-unsubscribe@yahoogroups.com>>>


                     <*> Your use of Yahoo! Groups is subject to:
                     http://docs.yahoo.com/info/____terms/
                     <http://docs.yahoo.com/info/__terms/>


                     <http://docs.yahoo.com/info/__terms/
                     <http://docs.yahoo.com/info/terms/>>




                             --
                             Developer | sunlightfoundation.com
                     <http://sunlightfoundation.com>
                     <http://sunlightfoundation.com__>
                     <http://sunlightfoundation.__com__

                     <http://sunlightfoundation.com__>>








                     --
                     Developer | sunlightfoundation.com
                     <http://sunlightfoundation.com> <http://sunlightfoundation.com__>



              --

            • Josh Tauberer
              ... Yeah this part is weird. The only status taken into account is whether the bill has been reported by committee. So reported bills get a boost. Enacted
              Message 6 of 9 , May 23, 2012
              • 0 Attachment
                On 05/23/2012 06:37 PM, Eric Mill wrote:
                > Sorting on prognosis is a cool idea. Does that mean that bills already
                > passed into law will always occur at the top, or the bottom, or...?

                Yeah this part is weird. The only status taken into account is whether
                the bill has been reported by committee. So reported bills get a boost.
                Enacted bills get the same boost, but no additional boost for being
                passed/enacted/etc. So everything besides introduced bills get a boost.

                > you'd probably want to only de-weight
                > plain resolutions and concurrent resolutions

                The actual weights are:

                simple resolution .2
                concurrent resolution: .3
                joint resolution: .35
                bill: 1.0

                So I probably went too far with the joint resolutions.

                About half of joint resolutions are proposing an amendment to the
                constitution (=no force of law). Rather than weighting them all 0.3, I
                should probably just see why the prognosis doesn't include a factor for
                constitutional amendment resolutions and weight those accordingly (i.e.
                none of those ever pass) and, overall, weight joint resolutions the same
                as bills.

                Thanks for the reminder about that.

                - Josh Tauberer (@JoshData)

                http://razor.occams.info

                On 05/23/2012 06:37 PM, Eric Mill wrote:
                > Sorting on prognosis is a cool idea. Does that mean that bills already
                > passed into law will always occur at the top, or the bottom, or...?
                >
                > You are the last person who needs to hear the definition of resolution
                > types, but just to make sure, you'd probably want to only de-weight
                > plain resolutions and concurrent resolutions, and not joint resolutions
                > (which are bills with the force of law and are signed by the President).
                >
                > On Wed, May 23, 2012 at 6:25 PM, Josh Tauberer <tauberer@...
                > <mailto:tauberer@...>> wrote:
                >
                > On 05/17/2012 11:36 AM, Eric Mill wrote:
                >
                > if you also
                > tuned out the post office renamings. Maybe that's the only way,
                > is just
                > to filter out specific classes of bills and resolutions.
                >
                >
                > Yeah, that is worth a try. I'm now sorting bills on the advanced
                > search by a modified-prognosis score that just omits any boost that
                > post office bills and some other bills get, and also arbitrarily
                > de-weights resolutions.
                >
                > http://www.govtrack.us/__congress/bills/browse
                > <http://www.govtrack.us/congress/bills/browse>
                >
                > Think it's more useful than e.g. last action date? Looking at the
                > results I think so but I'm not sure.
                >
                > (I guess I could do A/B testing with users but it's so much trouble
                > to be quantitative.)
                >
                >
                > - Josh Tauberer (@JoshData)
                >
                > http://razor.occams.info
                >
                > On 05/17/2012 11:36 AM, Eric Mill wrote:
                >
                > So fun.
                >
                > We had a nice (potential) success story at Sunlight the other
                > day here,
                > where we caught
                > <https://twitter.com/#!/__JohnWonderlich/status/__202385148161953793
                > <https://twitter.com/#!/JohnWonderlich/status/202385148161953793>> a
                >
                > nasty proposed FOIA exemption in a bill being actively crafted
                > inside
                > the House through our search engine (second item down on this list,
                > right now:
                > http://scout.__sunlightfoundation.com/search/__federal_bills/section%20552%__20of%20title%205
                > <http://scout.sunlightfoundation.com/search/federal_bills/section%20552%20of%20title%205>).
                > It has no official summary, at the time it had no official
                > "short title"
                > yet and was a big paragraph of text...and as you can see my
                > highlighter
                > is failing to even provide an excerpt for that result. John had
                > to open
                > up the official text on GPO and dig through the text to find out
                > why it
                > even came up. This was a very relevant bill, but it was only spotted
                > because it was new and because John is a diligent guy.
                >
                > It had been announced for floor activity, and was introduced by the
                > committee chair, so maybe those would help in relevance - if you
                > also
                > tuned out the post office renamings. Maybe that's the only way,
                > is just
                > to filter out specific classes of bills and resolutions.
                >
                > -- Eric
                >
                > On Thu, May 17, 2012 at 11:23 AM, Josh Tauberer
                > <tauberer@... <mailto:tauberer@...>
                > <mailto:tauberer@... <mailto:tauberer@...>>> wrote:
                >
                > Probably, yeah. Right now the API is based on the Django
                > ORM, which
                > hits MySQL. That's not great for search so I have to see if
                > I can
                > switch it transparently to using Haystack, or else build a
                > separate
                > API for search.
                >
                > But document relevance is.... ugh. Two fun facts: The
                > highest-prognosis bills are the dumb ones like post office
                > namings,
                > so that's not good for relevance. And the top visited bills are
                > almost never the ones people should really be looking at, so
                > popularity is bad too. Fun fun.
                >
                >
                > - Josh Tauberer (@JoshData)
                >
                > http://razor.occams.info
                >
                > On 05/17/2012 11:15 AM, Eric Mill wrote:
                >
                >
                >
                > Sweet. Are you going to dive into text search at all?
                > This has
                > become
                > the most useful thing to come out of RTC, though I haven't
                > documented
                > the search methods publicly yet. It's a hard problem,
                > but you're
                > developing all sorts of nifty ways of calculating
                > relevance, so
                > you'd be
                > in a good position to do it well.
                >
                > -- Eric
                >
                > On Thu, May 17, 2012 at 6:58 AM, Josh Tauberer
                > <tauberer@... <mailto:tauberer@...>
                > <mailto:tauberer@... <mailto:tauberer@...>>
                > <mailto:tauberer@... <mailto:tauberer@...>
                > <mailto:tauberer@... <mailto:tauberer@...>>>__>
                > wrote:
                >
                > On 05/16/2012 11:39 PM, Eric Mill wrote:
                > > Super awesome, really glad to see this happen. Nice job with the
                > docs,
                > > too (and a nice reminder to redo and update my own).
                >
                > Thanks. I drew on a few ideas from RTC, e.g. I made
                > sure the
                > infrastructure I chose (Tasypie) supported __[operator]
                > filtering.
                >
                >
                > > Are you using this in anything yourself now?
                >
                > Not yet. The main reason I made it is so when people
                > come to me
                > asking for CSV exports I have a way to pull them
                > quickly. If
                > people
                > want to build actual apps, even better. (Only half
                > joking.)
                >
                > Except that the API is tightly coupled to the site's
                > models,
                > so I
                > use it at that level. (Makes it practically
                > self-documenting
                > too.)
                >
                >
                >
                > - Josh Tauberer (@JoshData)
                >
                > http://razor.occams.info
                >
                > On 05/16/2012 11:39 PM, Eric Mill wrote:
                >
                >
                >
                > Super awesome, really glad to see this happen.
                > Nice job
                > with the
                > docs,
                > too (and a nice reminder to redo and update my own).
                >
                > Are you using this in anything yourself now?
                >
                > -- Eric
                >
                > On Wed, May 16, 2012 at 9:07 PM, Josh Tauberer
                > <tauberer@... <mailto:tauberer@...>
                > <mailto:tauberer@... <mailto:tauberer@...>>
                > <mailto:tauberer@... <mailto:tauberer@...>
                > <mailto:tauberer@... <mailto:tauberer@...>>>
                > <mailto:tauberer@... <mailto:tauberer@...>
                > <mailto:tauberer@... <mailto:tauberer@...>>
                > <mailto:tauberer@... <mailto:tauberer@...>
                > <mailto:tauberer@... <mailto:tauberer@...>>>__>__>
                >
                > wrote:
                >
                > I've put this off just about long enough-
                >
                > GovTrack now has an API!
                >
                > Currently bills and people are included in
                > the API.
                > (Votes and
                > committees will come later if there's interest.)
                >
                > More here:
                >
                > http://www.govtrack.us/______developers/api
                > <http://www.govtrack.us/____developers/api>
                > <http://www.govtrack.us/____developers/api
                > <http://www.govtrack.us/__developers/api>>
                >
                >
                > <http://www.govtrack.us/____developers/api
                > <http://www.govtrack.us/__developers/api>
                > <http://www.govtrack.us/__developers/api
                > <http://www.govtrack.us/developers/api>>>
                >
                > It's experimental at the moment. I don't
                > have any
                > caching on
                > it, no API
                > keys, etc. Go easy!
                >
                > --
                > - Josh Tauberer (@JoshData)
                >
                > http://razor.occams.info
                >
                >
                > ------------------------------______------
                >
                >
                >
                > Yahoo! Groups Links
                >
                > <http://groups.yahoo.com/__group/__govtrack/
                > <http://groups.yahoo.com/group/__govtrack/>>
                >
                > <http://groups.yahoo.com/____group/govtrack/
                > <http://groups.yahoo.com/__group/govtrack/>
                >
                > <http://groups.yahoo.com/__group/govtrack/
                > <http://groups.yahoo.com/group/govtrack/>>>
                >
                > <http://groups.yahoo.com/__group/__govtrack/join
                > <http://groups.yahoo.com/group/__govtrack/join>>
                >
                > <http://groups.yahoo.com/____group/govtrack/join
                > <http://groups.yahoo.com/__group/govtrack/join>
                >
                > <http://groups.yahoo.com/__group/govtrack/join
                > <http://groups.yahoo.com/group/govtrack/join>>>
                > (Yahoo! ID required)
                >
                > <mailto:govtrack-digest@>__yaho__ogroups.com
                > <http://yahoogroups.com>
                > <mailto:govtrack-digest@__yahoogroups.com
                > <mailto:govtrack-digest@yahoogroups.com>>>
                > <mailto:govtrack-digest@ <mailto:govtrack-digest@>
                > <mailto:govtrack-digest@
                > <mailto:govtrack-digest@>>__yah__o__ogroups.com
                > <http://yaho__ogroups.com>
                > <http://yahoogroups.com>
                > <mailto:govtrack-digest@
                > <mailto:govtrack-digest@>__yaho__ogroups.com
                > <http://yahoogroups.com>
                > <mailto:govtrack-digest@__yahoogroups.com
                > <mailto:govtrack-digest@yahoogroups.com>>>>
                > govtrack-fullfeatured@__yahoog____roups.com
                > <http://yahoog__roups.com>
                > <http://yahoogroups.com>
                > <mailto:govtrack-fullfeatured@
                > <mailto:govtrack-fullfeatured@>____yahoogroups.com
                > <http://yahoogroups.com>
                > <mailto:govtrack-fullfeatured@__yahoogroups.com
                > <mailto:govtrack-fullfeatured@yahoogroups.com>>>
                > <mailto:govtrack-fullfeatured@ <mailto:govtrack-fullfeatured@>
                > <mailto:govtrack-fullfeatured@
                > <mailto:govtrack-fullfeatured@>__>____yahoogroups.com
                > <http://yahoogroups.com>
                > <http://yahoogroups.com>
                >
                >
                > <mailto:govtrack-fullfeatured@
                > <mailto:govtrack-fullfeatured@>____yahoogroups.com
                > <http://yahoogroups.com>
                > <mailto:govtrack-fullfeatured@__yahoogroups.com
                > <mailto:govtrack-fullfeatured@yahoogroups.com>>>>
                >
                >
                > <http://yahoogroups.com>
                > <mailto:govtrack-unsubscribe@
                > <mailto:govtrack-unsubscribe@>____yahoogroups.com
                > <http://yahoogroups.com>
                > <mailto:govtrack-unsubscribe@__yahoogroups.com
                > <mailto:govtrack-unsubscribe@yahoogroups.com>>>
                > <mailto:govtrack-unsubscribe@ <mailto:govtrack-unsubscribe@>
                > <mailto:govtrack-unsubscribe@
                > <mailto:govtrack-unsubscribe@>>______yahoogroups.com
                > <http://yahoogroups.com>
                > <http://yahoogroups.com>
                >
                >
                > <mailto:govtrack-unsubscribe@
                > <mailto:govtrack-unsubscribe@>____yahoogroups.com
                > <http://yahoogroups.com>
                > <mailto:govtrack-unsubscribe@__yahoogroups.com
                > <mailto:govtrack-unsubscribe@yahoogroups.com>>>>
                >
                >
                > <http://docs.yahoo.com/info/____terms/
                > <http://docs.yahoo.com/info/__terms/>>
                >
                >
                > <http://docs.yahoo.com/info/____terms/
                > <http://docs.yahoo.com/info/__terms/>
                > <http://docs.yahoo.com/info/__terms/
                > <http://docs.yahoo.com/info/terms/>>>
                >
                >
                >
                >
                > --
                > Developer | sunlightfoundation.com
                > <http://sunlightfoundation.com>
                > <http://sunlightfoundation.com__>
                > <http://sunlightfoundation.__com__
                > <http://sunlightfoundation.com__>>
                > <http://sunlightfoundation.____com__
                >
                > <http://sunlightfoundation.__com__
                > <http://sunlightfoundation.com__>>>
                >
                >
                >
                >
                >
                >
                >
                >
                > --
                > Developer | sunlightfoundation.com
                > <http://sunlightfoundation.com>
                > <http://sunlightfoundation.com__>
                > <http://sunlightfoundation.__com__
                > <http://sunlightfoundation.com__>>
                >
                >
                >
                >
                >
                >
                >
                >
                >
                > --
                > Developer | sunlightfoundation.com
                > <http://sunlightfoundation.com> <http://sunlightfoundation.com__>
                >
                >
                >
                >
                > --
                > Developer | sunlightfoundation.com <http://sunlightfoundation.com>
                >
                >
                >
                >
              Your message has been successfully submitted and would be delivered to recipients shortly.