Loading ...
Sorry, an error occurred while loading the content.

Re: [govtrack] GovTrack API (experimental)

Expand Messages
  • Eric Mill
    Super awesome, really glad to see this happen. Nice job with the docs, too (and a nice reminder to redo and update my own). Are you using this in anything
    Message 1 of 9 , May 16, 2012
    • 0 Attachment
      Super awesome, really glad to see this happen. Nice job with the docs, too (and a nice reminder to redo and update my own).

      Are you using this in anything yourself now?

      -- Eric

      On Wed, May 16, 2012 at 9:07 PM, Josh Tauberer <tauberer@...> wrote:
      I've put this off just about long enough-

      GovTrack now has an API!

      Currently bills and people are included in the API. (Votes and
      committees will come later if there's interest.)

      More here:

      http://www.govtrack.us/developers/api

      It's experimental at the moment. I don't have any caching on it, no API
      keys, etc. Go easy!

      --
      - Josh Tauberer (@JoshData)

      http://razor.occams.info


      ------------------------------------

      Yahoo! Groups Links

      <*> To visit your group on the web, go to:
         http://groups.yahoo.com/group/govtrack/

      <*> Your email settings:
         Individual Email | Traditional

      <*> To change settings online go to:
         http://groups.yahoo.com/group/govtrack/join
         (Yahoo! ID required)

      <*> To change settings via email:
         govtrack-digest@yahoogroups.com
         govtrack-fullfeatured@yahoogroups.com

      <*> To unsubscribe from this group, send an email to:
         govtrack-unsubscribe@yahoogroups.com

      <*> Your use of Yahoo! Groups is subject to:
         http://docs.yahoo.com/info/terms/




      --

    • Josh Tauberer
      ... Thanks. I drew on a few ideas from RTC, e.g. I made sure the infrastructure I chose (Tasypie) supported __[operator] filtering. ... Not yet. The main
      Message 2 of 9 , May 17, 2012
      • 0 Attachment
        On 05/16/2012 11:39 PM, Eric Mill wrote:
        > Super awesome, really glad to see this happen. Nice job with the docs,
        > too (and a nice reminder to redo and update my own).

        Thanks. I drew on a few ideas from RTC, e.g. I made sure the
        infrastructure I chose (Tasypie) supported __[operator] filtering.

        > Are you using this in anything yourself now?

        Not yet. The main reason I made it is so when people come to me asking
        for CSV exports I have a way to pull them quickly. If people want to
        build actual apps, even better. (Only half joking.)

        Except that the API is tightly coupled to the site's models, so I use it
        at that level. (Makes it practically self-documenting too.)


        - Josh Tauberer (@JoshData)

        http://razor.occams.info

        On 05/16/2012 11:39 PM, Eric Mill wrote:
        >
        >
        > Super awesome, really glad to see this happen. Nice job with the docs,
        > too (and a nice reminder to redo and update my own).
        >
        > Are you using this in anything yourself now?
        >
        > -- Eric
        >
        > On Wed, May 16, 2012 at 9:07 PM, Josh Tauberer <tauberer@...
        > <mailto:tauberer@...>> wrote:
        >
        > I've put this off just about long enough-
        >
        > GovTrack now has an API!
        >
        > Currently bills and people are included in the API. (Votes and
        > committees will come later if there's interest.)
        >
        > More here:
        >
        > http://www.govtrack.us/developers/api
        >
        > It's experimental at the moment. I don't have any caching on it, no API
        > keys, etc. Go easy!
        >
        > --
        > - Josh Tauberer (@JoshData)
        >
        > http://razor.occams.info
        >
        >
        > ------------------------------------
        >
        > Yahoo! Groups Links
        >
        >
        > <mailto:govtrack-fullfeatured@yahoogroups.com>
        >
        >
        >
        >
        >
        > --
        > Developer | sunlightfoundation.com <http://sunlightfoundation.com>
        >
        >
        >
        >
      • Eric Mill
        Sweet. Are you going to dive into text search at all? This has become the most useful thing to come out of RTC, though I haven t documented the search methods
        Message 3 of 9 , May 17, 2012
        • 0 Attachment
          Sweet. Are you going to dive into text search at all? This has become the most useful thing to come out of RTC, though I haven't documented the search methods publicly yet. It's a hard problem, but you're developing all sorts of nifty ways of calculating relevance, so you'd be in a good position to do it well.

          -- Eric

          On Thu, May 17, 2012 at 6:58 AM, Josh Tauberer <tauberer@...> wrote:
          On 05/16/2012 11:39 PM, Eric Mill wrote:
          > Super awesome, really glad to see this happen. Nice job with the docs,
          > too (and a nice reminder to redo and update my own).

          Thanks. I drew on a few ideas from RTC, e.g. I made sure the infrastructure I chose (Tasypie) supported __[operator] filtering.


          > Are you using this in anything yourself now?

          Not yet. The main reason I made it is so when people come to me asking for CSV exports I have a way to pull them quickly. If people want to build actual apps, even better. (Only half joking.)

          Except that the API is tightly coupled to the site's models, so I use it at that level. (Makes it practically self-documenting too.)



          - Josh Tauberer (@JoshData)

          http://razor.occams.info

          On 05/16/2012 11:39 PM, Eric Mill wrote:


          Super awesome, really glad to see this happen. Nice job with the docs,
          too (and a nice reminder to redo and update my own).

          Are you using this in anything yourself now?

          -- Eric

          On Wed, May 16, 2012 at 9:07 PM, Josh Tauberer <tauberer@...
          <mailto:tauberer@...>> wrote:

             I've put this off just about long enough-

             GovTrack now has an API!

             Currently bills and people are included in the API. (Votes and
             committees will come later if there's interest.)

             More here:

             http://www.govtrack.us/developers/api

             It's experimental at the moment. I don't have any caching on it, no API
             keys, etc. Go easy!

             --
             - Josh Tauberer (@JoshData)

             http://razor.occams.info


             ------------------------------------

             Yahoo! Groups Links

             <*> To visit your group on the web, go to:
             http://groups.yahoo.com/group/govtrack/

             <*> Your email settings:
                 Individual Email | Traditional

             <*> To change settings online go to:
             http://groups.yahoo.com/group/govtrack/join
                 (Yahoo! ID required)

             <*> To change settings via email:
             govtrack-digest@yahoogroups.com <mailto:govtrack-digest@yahoogroups.com>
             govtrack-fullfeatured@yahoogroups.com
             <mailto:govtrack-fullfeatured@yahoogroups.com>


             <*> To unsubscribe from this group, send an email to:
             govtrack-unsubscribe@yahoogroups.com
             <mailto:govtrack-unsubscribe@yahoogroups.com>


             <*> Your use of Yahoo! Groups is subject to:
             http://docs.yahoo.com/info/terms/




          --
          Developer | sunlightfoundation.com <http://sunlightfoundation.com>







          --

        • Josh Tauberer
          Probably, yeah. Right now the API is based on the Django ORM, which hits MySQL. That s not great for search so I have to see if I can switch it transparently
          Message 4 of 9 , May 17, 2012
          • 0 Attachment
            Probably, yeah. Right now the API is based on the Django ORM, which hits
            MySQL. That's not great for search so I have to see if I can switch it
            transparently to using Haystack, or else build a separate API for search.

            But document relevance is.... ugh. Two fun facts: The highest-prognosis
            bills are the dumb ones like post office namings, so that's not good for
            relevance. And the top visited bills are almost never the ones people
            should really be looking at, so popularity is bad too. Fun fun.

            - Josh Tauberer (@JoshData)

            http://razor.occams.info

            On 05/17/2012 11:15 AM, Eric Mill wrote:
            >
            >
            > Sweet. Are you going to dive into text search at all? This has become
            > the most useful thing to come out of RTC, though I haven't documented
            > the search methods publicly yet. It's a hard problem, but you're
            > developing all sorts of nifty ways of calculating relevance, so you'd be
            > in a good position to do it well.
            >
            > -- Eric
            >
            > On Thu, May 17, 2012 at 6:58 AM, Josh Tauberer <tauberer@...
            > <mailto:tauberer@...>> wrote:
            >
            > On 05/16/2012 11:39 PM, Eric Mill wrote:
            > > Super awesome, really glad to see this happen. Nice job with the
            > docs,
            > > too (and a nice reminder to redo and update my own).
            >
            > Thanks. I drew on a few ideas from RTC, e.g. I made sure the
            > infrastructure I chose (Tasypie) supported __[operator] filtering.
            >
            >
            > > Are you using this in anything yourself now?
            >
            > Not yet. The main reason I made it is so when people come to me
            > asking for CSV exports I have a way to pull them quickly. If people
            > want to build actual apps, even better. (Only half joking.)
            >
            > Except that the API is tightly coupled to the site's models, so I
            > use it at that level. (Makes it practically self-documenting too.)
            >
            >
            >
            > - Josh Tauberer (@JoshData)
            >
            > http://razor.occams.info
            >
            > On 05/16/2012 11:39 PM, Eric Mill wrote:
            >
            >
            >
            > Super awesome, really glad to see this happen. Nice job with the
            > docs,
            > too (and a nice reminder to redo and update my own).
            >
            > Are you using this in anything yourself now?
            >
            > -- Eric
            >
            > On Wed, May 16, 2012 at 9:07 PM, Josh Tauberer
            > <tauberer@... <mailto:tauberer@...>
            > <mailto:tauberer@... <mailto:tauberer@...>>> wrote:
            >
            > I've put this off just about long enough-
            >
            > GovTrack now has an API!
            >
            > Currently bills and people are included in the API. (Votes and
            > committees will come later if there's interest.)
            >
            > More here:
            >
            > http://www.govtrack.us/__developers/api
            > <http://www.govtrack.us/developers/api>
            >
            > It's experimental at the moment. I don't have any caching on
            > it, no API
            > keys, etc. Go easy!
            >
            > --
            > - Josh Tauberer (@JoshData)
            >
            > http://razor.occams.info
            >
            >
            > ------------------------------__------
            >
            > Yahoo! Groups Links
            >
            >
            > (Yahoo! ID required)
            >
            > <mailto:govtrack-digest@__yahoogroups.com
            > <mailto:govtrack-digest@yahoogroups.com>>
            > govtrack-fullfeatured@__yahoogroups.com
            > <mailto:govtrack-fullfeatured@yahoogroups.com>
            > <mailto:govtrack-fullfeatured@__yahoogroups.com
            > <mailto:govtrack-fullfeatured@yahoogroups.com>>
            >
            >
            > <mailto:govtrack-unsubscribe@__yahoogroups.com
            > <mailto:govtrack-unsubscribe@yahoogroups.com>>
            >
            >
            >
            >
            >
            >
            > --
            > Developer | sunlightfoundation.com
            > <http://sunlightfoundation.com> <http://sunlightfoundation.com__>
            >
            >
            >
            >
            >
            >
            >
            > --
            > Developer | sunlightfoundation.com <http://sunlightfoundation.com>
            >
            >
            >
            >
          • Eric Mill
            So fun. We had a nice (potential) success story at Sunlight the other day here, where we
            Message 5 of 9 , May 17, 2012
            • 0 Attachment
              So fun. 

              We had a nice (potential) success story at Sunlight the other day here, where we caught a nasty proposed FOIA exemption in a bill being actively crafted inside the House through our search engine (second item down on this list, right now: http://scout.sunlightfoundation.com/search/federal_bills/section%20552%20of%20title%205). It has no official summary, at the time it had no official "short title" yet and was a big paragraph of text...and as you can see my highlighter is failing to even provide an excerpt for that result. John had to open up the official text on GPO and dig through the text to find out why it even came up. This was a very relevant bill, but it was only spotted because it was new and because John is a diligent guy.

              It had been announced for floor activity, and was introduced by the committee chair, so maybe those would help in relevance - if you also tuned out the post office renamings. Maybe that's the only way, is just to filter out specific classes of bills and resolutions.

              -- Eric

              On Thu, May 17, 2012 at 11:23 AM, Josh Tauberer <tauberer@...> wrote:
              Probably, yeah. Right now the API is based on the Django ORM, which hits MySQL. That's not great for search so I have to see if I can switch it transparently to using Haystack, or else build a separate API for search.

              But document relevance is.... ugh. Two fun facts: The highest-prognosis bills are the dumb ones like post office namings, so that's not good for relevance. And the top visited bills are almost never the ones people should really be looking at, so popularity is bad too. Fun fun.


              - Josh Tauberer (@JoshData)

              http://razor.occams.info

              On 05/17/2012 11:15 AM, Eric Mill wrote:


              Sweet. Are you going to dive into text search at all? This has become
              the most useful thing to come out of RTC, though I haven't documented
              the search methods publicly yet. It's a hard problem, but you're
              developing all sorts of nifty ways of calculating relevance, so you'd be
              in a good position to do it well.

              -- Eric

              On Thu, May 17, 2012 at 6:58 AM, Josh Tauberer <tauberer@...
              <mailto:tauberer@...>> wrote:

                 On 05/16/2012 11:39 PM, Eric Mill wrote:
                  > Super awesome, really glad to see this happen. Nice job with the
                 docs,
                  > too (and a nice reminder to redo and update my own).

                 Thanks. I drew on a few ideas from RTC, e.g. I made sure the
                 infrastructure I chose (Tasypie) supported __[operator] filtering.


                  > Are you using this in anything yourself now?

                 Not yet. The main reason I made it is so when people come to me
                 asking for CSV exports I have a way to pull them quickly. If people
                 want to build actual apps, even better. (Only half joking.)

                 Except that the API is tightly coupled to the site's models, so I
                 use it at that level. (Makes it practically self-documenting too.)



                 - Josh Tauberer (@JoshData)

                 http://razor.occams.info

                 On 05/16/2012 11:39 PM, Eric Mill wrote:



                     Super awesome, really glad to see this happen. Nice job with the
                     docs,
                     too (and a nice reminder to redo and update my own).

                     Are you using this in anything yourself now?

                     -- Eric

                     On Wed, May 16, 2012 at 9:07 PM, Josh Tauberer
                     <tauberer@... <mailto:tauberer@...>
                     <mailto:tauberer@... <mailto:tauberer@...>>> wrote:

                         I've put this off just about long enough-

                         GovTrack now has an API!

                         Currently bills and people are included in the API. (Votes and
                         committees will come later if there's interest.)

                         More here:

                     http://www.govtrack.us/__developers/api

                     <http://www.govtrack.us/developers/api>

                         It's experimental at the moment. I don't have any caching on
                     it, no API
                         keys, etc. Go easy!

                         --
                         - Josh Tauberer (@JoshData)

                     http://razor.occams.info


                         ------------------------------__------


                         Yahoo! Groups Links

                     <*> To visit your group on the web, go to:
                     http://groups.yahoo.com/group/__govtrack/

                     <http://groups.yahoo.com/group/govtrack/>

                     <*> Your email settings:
                             Individual Email | Traditional

                     <*> To change settings online go to:
                     http://groups.yahoo.com/group/__govtrack/join

                     <http://groups.yahoo.com/group/govtrack/join>
                             (Yahoo! ID required)

                     <*> To change settings via email:
                     govtrack-digest@yahoogroups.__com
                     <mailto:govtrack-digest@yahoogroups.com>
                     <mailto:govtrack-digest@__yahoogroups.com
                     <mailto:govtrack-digest@yahoogroups.com>>
                     govtrack-fullfeatured@__yahoogroups.com
                     <mailto:govtrack-fullfeatured@yahoogroups.com>
                     <mailto:govtrack-fullfeatured@__yahoogroups.com

                     <mailto:govtrack-fullfeatured@yahoogroups.com>>


                     <*> To unsubscribe from this group, send an email to:
                     govtrack-unsubscribe@__yahoogroups.com
                     <mailto:govtrack-unsubscribe@yahoogroups.com>
                     <mailto:govtrack-unsubscribe@__yahoogroups.com

                     <mailto:govtrack-unsubscribe@yahoogroups.com>>


                     <*> Your use of Yahoo! Groups is subject to:
                     http://docs.yahoo.com/info/__terms/

                     <http://docs.yahoo.com/info/terms/>




                     --
                     Developer | sunlightfoundation.com
                     <http://sunlightfoundation.com> <http://sunlightfoundation.com__>



              --

            • Josh Tauberer
              ... Yeah, that is worth a try. I m now sorting bills on the advanced search by a modified-prognosis score that just omits any boost that post office bills and
              Message 6 of 9 , May 23, 2012
              • 0 Attachment
                On 05/17/2012 11:36 AM, Eric Mill wrote:
                > if you also
                > tuned out the post office renamings. Maybe that's the only way, is just
                > to filter out specific classes of bills and resolutions.

                Yeah, that is worth a try. I'm now sorting bills on the advanced search
                by a modified-prognosis score that just omits any boost that post office
                bills and some other bills get, and also arbitrarily de-weights resolutions.

                http://www.govtrack.us/congress/bills/browse

                Think it's more useful than e.g. last action date? Looking at the
                results I think so but I'm not sure.

                (I guess I could do A/B testing with users but it's so much trouble to
                be quantitative.)

                - Josh Tauberer (@JoshData)

                http://razor.occams.info

                On 05/17/2012 11:36 AM, Eric Mill wrote:
                > So fun.
                >
                > We had a nice (potential) success story at Sunlight the other day here,
                > where we caught
                > <https://twitter.com/#!/JohnWonderlich/status/202385148161953793> a
                > nasty proposed FOIA exemption in a bill being actively crafted inside
                > the House through our search engine (second item down on this list,
                > right now:
                > http://scout.sunlightfoundation.com/search/federal_bills/section%20552%20of%20title%205).
                > It has no official summary, at the time it had no official "short title"
                > yet and was a big paragraph of text...and as you can see my highlighter
                > is failing to even provide an excerpt for that result. John had to open
                > up the official text on GPO and dig through the text to find out why it
                > even came up. This was a very relevant bill, but it was only spotted
                > because it was new and because John is a diligent guy.
                >
                > It had been announced for floor activity, and was introduced by the
                > committee chair, so maybe those would help in relevance - if you also
                > tuned out the post office renamings. Maybe that's the only way, is just
                > to filter out specific classes of bills and resolutions.
                >
                > -- Eric
                >
                > On Thu, May 17, 2012 at 11:23 AM, Josh Tauberer <tauberer@...
                > <mailto:tauberer@...>> wrote:
                >
                > Probably, yeah. Right now the API is based on the Django ORM, which
                > hits MySQL. That's not great for search so I have to see if I can
                > switch it transparently to using Haystack, or else build a separate
                > API for search.
                >
                > But document relevance is.... ugh. Two fun facts: The
                > highest-prognosis bills are the dumb ones like post office namings,
                > so that's not good for relevance. And the top visited bills are
                > almost never the ones people should really be looking at, so
                > popularity is bad too. Fun fun.
                >
                >
                > - Josh Tauberer (@JoshData)
                >
                > http://razor.occams.info
                >
                > On 05/17/2012 11:15 AM, Eric Mill wrote:
                >
                >
                >
                > Sweet. Are you going to dive into text search at all? This has
                > become
                > the most useful thing to come out of RTC, though I haven't
                > documented
                > the search methods publicly yet. It's a hard problem, but you're
                > developing all sorts of nifty ways of calculating relevance, so
                > you'd be
                > in a good position to do it well.
                >
                > -- Eric
                >
                > On Thu, May 17, 2012 at 6:58 AM, Josh Tauberer
                > <tauberer@... <mailto:tauberer@...>
                > <mailto:tauberer@... <mailto:tauberer@...>>> wrote:
                >
                > On 05/16/2012 11:39 PM, Eric Mill wrote:
                > > Super awesome, really glad to see this happen. Nice job with the
                > docs,
                > > too (and a nice reminder to redo and update my own).
                >
                > Thanks. I drew on a few ideas from RTC, e.g. I made sure the
                > infrastructure I chose (Tasypie) supported __[operator]
                > filtering.
                >
                >
                > > Are you using this in anything yourself now?
                >
                > Not yet. The main reason I made it is so when people come to me
                > asking for CSV exports I have a way to pull them quickly. If
                > people
                > want to build actual apps, even better. (Only half joking.)
                >
                > Except that the API is tightly coupled to the site's models,
                > so I
                > use it at that level. (Makes it practically self-documenting
                > too.)
                >
                >
                >
                > - Josh Tauberer (@JoshData)
                >
                > http://razor.occams.info
                >
                > On 05/16/2012 11:39 PM, Eric Mill wrote:
                >
                >
                >
                > Super awesome, really glad to see this happen. Nice job
                > with the
                > docs,
                > too (and a nice reminder to redo and update my own).
                >
                > Are you using this in anything yourself now?
                >
                > -- Eric
                >
                > On Wed, May 16, 2012 at 9:07 PM, Josh Tauberer
                > <tauberer@... <mailto:tauberer@...>
                > <mailto:tauberer@... <mailto:tauberer@...>>
                > <mailto:tauberer@... <mailto:tauberer@...>
                > <mailto:tauberer@... <mailto:tauberer@...>>>__>
                > wrote:
                >
                > I've put this off just about long enough-
                >
                > GovTrack now has an API!
                >
                > Currently bills and people are included in the API.
                > (Votes and
                > committees will come later if there's interest.)
                >
                > More here:
                >
                > http://www.govtrack.us/____developers/api
                > <http://www.govtrack.us/__developers/api>
                >
                > <http://www.govtrack.us/__developers/api
                > <http://www.govtrack.us/developers/api>>
                >
                > It's experimental at the moment. I don't have any
                > caching on
                > it, no API
                > keys, etc. Go easy!
                >
                > --
                > - Josh Tauberer (@JoshData)
                >
                > http://razor.occams.info
                >
                >
                > ------------------------------____------
                >
                >
                > Yahoo! Groups Links
                >
                >
                > <http://groups.yahoo.com/__group/govtrack/
                > <http://groups.yahoo.com/group/govtrack/>>
                >
                >
                > <http://groups.yahoo.com/__group/govtrack/join
                > <http://groups.yahoo.com/group/govtrack/join>>
                > (Yahoo! ID required)
                >
                > <mailto:govtrack-digest@yahoogroups.com>>
                > <mailto:govtrack-digest@
                > <mailto:govtrack-digest@>__yaho__ogroups.com
                > <http://yahoogroups.com>
                > <mailto:govtrack-digest@__yahoogroups.com
                > <mailto:govtrack-digest@yahoogroups.com>>>
                > govtrack-fullfeatured@__yahoog__roups.com
                > <http://yahoogroups.com>
                > <mailto:govtrack-fullfeatured@__yahoogroups.com
                > <mailto:govtrack-fullfeatured@yahoogroups.com>>
                > <mailto:govtrack-fullfeatured@
                > <mailto:govtrack-fullfeatured@>____yahoogroups.com
                > <http://yahoogroups.com>
                >
                > <mailto:govtrack-fullfeatured@__yahoogroups.com
                > <mailto:govtrack-fullfeatured@yahoogroups.com>>>
                >
                >
                > <mailto:govtrack-unsubscribe@__yahoogroups.com
                > <mailto:govtrack-unsubscribe@yahoogroups.com>>
                > <mailto:govtrack-unsubscribe@
                > <mailto:govtrack-unsubscribe@>____yahoogroups.com
                > <http://yahoogroups.com>
                >
                > <mailto:govtrack-unsubscribe@__yahoogroups.com
                > <mailto:govtrack-unsubscribe@yahoogroups.com>>>
                >
                >
                >
                > <http://docs.yahoo.com/info/__terms/
                > <http://docs.yahoo.com/info/terms/>>
                >
                >
                >
                >
                > --
                > Developer | sunlightfoundation.com
                > <http://sunlightfoundation.com>
                > <http://sunlightfoundation.com__>
                > <http://sunlightfoundation.__com__
                > <http://sunlightfoundation.com__>>
                >
                >
                >
                >
                >
                >
                >
                >
                > --
                > Developer | sunlightfoundation.com
                > <http://sunlightfoundation.com> <http://sunlightfoundation.com__>
                >
                >
                >
                >
                >
                >
                >
                >
                > --
                > Developer | sunlightfoundation.com <http://sunlightfoundation.com>
                >
              • Eric Mill
                Sorting on prognosis is a cool idea. Does that mean that bills already passed into law will always occur at the top, or the bottom, or...? You are the last
                Message 7 of 9 , May 23, 2012
                • 0 Attachment
                  Sorting on prognosis is a cool idea. Does that mean that bills already passed into law will always occur at the top, or the bottom, or...?

                  You are the last person who needs to hear the definition of resolution types, but just to make sure, you'd probably want to only de-weight plain resolutions and concurrent resolutions, and not joint resolutions (which are bills with the force of law and are signed by the President).

                  On Wed, May 23, 2012 at 6:25 PM, Josh Tauberer <tauberer@...> wrote:
                  On 05/17/2012 11:36 AM, Eric Mill wrote:
                  if you also
                  tuned out the post office renamings. Maybe that's the only way, is just
                  to filter out specific classes of bills and resolutions.

                  Yeah, that is worth a try. I'm now sorting bills on the advanced search by a modified-prognosis score that just omits any boost that post office bills and some other bills get, and also arbitrarily de-weights resolutions.

                  http://www.govtrack.us/congress/bills/browse

                  Think it's more useful than e.g. last action date? Looking at the results I think so but I'm not sure.

                  (I guess I could do A/B testing with users but it's so much trouble to be quantitative.)


                  - Josh Tauberer (@JoshData)

                  http://razor.occams.info

                  On 05/17/2012 11:36 AM, Eric Mill wrote:
                  So fun.

                  We had a nice (potential) success story at Sunlight the other day here,
                  where we caught
                  <https://twitter.com/#!/JohnWonderlich/status/202385148161953793> a

                  nasty proposed FOIA exemption in a bill being actively crafted inside
                  the House through our search engine (second item down on this list,
                  right now:
                  http://scout.sunlightfoundation.com/search/federal_bills/section%20552%20of%20title%205).
                  It has no official summary, at the time it had no official "short title"
                  yet and was a big paragraph of text...and as you can see my highlighter
                  is failing to even provide an excerpt for that result. John had to open
                  up the official text on GPO and dig through the text to find out why it
                  even came up. This was a very relevant bill, but it was only spotted
                  because it was new and because John is a diligent guy.

                  It had been announced for floor activity, and was introduced by the
                  committee chair, so maybe those would help in relevance - if you also
                  tuned out the post office renamings. Maybe that's the only way, is just
                  to filter out specific classes of bills and resolutions.

                  -- Eric

                  On Thu, May 17, 2012 at 11:23 AM, Josh Tauberer <tauberer@...
                  <mailto:tauberer@...>> wrote:

                     Probably, yeah. Right now the API is based on the Django ORM, which
                     hits MySQL. That's not great for search so I have to see if I can
                     switch it transparently to using Haystack, or else build a separate
                     API for search.

                     But document relevance is.... ugh. Two fun facts: The
                     highest-prognosis bills are the dumb ones like post office namings,
                     so that's not good for relevance. And the top visited bills are
                     almost never the ones people should really be looking at, so
                     popularity is bad too. Fun fun.


                     - Josh Tauberer (@JoshData)

                     http://razor.occams.info

                     On 05/17/2012 11:15 AM, Eric Mill wrote:



                         Sweet. Are you going to dive into text search at all? This has
                         become
                         the most useful thing to come out of RTC, though I haven't
                         documented
                         the search methods publicly yet. It's a hard problem, but you're
                         developing all sorts of nifty ways of calculating relevance, so
                         you'd be
                         in a good position to do it well.

                         -- Eric

                         On Thu, May 17, 2012 at 6:58 AM, Josh Tauberer
                         <tauberer@... <mailto:tauberer@...>
                         <mailto:tauberer@... <mailto:tauberer@...>>> wrote:

                             On 05/16/2012 11:39 PM, Eric Mill wrote:
                          > Super awesome, really glad to see this happen. Nice job with the
                             docs,
                          > too (and a nice reminder to redo and update my own).

                             Thanks. I drew on a few ideas from RTC, e.g. I made sure the
                             infrastructure I chose (Tasypie) supported __[operator]
                         filtering.


                          > Are you using this in anything yourself now?

                             Not yet. The main reason I made it is so when people come to me
                             asking for CSV exports I have a way to pull them quickly. If
                         people
                             want to build actual apps, even better. (Only half joking.)

                             Except that the API is tightly coupled to the site's models,
                         so I
                             use it at that level. (Makes it practically self-documenting
                         too.)



                             - Josh Tauberer (@JoshData)

                         http://razor.occams.info

                             On 05/16/2012 11:39 PM, Eric Mill wrote:



                                 Super awesome, really glad to see this happen. Nice job
                         with the
                                 docs,
                                 too (and a nice reminder to redo and update my own).

                                 Are you using this in anything yourself now?

                                 -- Eric

                                 On Wed, May 16, 2012 at 9:07 PM, Josh Tauberer
                         <tauberer@... <mailto:tauberer@...>
                         <mailto:tauberer@... <mailto:tauberer@...>>
                         <mailto:tauberer@... <mailto:tauberer@...>
                         <mailto:tauberer@... <mailto:tauberer@...>>>__>

                         wrote:

                                     I've put this off just about long enough-

                                     GovTrack now has an API!

                                     Currently bills and people are included in the API.
                         (Votes and
                                     committees will come later if there's interest.)

                                     More here:

                         http://www.govtrack.us/____developers/api
                         <http://www.govtrack.us/__developers/api>


                         <http://www.govtrack.us/__developers/api
                         <http://www.govtrack.us/developers/api>>

                                     It's experimental at the moment. I don't have any
                         caching on
                                 it, no API
                                     keys, etc. Go easy!

                                     --
                                     - Josh Tauberer (@JoshData)

                         http://razor.occams.info


                                     ------------------------------____------



                                     Yahoo! Groups Links

                         <*> To visit your group on the web, go to:
                         http://groups.yahoo.com/group/____govtrack/
                         <http://groups.yahoo.com/group/__govtrack/>

                         <http://groups.yahoo.com/__group/govtrack/

                         <http://groups.yahoo.com/group/govtrack/>>

                         <*> Your email settings:
                                         Individual Email | Traditional

                         <*> To change settings online go to:
                         http://groups.yahoo.com/group/____govtrack/join
                         <http://groups.yahoo.com/group/__govtrack/join>

                         <http://groups.yahoo.com/__group/govtrack/join

                         <http://groups.yahoo.com/group/govtrack/join>>
                                         (Yahoo! ID required)

                         <*> To change settings via email:
                                 govtrack-digest@yahoogroups.____com
                         <mailto:govtrack-digest@__yahoogroups.com
                         <mailto:govtrack-digest@yahoogroups.com>>
                         <mailto:govtrack-digest@
                         <mailto:govtrack-digest@>__yaho__ogroups.com
                         <http://yahoogroups.com>
                         <mailto:govtrack-digest@__yahoogroups.com
                         <mailto:govtrack-digest@yahoogroups.com>>>
                                 govtrack-fullfeatured@__yahoog__roups.com
                         <http://yahoogroups.com>
                         <mailto:govtrack-fullfeatured@__yahoogroups.com
                         <mailto:govtrack-fullfeatured@yahoogroups.com>>
                         <mailto:govtrack-fullfeatured@
                         <mailto:govtrack-fullfeatured@>____yahoogroups.com
                         <http://yahoogroups.com>


                         <mailto:govtrack-fullfeatured@__yahoogroups.com
                         <mailto:govtrack-fullfeatured@yahoogroups.com>>>


                         <*> To unsubscribe from this group, send an email to:
                                 govtrack-unsubscribe@__yahoogr__oups.com
                         <http://yahoogroups.com>
                         <mailto:govtrack-unsubscribe@__yahoogroups.com
                         <mailto:govtrack-unsubscribe@yahoogroups.com>>
                         <mailto:govtrack-unsubscribe@
                         <mailto:govtrack-unsubscribe@>____yahoogroups.com
                         <http://yahoogroups.com>


                         <mailto:govtrack-unsubscribe@__yahoogroups.com
                         <mailto:govtrack-unsubscribe@yahoogroups.com>>>


                         <*> Your use of Yahoo! Groups is subject to:
                         http://docs.yahoo.com/info/____terms/
                         <http://docs.yahoo.com/info/__terms/>


                         <http://docs.yahoo.com/info/__terms/
                         <http://docs.yahoo.com/info/terms/>>




                                 --
                                 Developer | sunlightfoundation.com
                         <http://sunlightfoundation.com>
                         <http://sunlightfoundation.com__>
                         <http://sunlightfoundation.__com__

                         <http://sunlightfoundation.com__>>








                         --
                         Developer | sunlightfoundation.com
                         <http://sunlightfoundation.com> <http://sunlightfoundation.com__>



                  --

                • Josh Tauberer
                  ... Yeah this part is weird. The only status taken into account is whether the bill has been reported by committee. So reported bills get a boost. Enacted
                  Message 8 of 9 , May 23, 2012
                  • 0 Attachment
                    On 05/23/2012 06:37 PM, Eric Mill wrote:
                    > Sorting on prognosis is a cool idea. Does that mean that bills already
                    > passed into law will always occur at the top, or the bottom, or...?

                    Yeah this part is weird. The only status taken into account is whether
                    the bill has been reported by committee. So reported bills get a boost.
                    Enacted bills get the same boost, but no additional boost for being
                    passed/enacted/etc. So everything besides introduced bills get a boost.

                    > you'd probably want to only de-weight
                    > plain resolutions and concurrent resolutions

                    The actual weights are:

                    simple resolution .2
                    concurrent resolution: .3
                    joint resolution: .35
                    bill: 1.0

                    So I probably went too far with the joint resolutions.

                    About half of joint resolutions are proposing an amendment to the
                    constitution (=no force of law). Rather than weighting them all 0.3, I
                    should probably just see why the prognosis doesn't include a factor for
                    constitutional amendment resolutions and weight those accordingly (i.e.
                    none of those ever pass) and, overall, weight joint resolutions the same
                    as bills.

                    Thanks for the reminder about that.

                    - Josh Tauberer (@JoshData)

                    http://razor.occams.info

                    On 05/23/2012 06:37 PM, Eric Mill wrote:
                    > Sorting on prognosis is a cool idea. Does that mean that bills already
                    > passed into law will always occur at the top, or the bottom, or...?
                    >
                    > You are the last person who needs to hear the definition of resolution
                    > types, but just to make sure, you'd probably want to only de-weight
                    > plain resolutions and concurrent resolutions, and not joint resolutions
                    > (which are bills with the force of law and are signed by the President).
                    >
                    > On Wed, May 23, 2012 at 6:25 PM, Josh Tauberer <tauberer@...
                    > <mailto:tauberer@...>> wrote:
                    >
                    > On 05/17/2012 11:36 AM, Eric Mill wrote:
                    >
                    > if you also
                    > tuned out the post office renamings. Maybe that's the only way,
                    > is just
                    > to filter out specific classes of bills and resolutions.
                    >
                    >
                    > Yeah, that is worth a try. I'm now sorting bills on the advanced
                    > search by a modified-prognosis score that just omits any boost that
                    > post office bills and some other bills get, and also arbitrarily
                    > de-weights resolutions.
                    >
                    > http://www.govtrack.us/__congress/bills/browse
                    > <http://www.govtrack.us/congress/bills/browse>
                    >
                    > Think it's more useful than e.g. last action date? Looking at the
                    > results I think so but I'm not sure.
                    >
                    > (I guess I could do A/B testing with users but it's so much trouble
                    > to be quantitative.)
                    >
                    >
                    > - Josh Tauberer (@JoshData)
                    >
                    > http://razor.occams.info
                    >
                    > On 05/17/2012 11:36 AM, Eric Mill wrote:
                    >
                    > So fun.
                    >
                    > We had a nice (potential) success story at Sunlight the other
                    > day here,
                    > where we caught
                    > <https://twitter.com/#!/__JohnWonderlich/status/__202385148161953793
                    > <https://twitter.com/#!/JohnWonderlich/status/202385148161953793>> a
                    >
                    > nasty proposed FOIA exemption in a bill being actively crafted
                    > inside
                    > the House through our search engine (second item down on this list,
                    > right now:
                    > http://scout.__sunlightfoundation.com/search/__federal_bills/section%20552%__20of%20title%205
                    > <http://scout.sunlightfoundation.com/search/federal_bills/section%20552%20of%20title%205>).
                    > It has no official summary, at the time it had no official
                    > "short title"
                    > yet and was a big paragraph of text...and as you can see my
                    > highlighter
                    > is failing to even provide an excerpt for that result. John had
                    > to open
                    > up the official text on GPO and dig through the text to find out
                    > why it
                    > even came up. This was a very relevant bill, but it was only spotted
                    > because it was new and because John is a diligent guy.
                    >
                    > It had been announced for floor activity, and was introduced by the
                    > committee chair, so maybe those would help in relevance - if you
                    > also
                    > tuned out the post office renamings. Maybe that's the only way,
                    > is just
                    > to filter out specific classes of bills and resolutions.
                    >
                    > -- Eric
                    >
                    > On Thu, May 17, 2012 at 11:23 AM, Josh Tauberer
                    > <tauberer@... <mailto:tauberer@...>
                    > <mailto:tauberer@... <mailto:tauberer@...>>> wrote:
                    >
                    > Probably, yeah. Right now the API is based on the Django
                    > ORM, which
                    > hits MySQL. That's not great for search so I have to see if
                    > I can
                    > switch it transparently to using Haystack, or else build a
                    > separate
                    > API for search.
                    >
                    > But document relevance is.... ugh. Two fun facts: The
                    > highest-prognosis bills are the dumb ones like post office
                    > namings,
                    > so that's not good for relevance. And the top visited bills are
                    > almost never the ones people should really be looking at, so
                    > popularity is bad too. Fun fun.
                    >
                    >
                    > - Josh Tauberer (@JoshData)
                    >
                    > http://razor.occams.info
                    >
                    > On 05/17/2012 11:15 AM, Eric Mill wrote:
                    >
                    >
                    >
                    > Sweet. Are you going to dive into text search at all?
                    > This has
                    > become
                    > the most useful thing to come out of RTC, though I haven't
                    > documented
                    > the search methods publicly yet. It's a hard problem,
                    > but you're
                    > developing all sorts of nifty ways of calculating
                    > relevance, so
                    > you'd be
                    > in a good position to do it well.
                    >
                    > -- Eric
                    >
                    > On Thu, May 17, 2012 at 6:58 AM, Josh Tauberer
                    > <tauberer@... <mailto:tauberer@...>
                    > <mailto:tauberer@... <mailto:tauberer@...>>
                    > <mailto:tauberer@... <mailto:tauberer@...>
                    > <mailto:tauberer@... <mailto:tauberer@...>>>__>
                    > wrote:
                    >
                    > On 05/16/2012 11:39 PM, Eric Mill wrote:
                    > > Super awesome, really glad to see this happen. Nice job with the
                    > docs,
                    > > too (and a nice reminder to redo and update my own).
                    >
                    > Thanks. I drew on a few ideas from RTC, e.g. I made
                    > sure the
                    > infrastructure I chose (Tasypie) supported __[operator]
                    > filtering.
                    >
                    >
                    > > Are you using this in anything yourself now?
                    >
                    > Not yet. The main reason I made it is so when people
                    > come to me
                    > asking for CSV exports I have a way to pull them
                    > quickly. If
                    > people
                    > want to build actual apps, even better. (Only half
                    > joking.)
                    >
                    > Except that the API is tightly coupled to the site's
                    > models,
                    > so I
                    > use it at that level. (Makes it practically
                    > self-documenting
                    > too.)
                    >
                    >
                    >
                    > - Josh Tauberer (@JoshData)
                    >
                    > http://razor.occams.info
                    >
                    > On 05/16/2012 11:39 PM, Eric Mill wrote:
                    >
                    >
                    >
                    > Super awesome, really glad to see this happen.
                    > Nice job
                    > with the
                    > docs,
                    > too (and a nice reminder to redo and update my own).
                    >
                    > Are you using this in anything yourself now?
                    >
                    > -- Eric
                    >
                    > On Wed, May 16, 2012 at 9:07 PM, Josh Tauberer
                    > <tauberer@... <mailto:tauberer@...>
                    > <mailto:tauberer@... <mailto:tauberer@...>>
                    > <mailto:tauberer@... <mailto:tauberer@...>
                    > <mailto:tauberer@... <mailto:tauberer@...>>>
                    > <mailto:tauberer@... <mailto:tauberer@...>
                    > <mailto:tauberer@... <mailto:tauberer@...>>
                    > <mailto:tauberer@... <mailto:tauberer@...>
                    > <mailto:tauberer@... <mailto:tauberer@...>>>__>__>
                    >
                    > wrote:
                    >
                    > I've put this off just about long enough-
                    >
                    > GovTrack now has an API!
                    >
                    > Currently bills and people are included in
                    > the API.
                    > (Votes and
                    > committees will come later if there's interest.)
                    >
                    > More here:
                    >
                    > http://www.govtrack.us/______developers/api
                    > <http://www.govtrack.us/____developers/api>
                    > <http://www.govtrack.us/____developers/api
                    > <http://www.govtrack.us/__developers/api>>
                    >
                    >
                    > <http://www.govtrack.us/____developers/api
                    > <http://www.govtrack.us/__developers/api>
                    > <http://www.govtrack.us/__developers/api
                    > <http://www.govtrack.us/developers/api>>>
                    >
                    > It's experimental at the moment. I don't
                    > have any
                    > caching on
                    > it, no API
                    > keys, etc. Go easy!
                    >
                    > --
                    > - Josh Tauberer (@JoshData)
                    >
                    > http://razor.occams.info
                    >
                    >
                    > ------------------------------______------
                    >
                    >
                    >
                    > Yahoo! Groups Links
                    >
                    > <http://groups.yahoo.com/__group/__govtrack/
                    > <http://groups.yahoo.com/group/__govtrack/>>
                    >
                    > <http://groups.yahoo.com/____group/govtrack/
                    > <http://groups.yahoo.com/__group/govtrack/>
                    >
                    > <http://groups.yahoo.com/__group/govtrack/
                    > <http://groups.yahoo.com/group/govtrack/>>>
                    >
                    > <http://groups.yahoo.com/__group/__govtrack/join
                    > <http://groups.yahoo.com/group/__govtrack/join>>
                    >
                    > <http://groups.yahoo.com/____group/govtrack/join
                    > <http://groups.yahoo.com/__group/govtrack/join>
                    >
                    > <http://groups.yahoo.com/__group/govtrack/join
                    > <http://groups.yahoo.com/group/govtrack/join>>>
                    > (Yahoo! ID required)
                    >
                    > <mailto:govtrack-digest@>__yaho__ogroups.com
                    > <http://yahoogroups.com>
                    > <mailto:govtrack-digest@__yahoogroups.com
                    > <mailto:govtrack-digest@yahoogroups.com>>>
                    > <mailto:govtrack-digest@ <mailto:govtrack-digest@>
                    > <mailto:govtrack-digest@
                    > <mailto:govtrack-digest@>>__yah__o__ogroups.com
                    > <http://yaho__ogroups.com>
                    > <http://yahoogroups.com>
                    > <mailto:govtrack-digest@
                    > <mailto:govtrack-digest@>__yaho__ogroups.com
                    > <http://yahoogroups.com>
                    > <mailto:govtrack-digest@__yahoogroups.com
                    > <mailto:govtrack-digest@yahoogroups.com>>>>
                    > govtrack-fullfeatured@__yahoog____roups.com
                    > <http://yahoog__roups.com>
                    > <http://yahoogroups.com>
                    > <mailto:govtrack-fullfeatured@
                    > <mailto:govtrack-fullfeatured@>____yahoogroups.com
                    > <http://yahoogroups.com>
                    > <mailto:govtrack-fullfeatured@__yahoogroups.com
                    > <mailto:govtrack-fullfeatured@yahoogroups.com>>>
                    > <mailto:govtrack-fullfeatured@ <mailto:govtrack-fullfeatured@>
                    > <mailto:govtrack-fullfeatured@
                    > <mailto:govtrack-fullfeatured@>__>____yahoogroups.com
                    > <http://yahoogroups.com>
                    > <http://yahoogroups.com>
                    >
                    >
                    > <mailto:govtrack-fullfeatured@
                    > <mailto:govtrack-fullfeatured@>____yahoogroups.com
                    > <http://yahoogroups.com>
                    > <mailto:govtrack-fullfeatured@__yahoogroups.com
                    > <mailto:govtrack-fullfeatured@yahoogroups.com>>>>
                    >
                    >
                    > <http://yahoogroups.com>
                    > <mailto:govtrack-unsubscribe@
                    > <mailto:govtrack-unsubscribe@>____yahoogroups.com
                    > <http://yahoogroups.com>
                    > <mailto:govtrack-unsubscribe@__yahoogroups.com
                    > <mailto:govtrack-unsubscribe@yahoogroups.com>>>
                    > <mailto:govtrack-unsubscribe@ <mailto:govtrack-unsubscribe@>
                    > <mailto:govtrack-unsubscribe@
                    > <mailto:govtrack-unsubscribe@>>______yahoogroups.com
                    > <http://yahoogroups.com>
                    > <http://yahoogroups.com>
                    >
                    >
                    > <mailto:govtrack-unsubscribe@
                    > <mailto:govtrack-unsubscribe@>____yahoogroups.com
                    > <http://yahoogroups.com>
                    > <mailto:govtrack-unsubscribe@__yahoogroups.com
                    > <mailto:govtrack-unsubscribe@yahoogroups.com>>>>
                    >
                    >
                    > <http://docs.yahoo.com/info/____terms/
                    > <http://docs.yahoo.com/info/__terms/>>
                    >
                    >
                    > <http://docs.yahoo.com/info/____terms/
                    > <http://docs.yahoo.com/info/__terms/>
                    > <http://docs.yahoo.com/info/__terms/
                    > <http://docs.yahoo.com/info/terms/>>>
                    >
                    >
                    >
                    >
                    > --
                    > Developer | sunlightfoundation.com
                    > <http://sunlightfoundation.com>
                    > <http://sunlightfoundation.com__>
                    > <http://sunlightfoundation.__com__
                    > <http://sunlightfoundation.com__>>
                    > <http://sunlightfoundation.____com__
                    >
                    > <http://sunlightfoundation.__com__
                    > <http://sunlightfoundation.com__>>>
                    >
                    >
                    >
                    >
                    >
                    >
                    >
                    >
                    > --
                    > Developer | sunlightfoundation.com
                    > <http://sunlightfoundation.com>
                    > <http://sunlightfoundation.com__>
                    > <http://sunlightfoundation.__com__
                    > <http://sunlightfoundation.com__>>
                    >
                    >
                    >
                    >
                    >
                    >
                    >
                    >
                    >
                    > --
                    > Developer | sunlightfoundation.com
                    > <http://sunlightfoundation.com> <http://sunlightfoundation.com__>
                    >
                    >
                    >
                    >
                    > --
                    > Developer | sunlightfoundation.com <http://sunlightfoundation.com>
                    >
                    >
                    >
                    >
                  Your message has been successfully submitted and would be delivered to recipients shortly.