FYI to anyone tracking Senate votes, two senators changed their vote in
roll call #28. The file on GovTrack has been updated.

John, writing below & attached, indicated there might have been
confusion between votes #28 and #29, but I haven't substantiated that.
(If it is true, the congressional record may already have been revised.)

Josh

-------- Original Message --------
Subject:  Vote confusion and problems with Senate Roll Call #28
Date:  Fri, 9 Mar 2012 17:15:11 -0500
From:  John McComb <john@...>
To:  'Joshua Tauberer' <jt@...>



Josh,

Yesterday there was some confusion by two Senators on their votes on the
Vitter amendment, Roll Call #28. They changed their vote, but referred
to the wrong Roll Call #.

The end result was that the Senate vote XML file for Roll Call #28 was
wrong until early this morning. It has now been corrected.

You have the wrong vote information for Senators Collins and Murkowski
who should be shown as voting Nay on Roll Call #28 with results of 44Y-54N.

The attached email is from my exchange with the Secretary of the Senate
about this issue and the fact that they have now corrected the XML file.

John

------------------------------------------------------------------------

John A. McComb
*Wayburn Wilderness House*
122 C St NW Ste 240
Washington, DC 20001-2109
202-266-0433 (work)
703-801-4351 (cell)
202-628-1988 (fax)
john@... <mailto:john@...>
www.jmccomb.com <http://www.jmccomb.com/>

Hi, all.

I know I already gave someone a real headache for not giving a heads up
and feel bad about that.

If you haven't seen, GovTrack is now on "2.0". The whole web backend got
rewritten. Everything public-facing has changed.

If anyone is using one of the GovTrack "APIs" --- i.e. an HTTP call to a
URL with an .xpd extension --- those are now running at a different
location: http://legacy.govtrack.us rather than www.govtrack.us.

These URLs did not yet make it over to the new site and probably will
look completely different anyway when I start building proper APIs.

If you don't use those APIs, nothing to worry about. Nothing changed in
the data files.

Thanks, and apologies if I messed things up for you.

--
- Josh Tauberer (@JoshData)

http://razor.occams.info

At least one bill's xml has invalid characters:
http://www.govtrack.us/data/us/112/bills/h4517.xml

The reason for this is that the source on thomas (maybe ultimately at CRS) has
invalid characters. So far what I've been seeing is the SUB (0x1A) control
character.

Other not-yet-scraped bills have this issue too (these are just the ones I have
identified):
http://hdl.loc.gov/loc.uscongress/legislation.112hr4715
http://hdl.loc.gov/loc.uscongress/legislation.112hr4716

I discovered this issue through the scraper at washingtonwatch

At the very least the scraper needs to strip out invalid characters, but
ultimately the upstream source of the bad characters needs to be fixed as well.
Anyone know who to contact for that?

Thanks, Francis.

I corrected the XML file and the scraper yesterday afternoon, so
hopefully the data is now good.

And I let my LOC contact know about HR4517 but I'll also pass on the
other two --- thanks for those links.

- Josh Tauberer (@JoshData)

http://razor.occams.info

On 04/27/2012 12:19 PM, Francis wrote:
> At least one bill's xml has invalid characters:
> http://www.govtrack.us/data/us/112/bills/h4517.xml
>
> The reason for this is that the source on thomas (maybe ultimately at CRS) has
invalid characters. So far what I've been seeing is the SUB (0x1A) control
character.
>
> Other not-yet-scraped bills have this issue too (these are just the ones I
have identified):
> http://hdl.loc.gov/loc.uscongress/legislation.112hr4715
> http://hdl.loc.gov/loc.uscongress/legislation.112hr4716
>
> I discovered this issue through the scraper at washingtonwatch
>
> At the very least the scraper needs to strip out invalid characters, but
ultimately the upstream source of the bad characters needs to be fixed as well.
Anyone know who to contact for that?
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>

I'm looking for machine-readable Bioguide data, in particular the short
biographical paragraphs for currently-serving congressmen.

I know about bioguide html: http://bioguide.congress.gov/
I know that a bioguide xml dtd exists: http://xml.house.gov/#bioguide
I know that at least four bioguide xml documents exist:
http://xml.house.gov/bioguide/BioguideXML.zip

I know that the biographical text on the bioguide website corresponds exactly to
the text in the four bioguide xml documents.

I know that in January of 2009, Josh produced a dump of bioguide data scraped
from the html site into CSVs:
http://groups.yahoo.com/group/govtrack/message/696
http://www.govtrack.us/data/us/bioguide3.csv

It was also scraped by Bill Farrell, and the data was at a now-defunct
pythia.progressivenation.net:
http://groups.yahoo.com/group/govtrack/message/55
http://pythia.progressivenation.net/modules/tinycontent3/index.php?id=15

I strongly suspect that the bioguide HTML is generated directly from the
bioguide XML, or at least from some source common to both.

However, *where is the bioguide XML*?

I re-ran my bioguide scraper and updated those files:

http://www.govtrack.us/data/us/bioguide1.csv
http://www.govtrack.us/data/us/bioguide2.csv
http://www.govtrack.us/data/us/bioguide3.csv

However, something changed in the character encoding so it looks like
non-ascii characters are now messed up or in something besides utf8.

Looking at a diff of changes there were some corrections in the data and
some people were removed... I am hoping the removed identifiers were for
dupes for name changes, but I'd have to look more closely to know for sure.

Deletions obviously pose an interesting problem if anyone is using
bioguide IDs as primary keys!

- Josh Tauberer (@JoshData)

http://razor.occams.info

On 05/04/2012 06:30 PM, Francis wrote:
> I'm looking for machine-readable Bioguide data, in particular the short
biographical paragraphs for currently-serving congressmen.
>
> I know about bioguide html: http://bioguide.congress.gov/
> I know that a bioguide xml dtd exists: http://xml.house.gov/#bioguide
> I know that at least four bioguide xml documents exist:
http://xml.house.gov/bioguide/BioguideXML.zip
>
> I know that the biographical text on the bioguide website corresponds exactly
to the text in the four bioguide xml documents.
>
> I know that in January of 2009, Josh produced a dump of bioguide data scraped
from the html site into CSVs:
> http://groups.yahoo.com/group/govtrack/message/696
> http://www.govtrack.us/data/us/bioguide3.csv
>
> It was also scraped by Bill Farrell, and the data was at a now-defunct
pythia.progressivenation.net:
> http://groups.yahoo.com/group/govtrack/message/55
> http://pythia.progressivenation.net/modules/tinycontent3/index.php?id=15
>
> I strongly suspect that the bioguide HTML is generated directly from the
bioguide XML, or at least from some source common to both.
>
> However, *where is the bioguide XML*?
>
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>

I've put this off just about long enough-

GovTrack now has an API!

Currently bills and people are included in the API. (Votes and
committees will come later if there's interest.)

More here:

http://www.govtrack.us/developers/api

It's experimental at the moment. I don't have any caching on it, no API
keys, etc. Go easy!

--
- Josh Tauberer (@JoshData)

http://razor.occams.info

On 05/16/2012 11:39 PM, Eric Mill wrote:
  > Super awesome, really glad to see this happen. Nice job with the docs,
  > too (and a nice reminder to redo and update my own).

Thanks. I drew on a few ideas from RTC, e.g. I made sure the
infrastructure I chose (Tasypie) supported __[operator] filtering.

  > Are you using this in anything yourself now?

Not yet. The main reason I made it is so when people come to me asking
for CSV exports I have a way to pull them quickly. If people want to
build actual apps, even better. (Only half joking.)

Except that the API is tightly coupled to the site's models, so I use it
at that level. (Makes it practically self-documenting too.)


- Josh Tauberer (@JoshData)

http://razor.occams.info

On 05/16/2012 11:39 PM, Eric Mill wrote:
>
>
> Super awesome, really glad to see this happen. Nice job with the docs,
> too (and a nice reminder to redo and update my own).
>
> Are you using this in anything yourself now?
>
> -- Eric
>
> On Wed, May 16, 2012 at 9:07 PM, Josh Tauberer <tauberer@...
> <mailto:tauberer@...>> wrote:
>
>     I've put this off just about long enough-
>
>     GovTrack now has an API!
>
>     Currently bills and people are included in the API. (Votes and
>     committees will come later if there's interest.)
>
>     More here:
>
>     http://www.govtrack.us/developers/api
>
>     It's experimental at the moment. I don't have any caching on it, no API
>     keys, etc. Go easy!
>
>     --
>     - Josh Tauberer (@JoshData)
>
>     http://razor.occams.info
>
>
>     ------------------------------------
>
>     Yahoo! Groups Links
>
>
>     <mailto:govtrack-fullfeatured@yahoogroups.com>
>
>
>
>
>
> --
> Developer | sunlightfoundation.com <http://sunlightfoundation.com>
>
>
>
>

Probably, yeah. Right now the API is based on the Django ORM, which hits
MySQL. That's not great for search so I have to see if I can switch it
transparently to using Haystack, or else build a separate API for search.

But document relevance is.... ugh. Two fun facts: The highest-prognosis
bills are the dumb ones like post office namings, so that's not good for
relevance. And the top visited bills are almost never the ones people
should really be looking at, so popularity is bad too. Fun fun.

- Josh Tauberer (@JoshData)

http://razor.occams.info

On 05/17/2012 11:15 AM, Eric Mill wrote:
>
>
> Sweet. Are you going to dive into text search at all? This has become
> the most useful thing to come out of RTC, though I haven't documented
> the search methods publicly yet. It's a hard problem, but you're
> developing all sorts of nifty ways of calculating relevance, so you'd be
> in a good position to do it well.
>
> -- Eric
>
> On Thu, May 17, 2012 at 6:58 AM, Josh Tauberer <tauberer@...
> <mailto:tauberer@...>> wrote:
>
>     On 05/16/2012 11:39 PM, Eric Mill wrote:
>      > Super awesome, really glad to see this happen. Nice job with the
>     docs,
>      > too (and a nice reminder to redo and update my own).
>
>     Thanks. I drew on a few ideas from RTC, e.g. I made sure the
>     infrastructure I chose (Tasypie) supported __[operator] filtering.
>
>
>      > Are you using this in anything yourself now?
>
>     Not yet. The main reason I made it is so when people come to me
>     asking for CSV exports I have a way to pull them quickly. If people
>     want to build actual apps, even better. (Only half joking.)
>
>     Except that the API is tightly coupled to the site's models, so I
>     use it at that level. (Makes it practically self-documenting too.)
>
>
>
>     - Josh Tauberer (@JoshData)
>
>     http://razor.occams.info
>
>     On 05/16/2012 11:39 PM, Eric Mill wrote:
>
>
>
>         Super awesome, really glad to see this happen. Nice job with the
>         docs,
>         too (and a nice reminder to redo and update my own).
>
>         Are you using this in anything yourself now?
>
>         -- Eric
>
>         On Wed, May 16, 2012 at 9:07 PM, Josh Tauberer
>         <tauberer@... <mailto:tauberer@...>
>         <mailto:tauberer@... <mailto:tauberer@...>>> wrote:
>
>             I've put this off just about long enough-
>
>             GovTrack now has an API!
>
>             Currently bills and people are included in the API. (Votes and
>             committees will come later if there's interest.)
>
>             More here:
>
>         http://www.govtrack.us/__developers/api
>         <http://www.govtrack.us/developers/api>
>
>             It's experimental at the moment. I don't have any caching on
>         it, no API
>             keys, etc. Go easy!
>
>             --
>             - Josh Tauberer (@JoshData)
>
>         http://razor.occams.info
>
>
>             ------------------------------__------
>
>             Yahoo! Groups Links
>
>
>                 (Yahoo! ID required)
>
>         <mailto:govtrack-digest@__yahoogroups.com
>         <mailto:govtrack-digest@yahoogroups.com>>
>         govtrack-fullfeatured@__yahoogroups.com
>         <mailto:govtrack-fullfeatured@yahoogroups.com>
>         <mailto:govtrack-fullfeatured@__yahoogroups.com
>         <mailto:govtrack-fullfeatured@yahoogroups.com>>
>
>
>         <mailto:govtrack-unsubscribe@__yahoogroups.com
>         <mailto:govtrack-unsubscribe@yahoogroups.com>>
>
>
>
>
>
>
>         --
>         Developer | sunlightfoundation.com
>         <http://sunlightfoundation.com> <http://sunlightfoundation.com__>
>
>
>
>
>
>
>
> --
> Developer | sunlightfoundation.com <http://sunlightfoundation.com>
>
>
>
>

On 05/17/2012 11:36 AM, Eric Mill wrote:
> if you also
> tuned out the post office renamings. Maybe that's the only way, is just
> to filter out specific classes of bills and resolutions.

Yeah, that is worth a try. I'm now sorting bills on the advanced search
by a modified-prognosis score that just omits any boost that post office
bills and some other bills get, and also arbitrarily de-weights resolutions.

http://www.govtrack.us/congress/bills/browse

Think it's more useful than e.g. last action date? Looking at the
results I think so but I'm not sure.

(I guess I could do A/B testing with users but it's so much trouble to
be quantitative.)

- Josh Tauberer (@JoshData)

http://razor.occams.info

On 05/17/2012 11:36 AM, Eric Mill wrote:
> So fun.
>
> We had a nice (potential) success story at Sunlight the other day here,
> where we caught
> <https://twitter.com/#!/JohnWonderlich/status/202385148161953793> a
> nasty proposed FOIA exemption in a bill being actively crafted inside
> the House through our search engine (second item down on this list,
> right now:
>
http://scout.sunlightfoundation.com/search/federal_bills/section%20552%20of%20ti\
tle%205).
> It has no official summary, at the time it had no official "short title"
> yet and was a big paragraph of text...and as you can see my highlighter
> is failing to even provide an excerpt for that result. John had to open
> up the official text on GPO and dig through the text to find out why it
> even came up. This was a very relevant bill, but it was only spotted
> because it was new and because John is a diligent guy.
>
> It had been announced for floor activity, and was introduced by the
> committee chair, so maybe those would help in relevance - if you also
> tuned out the post office renamings. Maybe that's the only way, is just
> to filter out specific classes of bills and resolutions.
>
> -- Eric
>
> On Thu, May 17, 2012 at 11:23 AM, Josh Tauberer <tauberer@...
> <mailto:tauberer@...>> wrote:
>
>     Probably, yeah. Right now the API is based on the Django ORM, which
>     hits MySQL. That's not great for search so I have to see if I can
>     switch it transparently to using Haystack, or else build a separate
>     API for search.
>
>     But document relevance is.... ugh. Two fun facts: The
>     highest-prognosis bills are the dumb ones like post office namings,
>     so that's not good for relevance. And the top visited bills are
>     almost never the ones people should really be looking at, so
>     popularity is bad too. Fun fun.
>
>
>     - Josh Tauberer (@JoshData)
>
>     http://razor.occams.info
>
>     On 05/17/2012 11:15 AM, Eric Mill wrote:
>
>
>
>         Sweet. Are you going to dive into text search at all? This has
>         become
>         the most useful thing to come out of RTC, though I haven't
>         documented
>         the search methods publicly yet. It's a hard problem, but you're
>         developing all sorts of nifty ways of calculating relevance, so
>         you'd be
>         in a good position to do it well.
>
>         -- Eric
>
>         On Thu, May 17, 2012 at 6:58 AM, Josh Tauberer
>         <tauberer@... <mailto:tauberer@...>
>         <mailto:tauberer@... <mailto:tauberer@...>>> wrote:
>
>             On 05/16/2012 11:39 PM, Eric Mill wrote:
>          > Super awesome, really glad to see this happen. Nice job with the
>             docs,
>          > too (and a nice reminder to redo and update my own).
>
>             Thanks. I drew on a few ideas from RTC, e.g. I made sure the
>             infrastructure I chose (Tasypie) supported __[operator]
>         filtering.
>
>
>          > Are you using this in anything yourself now?
>
>             Not yet. The main reason I made it is so when people come to me
>             asking for CSV exports I have a way to pull them quickly. If
>         people
>             want to build actual apps, even better. (Only half joking.)
>
>             Except that the API is tightly coupled to the site's models,
>         so I
>             use it at that level. (Makes it practically self-documenting
>         too.)
>
>
>
>             - Josh Tauberer (@JoshData)
>
>         http://razor.occams.info
>
>             On 05/16/2012 11:39 PM, Eric Mill wrote:
>
>
>
>                 Super awesome, really glad to see this happen. Nice job
>         with the
>                 docs,
>                 too (and a nice reminder to redo and update my own).
>
>                 Are you using this in anything yourself now?
>
>                 -- Eric
>
>                 On Wed, May 16, 2012 at 9:07 PM, Josh Tauberer
>         <tauberer@... <mailto:tauberer@...>
>         <mailto:tauberer@... <mailto:tauberer@...>>
>         <mailto:tauberer@... <mailto:tauberer@...>
>         <mailto:tauberer@... <mailto:tauberer@...>>>__>
>         wrote:
>
>                     I've put this off just about long enough-
>
>                     GovTrack now has an API!
>
>                     Currently bills and people are included in the API.
>         (Votes and
>                     committees will come later if there's interest.)
>
>                     More here:
>
>         http://www.govtrack.us/____developers/api
>         <http://www.govtrack.us/__developers/api>
>
>         <http://www.govtrack.us/__developers/api
>         <http://www.govtrack.us/developers/api>>
>
>                     It's experimental at the moment. I don't have any
>         caching on
>                 it, no API
>                     keys, etc. Go easy!
>
>                     --
>                     - Josh Tauberer (@JoshData)
>
>         http://razor.occams.info
>
>
>                     ------------------------------____------
>
>
>                     Yahoo! Groups Links
>
>
>         <http://groups.yahoo.com/__group/govtrack/
>         <http://groups.yahoo.com/group/govtrack/>>
>
>
>         <http://groups.yahoo.com/__group/govtrack/join
>         <http://groups.yahoo.com/group/govtrack/join>>
>                         (Yahoo! ID required)
>
>         <mailto:govtrack-digest@yahoogroups.com>>
>         <mailto:govtrack-digest@
>         <mailto:govtrack-digest@>__yaho__ogroups.com
>         <http://yahoogroups.com>
>         <mailto:govtrack-digest@__yahoogroups.com
>         <mailto:govtrack-digest@yahoogroups.com>>>
>                 govtrack-fullfeatured@__yahoog__roups.com
>         <http://yahoogroups.com>
>         <mailto:govtrack-fullfeatured@__yahoogroups.com
>         <mailto:govtrack-fullfeatured@yahoogroups.com>>
>         <mailto:govtrack-fullfeatured@
>         <mailto:govtrack-fullfeatured@>____yahoogroups.com
>         <http://yahoogroups.com>
>
>         <mailto:govtrack-fullfeatured@__yahoogroups.com
>         <mailto:govtrack-fullfeatured@yahoogroups.com>>>
>
>
>         <mailto:govtrack-unsubscribe@__yahoogroups.com
>         <mailto:govtrack-unsubscribe@yahoogroups.com>>
>         <mailto:govtrack-unsubscribe@
>         <mailto:govtrack-unsubscribe@>____yahoogroups.com
>         <http://yahoogroups.com>
>
>         <mailto:govtrack-unsubscribe@__yahoogroups.com
>         <mailto:govtrack-unsubscribe@yahoogroups.com>>>
>
>
>
>         <http://docs.yahoo.com/info/__terms/
>         <http://docs.yahoo.com/info/terms/>>
>
>
>
>
>                 --
>                 Developer | sunlightfoundation.com
>         <http://sunlightfoundation.com>
>         <http://sunlightfoundation.com__>
>         <http://sunlightfoundation.__com__
>         <http://sunlightfoundation.com__>>
>
>
>
>
>
>
>
>
>         --
>         Developer | sunlightfoundation.com
>         <http://sunlightfoundation.com> <http://sunlightfoundation.com__>
>
>
>
>
>
>
>
>
> --
> Developer | sunlightfoundation.com <http://sunlightfoundation.com>
>

On 05/23/2012 06:37 PM, Eric Mill wrote:
  > Sorting on prognosis is a cool idea. Does that mean that bills already
  > passed into law will always occur at the top, or the bottom, or...?

Yeah this part is weird. The only status taken into account is whether
the bill has been reported by committee. So reported bills get a boost.
Enacted bills get the same boost, but no additional boost for being
passed/enacted/etc. So everything besides introduced bills get a boost.

  > you'd probably want to only de-weight
  > plain resolutions and concurrent resolutions

The actual weights are:

     simple resolution .2
     concurrent resolution: .3
     joint resolution: .35
     bill: 1.0

So I probably went too far with the joint resolutions.

About half of joint resolutions are proposing an amendment to the
constitution (=no force of law). Rather than weighting them all 0.3, I
should probably just see why the prognosis doesn't include a factor for
constitutional amendment resolutions and weight those accordingly (i.e.
none of those ever pass) and, overall, weight joint resolutions the same
as bills.

Thanks for the reminder about that.

- Josh Tauberer (@JoshData)

http://razor.occams.info

On 05/23/2012 06:37 PM, Eric Mill wrote:
> Sorting on prognosis is a cool idea. Does that mean that bills already
> passed into law will always occur at the top, or the bottom, or...?
>
> You are the last person who needs to hear the definition of resolution
> types, but just to make sure, you'd probably want to only de-weight
> plain resolutions and concurrent resolutions, and not joint resolutions
> (which are bills with the force of law and are signed by the President).
>
> On Wed, May 23, 2012 at 6:25 PM, Josh Tauberer <tauberer@...
> <mailto:tauberer@...>> wrote:
>
>     On 05/17/2012 11:36 AM, Eric Mill wrote:
>
>         if you also
>         tuned out the post office renamings. Maybe that's the only way,
>         is just
>         to filter out specific classes of bills and resolutions.
>
>
>     Yeah, that is worth a try. I'm now sorting bills on the advanced
>     search by a modified-prognosis score that just omits any boost that
>     post office bills and some other bills get, and also arbitrarily
>     de-weights resolutions.
>
>     http://www.govtrack.us/__congress/bills/browse
>     <http://www.govtrack.us/congress/bills/browse>
>
>     Think it's more useful than e.g. last action date? Looking at the
>     results I think so but I'm not sure.
>
>     (I guess I could do A/B testing with users but it's so much trouble
>     to be quantitative.)
>
>
>     - Josh Tauberer (@JoshData)
>
>     http://razor.occams.info
>
>     On 05/17/2012 11:36 AM, Eric Mill wrote:
>
>         So fun.
>
>         We had a nice (potential) success story at Sunlight the other
>         day here,
>         where we caught
>         <https://twitter.com/#!/__JohnWonderlich/status/__202385148161953793
>         <https://twitter.com/#!/JohnWonderlich/status/202385148161953793>> a
>
>         nasty proposed FOIA exemption in a bill being actively crafted
>         inside
>         the House through our search engine (second item down on this list,
>         right now:
>        
http://scout.__sunlightfoundation.com/search/__federal_bills/section%20552%__20o\
f%20title%205
>        
<http://scout.sunlightfoundation.com/search/federal_bills/section%20552%20of%20t\
itle%205>).
>         It has no official summary, at the time it had no official
>         "short title"
>         yet and was a big paragraph of text...and as you can see my
>         highlighter
>         is failing to even provide an excerpt for that result. John had
>         to open
>         up the official text on GPO and dig through the text to find out
>         why it
>         even came up. This was a very relevant bill, but it was only spotted
>         because it was new and because John is a diligent guy.
>
>         It had been announced for floor activity, and was introduced by the
>         committee chair, so maybe those would help in relevance - if you
>         also
>         tuned out the post office renamings. Maybe that's the only way,
>         is just
>         to filter out specific classes of bills and resolutions.
>
>         -- Eric
>
>         On Thu, May 17, 2012 at 11:23 AM, Josh Tauberer
>         <tauberer@... <mailto:tauberer@...>
>         <mailto:tauberer@... <mailto:tauberer@...>>> wrote:
>
>             Probably, yeah. Right now the API is based on the Django
>         ORM, which
>             hits MySQL. That's not great for search so I have to see if
>         I can
>             switch it transparently to using Haystack, or else build a
>         separate
>             API for search.
>
>             But document relevance is.... ugh. Two fun facts: The
>             highest-prognosis bills are the dumb ones like post office
>         namings,
>             so that's not good for relevance. And the top visited bills are
>             almost never the ones people should really be looking at, so
>             popularity is bad too. Fun fun.
>
>
>             - Josh Tauberer (@JoshData)
>
>         http://razor.occams.info
>
>             On 05/17/2012 11:15 AM, Eric Mill wrote:
>
>
>
>                 Sweet. Are you going to dive into text search at all?
>         This has
>                 become
>                 the most useful thing to come out of RTC, though I haven't
>                 documented
>                 the search methods publicly yet. It's a hard problem,
>         but you're
>                 developing all sorts of nifty ways of calculating
>         relevance, so
>                 you'd be
>                 in a good position to do it well.
>
>                 -- Eric
>
>                 On Thu, May 17, 2012 at 6:58 AM, Josh Tauberer
>         <tauberer@... <mailto:tauberer@...>
>         <mailto:tauberer@... <mailto:tauberer@...>>
>         <mailto:tauberer@... <mailto:tauberer@...>
>         <mailto:tauberer@... <mailto:tauberer@...>>>__>
>         wrote:
>
>                     On 05/16/2012 11:39 PM, Eric Mill wrote:
>          > Super awesome, really glad to see this happen. Nice job with the
>                     docs,
>          > too (and a nice reminder to redo and update my own).
>
>                     Thanks. I drew on a few ideas from RTC, e.g. I made
>         sure the
>                     infrastructure I chose (Tasypie) supported __[operator]
>                 filtering.
>
>
>          > Are you using this in anything yourself now?
>
>                     Not yet. The main reason I made it is so when people
>         come to me
>                     asking for CSV exports I have a way to pull them
>         quickly. If
>                 people
>                     want to build actual apps, even better. (Only half
>         joking.)
>
>                     Except that the API is tightly coupled to the site's
>         models,
>                 so I
>                     use it at that level. (Makes it practically
>         self-documenting
>                 too.)
>
>
>
>                     - Josh Tauberer (@JoshData)
>
>         http://razor.occams.info
>
>                     On 05/16/2012 11:39 PM, Eric Mill wrote:
>
>
>
>                         Super awesome, really glad to see this happen.
>         Nice job
>                 with the
>                         docs,
>                         too (and a nice reminder to redo and update my own).
>
>                         Are you using this in anything yourself now?
>
>                         -- Eric
>
>                         On Wed, May 16, 2012 at 9:07 PM, Josh Tauberer
>         <tauberer@... <mailto:tauberer@...>
>         <mailto:tauberer@... <mailto:tauberer@...>>
>         <mailto:tauberer@... <mailto:tauberer@...>
>         <mailto:tauberer@... <mailto:tauberer@...>>>
>         <mailto:tauberer@... <mailto:tauberer@...>
>         <mailto:tauberer@... <mailto:tauberer@...>>
>         <mailto:tauberer@... <mailto:tauberer@...>
>         <mailto:tauberer@... <mailto:tauberer@...>>>__>__>
>
>                 wrote:
>
>                             I've put this off just about long enough-
>
>                             GovTrack now has an API!
>
>                             Currently bills and people are included in
>         the API.
>                 (Votes and
>                             committees will come later if there's interest.)
>
>                             More here:
>
>         http://www.govtrack.us/______developers/api
>         <http://www.govtrack.us/____developers/api>
>         <http://www.govtrack.us/____developers/api
>         <http://www.govtrack.us/__developers/api>>
>
>
>         <http://www.govtrack.us/____developers/api
>         <http://www.govtrack.us/__developers/api>
>         <http://www.govtrack.us/__developers/api
>         <http://www.govtrack.us/developers/api>>>
>
>                             It's experimental at the moment. I don't
>         have any
>                 caching on
>                         it, no API
>                             keys, etc. Go easy!
>
>                             --
>                             - Josh Tauberer (@JoshData)
>
>         http://razor.occams.info
>
>
>                             ------------------------------______------
>
>
>
>                             Yahoo! Groups Links
>
>         <http://groups.yahoo.com/__group/__govtrack/
>         <http://groups.yahoo.com/group/__govtrack/>>
>
>         <http://groups.yahoo.com/____group/govtrack/
>         <http://groups.yahoo.com/__group/govtrack/>
>
>         <http://groups.yahoo.com/__group/govtrack/
>         <http://groups.yahoo.com/group/govtrack/>>>
>
>         <http://groups.yahoo.com/__group/__govtrack/join
>         <http://groups.yahoo.com/group/__govtrack/join>>
>
>         <http://groups.yahoo.com/____group/govtrack/join
>         <http://groups.yahoo.com/__group/govtrack/join>
>
>         <http://groups.yahoo.com/__group/govtrack/join
>         <http://groups.yahoo.com/group/govtrack/join>>>
>                                 (Yahoo! ID required)
>
>         <mailto:govtrack-digest@>__yaho__ogroups.com
>         <http://yahoogroups.com>
>         <mailto:govtrack-digest@__yahoogroups.com
>         <mailto:govtrack-digest@yahoogroups.com>>>
>         <mailto:govtrack-digest@ <mailto:govtrack-digest@>
>         <mailto:govtrack-digest@
>         <mailto:govtrack-digest@>>__yah__o__ogroups.com
>         <http://yaho__ogroups.com>
>         <http://yahoogroups.com>
>         <mailto:govtrack-digest@
>         <mailto:govtrack-digest@>__yaho__ogroups.com
>         <http://yahoogroups.com>
>         <mailto:govtrack-digest@__yahoogroups.com
>         <mailto:govtrack-digest@yahoogroups.com>>>>
>                         govtrack-fullfeatured@__yahoog____roups.com
>         <http://yahoog__roups.com>
>         <http://yahoogroups.com>
>         <mailto:govtrack-fullfeatured@
>         <mailto:govtrack-fullfeatured@>____yahoogroups.com
>         <http://yahoogroups.com>
>         <mailto:govtrack-fullfeatured@__yahoogroups.com
>         <mailto:govtrack-fullfeatured@yahoogroups.com>>>
>         <mailto:govtrack-fullfeatured@ <mailto:govtrack-fullfeatured@>
>         <mailto:govtrack-fullfeatured@
>         <mailto:govtrack-fullfeatured@>__>____yahoogroups.com
>         <http://yahoogroups.com>
>         <http://yahoogroups.com>
>
>
>         <mailto:govtrack-fullfeatured@
>         <mailto:govtrack-fullfeatured@>____yahoogroups.com
>         <http://yahoogroups.com>
>         <mailto:govtrack-fullfeatured@__yahoogroups.com
>         <mailto:govtrack-fullfeatured@yahoogroups.com>>>>
>
>
>         <http://yahoogroups.com>
>         <mailto:govtrack-unsubscribe@
>         <mailto:govtrack-unsubscribe@>____yahoogroups.com
>         <http://yahoogroups.com>
>         <mailto:govtrack-unsubscribe@__yahoogroups.com
>         <mailto:govtrack-unsubscribe@yahoogroups.com>>>
>         <mailto:govtrack-unsubscribe@ <mailto:govtrack-unsubscribe@>
>         <mailto:govtrack-unsubscribe@
>         <mailto:govtrack-unsubscribe@>>______yahoogroups.com
>         <http://yahoogroups.com>
>         <http://yahoogroups.com>
>
>
>         <mailto:govtrack-unsubscribe@
>         <mailto:govtrack-unsubscribe@>____yahoogroups.com
>         <http://yahoogroups.com>
>         <mailto:govtrack-unsubscribe@__yahoogroups.com
>         <mailto:govtrack-unsubscribe@yahoogroups.com>>>>
>
>
>         <http://docs.yahoo.com/info/____terms/
>         <http://docs.yahoo.com/info/__terms/>>
>
>
>         <http://docs.yahoo.com/info/____terms/
>         <http://docs.yahoo.com/info/__terms/>
>         <http://docs.yahoo.com/info/__terms/
>         <http://docs.yahoo.com/info/terms/>>>
>
>
>
>
>                         --
>                         Developer | sunlightfoundation.com
>         <http://sunlightfoundation.com>
>         <http://sunlightfoundation.com__>
>         <http://sunlightfoundation.__com__
>         <http://sunlightfoundation.com__>>
>         <http://sunlightfoundation.____com__
>
>         <http://sunlightfoundation.__com__
>         <http://sunlightfoundation.com__>>>
>
>
>
>
>
>
>
>
>                 --
>                 Developer | sunlightfoundation.com
>         <http://sunlightfoundation.com>
>         <http://sunlightfoundation.com__>
>         <http://sunlightfoundation.__com__
>         <http://sunlightfoundation.com__>>
>
>
>
>
>
>
>
>
>
>         --
>         Developer | sunlightfoundation.com
>         <http://sunlightfoundation.com> <http://sunlightfoundation.com__>
>
>
>
>
> --
> Developer | sunlightfoundation.com <http://sunlightfoundation.com>
>
>
>
>

-- - Josh Tauberer (@JoshData)
http://razor.occams.info

Just a heads-up that the new congressman from AZ isn't in the data yet
so there are a bunch of voting records with missing data. I'm hoping to
get to it by mid-day tomorrow...

--
- Josh Tauberer (@JoshData)

http://razor.occams.info

Eric- Thanks for reporting this bug 15 months ago. :-) It is now fixed.
(Although note that the committee code differs from the House's
committee code because it must have changed at some point on the Clerk's
website.)

The committee was also missing from GovTrack entirely because I had
marked the committee as having been discontinued (obsolete=1 in the
XML). That's now fixed.

- Josh Tauberer (@JoshData)

http://razor.occams.info


On 03/18/2011 12:07 PM, Eric Mill wrote:
> Small bug report: H.R. 1148 is referred in part to the House Ethics
> committee, and is missing the "HSSO" code to go along with it:
> http://www.govtrack.us/data/us/112/bills/h1148.xml
>
> -- Eric
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>

-- - Josh Tauberer (@JoshData)
http://razor.occams.info

- Josh Tauberer (@JoshData)
http://razor.occams.info

Dear data re-users,

This is advanced notice that on July 20 I'll be revising the terms of GovTrack's generic license agreement by adding the following paragraph:

* During the time in which your organization is reusing GovTrack's database, your website must block visitors referred by the websites of sponsors of the Netroots Nation conference. If you make the data available in bulk to others, your license agreement must carry over the same terms.

In techPresident today, Netroots Nation's executive director Raven Brooks encouraged progressive advocacy groups to boycott NationBuilder, a non-partisan technology platform that helps campaigns build their websites, because NationBuilder sells services to right-leaning organizations. There's a complicated history here that's touched on in the techPresident article (http://techpresident.com/news/22556/nationbuilders-mammoth-deal-state-level-republican-committee-sparks-calls-boycott), but Brooks's point seems to boil down to a belief that there can be no nonpartisan political tools.

GovTrack, and most tools that reuse its database, is a nonpartisan tool that has played an important role in political activism over the last several years on both sides of the political aisle. It is astonishing to me that anyone would think that technology infrastructure should choose sides. Especially since it appears that the sponsors of past Netroots Nations conferences have been users of nonpartisan political technology platforms like GovTrack. If they are going to boycott tools like GovTrack, they certainly won't notice the change to GovTrack's license terms.

Your feedback on these changes is welcome, especially if the feedback is in the style of satire. But this isn't a joke. Ridiculous boycotts of technology startups require ridiculous responses.

-- - Josh Tauberer (@JoshData)
http://razor.occams.info

- Josh Tauberer (@JoshData)
http://razor.occams.info

You may not disparage services for being nonpartisan.
( http://www.govtrack.us/developers/license)

In response to your original license draft, I told friends in the progressive movement that partisan services tend to be more poorly-built than nonpartisan ones and, therefore, by acting so vehemently against NationBuilder, they would reduce the quality of tools available to them. This morning, I further observed that, on the flip side, this means that if the entire progressive movement refused to use NationBuilder, it would effectively be forced to be a partisan service, and that would probably make the product worse than it otherwise would be and thereby keep better tools out of the hands of their political opponents. Does that qualify as disparagement under the license?

Dear GovTrack,

I'm having some trouble with some of the API methods.

This seems to be a successful call:
http://www.govtrack.us/api/v1/bill/?congress=112&order_by=-current_status_date&o\
ffset=20

However this is not:
http://www.govtrack.us/api/v1/bill/?congress=112&order_by=-current_status_date&o\
ffset=20&bill_type=house_bill

So filtering results by bill_type does not seem to work as expected. The trouble
seems to be with any parameters which do not have a * or ** next to them in the
API documentation. Is a different format required for these parameters? Or are
they just not available yet through the API?

Many thanks for any clarification you can provide.

Hi,

The backend is probably missing the conversion from the string constant
to the proper numeric constant used in the database. I'll take a look.

- Josh Tauberer (@JoshData)

http://razor.occams.info

On 08/06/2012 12:27 PM, kmustatea wrote:
> Dear GovTrack,
>
> I'm having some trouble with some of the API methods.
>
> This seems to be a successful call:
>
http://www.govtrack.us/api/v1/bill/?congress=112&order_by=-current_status_date&o\
ffset=20
>
> However this is not:
>
http://www.govtrack.us/api/v1/bill/?congress=112&order_by=-current_status_date&o\
ffset=20&bill_type=house_bill
>
> So filtering results by bill_type does not seem to work as expected. The
trouble seems to be with any parameters which do not have a * or ** next to them
in the API documentation. Is a different format required for these parameters?
Or are they just not available yet through the API?
>
> Many thanks for any clarification you can provide.
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>

I'm looking for any existing RDF ontologies or vocabularies for US
government stuff, in particular for bills, political persons (e.g. those
with bioguide IDs), and government agencies and bureaus. Does anyone
know of any or where I might look for them? (I'm hoping not to reinvent
the wheel.)

For example, Josh Tauberer many years ago created several rdf ontologies
for some work he did on rdf-izing his scraped govtrack data. I am aware
of three:

usgov http://www.rdfabout.com/rdf/schema/usgovt/
pol http://www.rdfabout.com/rdf/schema/politico/
bill http://www.rdfabout.com/rdf/schema/usbill/

There is still rdf data available that uses these at
http://www.govtrack.us/data/rdf/, but the ontology descriptions are
gone. (Each of the urls above redirects to a sparql endpoint
http://www.rdfabout.com/sparql which appears to be dead.)

- Josh Tauberer (@JoshData)
http://razor.occams.info

I'm looking for any existing RDF ontologies or vocabularies for US
government stuff, in particular for bills, political persons (e.g. those
with bioguide IDs), and government agencies and bureaus. Does anyone
know of any or where I might look for them? (I'm hoping not to reinvent
the wheel.)
For example, Josh Tauberer many years ago created several rdf ontologies
for some work he did on rdf-izing his scraped govtrack data. I am aware
of three:
usgov http://www.rdfabout.com/rdf/schema/usgovt/
pol http://www.rdfabout.com/rdf/schema/politico/
bill http://www.rdfabout.com/rdf/schema/usbill/
There is still rdf data available that uses these at
http://www.govtrack.us/data/rdf/, but the ontology descriptions are
gone. (Each of the urls above redirects to a sparql endpoint
http://www.rdfabout.com/sparql which appears to be dead.)
------------------------------------
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/govtrack/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/govtrack/join
(Yahoo! ID required)
<*> To change settings via email:
govtrack-digest@yahoogroups.com govtrack-fullfeatured@yahoogroups.com
<*> To unsubscribe from this group, send an email to:
govtrack-unsubscribe@yahoogroups.com
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/