Loading ...
Sorry, an error occurred while loading the content.

Re: [govtrack] Re: Data sources and people.xml/people/person/role@district

Expand Messages
  • Derek Willis
    Peter, We have ICPSRIDs for more than 2,600 members - if you think that would be a valuable addition to the API, I m happy to add that to the members response.
    Message 1 of 8 , May 31 10:51 AM
    • 0 Attachment
      Peter,

      We have ICPSRIDs for more than 2,600 members - if you think that would be a valuable addition to the API, I'm happy to add that to the members response. The committee names are tough because they can change from congress to congress. So what was the House Banking Committee is now the House Financial Services Committee. Depending on which party is in the majority, the House Education and Labor Committee becomes the House Education and the Workforce Committee, and so on. For my money, the official source is Congress itself, which means looking up committee assignments in the Record, among other methods. I haven't found phantom committees in Stewart's data, but rather phantom assignments in which a member is recorded as a member of a committee when official records do not reflect such an assignment.

      Derek



      On Tue, May 31, 2011 at 1:30 PM, codekiln <ptr.nore@...> wrote:
       



      Darek and Josh,
      Thanks for the citations and the info. Using people.xml I was able to extract ICPSRID for 165 out of the 793 identities I am researching; I've put the results below for google [2].

      It seems as though the four or five data sources I have located are not always consistent on the committee names. Darek, what official sources did you use to double check Stewart's data? I had actually wondered whether there were phantom committees in Stewart's data because the committee name capitalization is inconsistent and the assignments lack subcommittees.

      Thanks for tipping me off to your work at NYT Congress API; I noticed the "data for earlier Congresses will be available soon" note in the committee call documentation [1].

      What a lively digital community there is around politician data; it seems like every few months a new API appears online.

      Thanks,
      Peter

      [1] http://developer.nytimes.com/docs/congress_api#h3-committees
      [2]
      OSID|CRPNAME|govtrack_icpsrid
      N00000010|"Burton, Dan"|15014
      N00000048|"Hefley, Joel"|15419
      N00000153|"Neal, Richard E"|15616
      N00000245|"Kerry, John"|14920
      N00000270|"Markey, Edward J"|14435
      N00000275|"Frank, Barney"|14824
      N00000308|"Kennedy, Edward M"|10808
      N00000444|"Gregg, Judd"|14826
      N00000480|"Snowe, Olympia J"|14661
      N00000534|"Jeffords, James M"|14240
      N00000561|"Johnson, Nancy L"|15028
      N00000581|"Dodd, Christopher J"|14213
      N00000616|"Lieberman, Joe"|15704
      N00000652|"Shays, Christopher"|15449
      N00000659|"Lautenberg, Frank R"|14914
      N00000716|"Payne, Donald M"|15619
      N00000781|"Pallone, Frank Jr"|15454
      N00000834|"Saxton, Jim"|15112
      N00000964|"Rangel, Charles B"|13035
      N00001003|"Engel, Eliot L"|15603
      N00001024|"Lowey, Nita M"|15612
      N00001082|"Towns, Edolphus"|15072
      N00001093|"Schumer, Charles E"|14858
      N00001143|"Ackerman, Gary"|15000
      N00001214|"McNulty, Michael R"|15614
      N00001261|"Walsh, James T"|15630
      N00001267|"Boehlert, Sherwood"|15007
      N00001311|"Slaughter, Louise M"|15444
      N00001329|"Houghton, Amo"|15423
      N00001408|"Murtha, John P"|14072
      N00001509|"Kanjorski, Paul E"|15104
      N00001535|"Weldon, Curt"|15447
      N00001604|"Specter, Arlen"|14910
      N00001669|"Biden, Joseph R Jr"|14101
      N00001685|"Rockefeller, Jay"|14922
      N00001691|"Levin, Carl"|14709
      N00001701|"Mineta, Norman Y."|14257
      N00001758|"Grassley, Chuck"|14226
      N00001762|"Inouye, Daniel K"|4812
      N00001764|"Lugar, Richard G"|14506
      N00001783|"Dingell, John D"|2605
      N00001806|"Oberstar, James L"|14265
      N00001811|"Smith, Lamar"|15445
      N00001813|"Serrano, Jose E"|29134
      N00001817|"Young, C W Bill"|13047
      N00001821|"Hoyer, Steny H"|14873
      N00001861|"Waxman, Henry A"|14280
      N00001945|"Mikulski, Barbara A"|14440
      N00001955|"Cardin, Ben"|15408
      N00001979|"Sarbanes, Paul S"|13039
      N00002061|"Warner, John W"|14712
      N00002073|"Wolf, Frank R"|14869
      N00002091|"Craig, Larry"|14809
      N00002171|"Boucher, Rick"|15010
      N00002198|"Rahall, Nick"|14448
      N00002200|"Byrd, Robert C"|1366
      N00002214|"Mollohan, Alan B"|15083
      N00002247|"Coble, Howard"|15092
      N00002260|"Price, David"|15438
      N00002377|"Ballenger, Cass"|15402
      N00002423|"Hollings, Fritz"|11204
      N00002492|"Spratt, John M Jr"|15064
      N00002577|"Lewis, John"|15431
      N00002742|"Graham, Bob"|15503
      N00002782|"Stearns, Cliff"|15627
      N00002858|"Ros-Lehtinen, Ileana"|15634
      N00002877|"Shaw, E Clay Jr"|14860
      N00002982|"Bilirakis, Michael"|15006
      N00003126|"Gordon, Bart"|15100
      N00003132|"Cooper, Jim"|15019
      N00003209|"Duncan, John J Jr"|15455
      N00003254|"Tanner, John"|15628
      N00003328|"Cochran, Thad"|14009
      N00003329|"Lott, Trent"|14031
      N00003350|"Taylor, Gene"|15637
      N00003389|"McConnell, Mitch"|14921
      N00003437|"Bunning, Jim"|15406
      N00003473|"Rogers, Hal"|14854
      N00003522|"Kaptur, Marcy"|15029
      N00003651|"Regula, Ralph"|14045
      N00003660|"Gillmor, Paul E"|15604
      N00003709|"DeWine, Mike"|15020
      N00003736|"Oxley, Michael G"|14875
      N00003813|"Visclosky, Pete"|15124
      N00003950|"Levin, Sander"|15033
      N00004029|"Conyers, John Jr"|10713
      N00004070|"Kildee, Dale E"|14430
      N00004133|"Upton, Fred"|15446
      N00004207|"Harkin, Tom"|14230
      N00004280|"Leach, Jim"|14432
      N00004291|"Sensenbrenner, F James Jr"|14657
      N00004309|"Kohl, Herb"|15703
      N00004330|"Kleczka, Jerry"|15082
      N00004394|"Obey, David R"|12036
      N00004426|"Petri, Tom"|14675
      N00004489|"Sabo, Martin Olav"|14656
      N00004583|"Daschle, Tom"|14617
      N00004613|"Conrad, Kent"|15502
      N00004615|"Dorgan, Byron L"|14812
      N00004638|"Burns, Conrad"|15701
      N00004643|"Baucus, Max"|14203
      N00004698|"Crane, Phil"|12041
      N00004702|"Hyde, Henry J"|14239
      N00004781|"Hastert, Dennis"|15417
      N00004856|"Lipinski, Bill"|15036
      N00004912|"Evans, Lane"|15023
      N00004956|"Costello, Jerry F"|15453
      N00004981|"Durbin, Dick"|15021
      N00005037|"Gephardt, Richard A"|14421
      N00005105|"Skelton, Ike"|14451
      N00005178|"Bond, Christopher S 'Kit'"|15501
      N00005285|"Roberts, Pat"|14852
      N00005331|"Bereuter, Doug"|14605
      N00005372|"Tauzin, Billy"|14679
      N00005385|"Breaux, John"|13056
      N00005407|"Baker, Richard"|15401
      N00005414|"McCrery, Jim"|15451
      N00005582|"Inhofe, James M"|15424
      N00005617|"Nickles, Don"|14908
      N00005645|"Hall, Ralph M"|14828
      N00005656|"Barton, Joe"|15085
      N00005677|"Frost, Martin"|14626
      N00005892|"DeLay, Tom"|15094
      N00005906|"Paul, Ron"|14290
      N00005998|"Ortiz, Solomon P"|15049
      N00006060|"Stenholm, Charles W"|14664
      N00006202|"Campbell, Ben Nighthorse"|15407
      N00006237|"Cheney, Dick"|14611
      N00006246|"Thomas, Craig"|15633
      N00006406|"Kyl, Jon"|15429
      N00006424|"McCain, John"|15039
      N00006486|"Kolbe, Jim"|15105
      N00006515|"Domenici, Pete V"|14103
      N00006518|"Bingaman, Jeff"|14912
      N00006692|"Boxer, Barbara"|15011
      N00006932|"Dreier, David"|14813
      N00006983|"Hunter, Duncan"|14835
      N00007087|"Lewis, Jerry"|14644
      N00007124|"Cox, Christopher"|15601
      N00007151|"Rohrabacher, Dana"|15621
      N00007231|"Gallegly, Elton"|15413
      N00007360|"Pelosi, Nancy"|15448
      N00007382|"Lantos, Tom"|14837
      N00007390|"Miller, George"|14256
      N00007397|"Stark, Pete"|14053
      N00007584|"Herger, Wally"|15420
      N00007653|"Akaka, Daniel K"|14400
      N00007665|"Abercrombie, Neil"|15245
      N00007724|"Wyden, Ron"|14871
      N00007781|"DeFazio, Peter"|15410
      N00007918|"Dicks, Norm"|14413
      N00007997|"Stevens, Ted"|12109
      N00007999|"Young, Don"|14066
      N00008094|"Berman, Howard L"|15005
      N00009816|"Smith, Chris"|14863
      N00009829|"McDermott, Jim"|15613
      N00009869|"Hatch, Orrin G"|14503
      N00009918|"Leahy, Patrick"|14307
      N00009920|"Shelby, Richard C"|14659
      N00009922|"Reid, Harry"|15054
      N00009926|"Nelson, Bill"|14651
      N00010084|"Johnson, Tim"|15425
      N00011971|"Lungren, Dan"|14647
      N00012508|"Carper, Tom"|15015
      N99999981|"Rumsfeld, Donald H."|10622



      --- In govtrack@yahoogroups.com, Derek Willis <dwillis@...> wrote:
      >
      > In constructing committee membership data for the NYT Congress API, I used
      > data found via Charles Stewart's site:
      >
      > http://web.mit.edu/17.251/www/data_page.html#2
      >
      > As Josh says, older data contains errors in it - I recall incorrect party
      > affiliations and phantom committee assignments when I checked them against
      > official sources. So much so that our API only vouches for 110-112th
      > committee data...
      >
      > Derek
      >
      >
      >
      > On Tue, May 31, 2011 at 12:18 PM, Josh Tauberer <tauberer@...>wrote:

      >
      > >
      > >
      > > Hi, Peter.
      > >
      > > > 1.) What is the source for committees.xml? Is there a page like
      > > people.xml's "Source Data" page for committees.xml on Govtrack?
      > >
      > > The committee data is automatically scraped from
      > >
      > >
      > > http://www.senate.gov/pagelayout/committees/b_three_sections_with_teasers/membership.htm
      > >
      > > and
      > >
      > > http://clerk.house.gov/committee_info/index.html
      > >
      > > (As I mentioned on the labs list, the file was last generated a few
      > > months ago- March 3. I can re-generate it to get the latest info.)
      > >
      > > > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the
      > > same data available for other years?
      > >
      > > No I wasn't collecting that info then, and archival data is not
      > > available from the House/Senate (at least in the form I scrape).
      > >
      > > > 3.) The "Source Data" page says that people.xml "has been put
      > > together from a variety of sources and is maintained by hand." What are
      > > the sources?
      > >
      > > A full list of data sources is at govtrack.us/credits.xpd. That said,
      > > the amount of data I used from each source and the quality of the
      > > sources varied a lot. For information from about 2003 and on, the info
      > > about Members of Congress has been entered by hand by me.
      > >
      > > As you go further back in time, the quality of party affiliations,
      > > district assignments, and links to other IDs (e.g. ICPSR) grows worse.
      > > But it's probably the best anyone has anyway.
      > >
      > > Good luck. If you manage to put together a database of additional
      > > information, I hope you'll share it.
      > >
      > > - Josh Tauberer
      > > - CivicImpulse
      > >
      > > http://razor.occams.info
      > > http://www.civicimpulse.com
      > >
      > > "Yields falsehood when preceded by its quotation! Yields
      > > falsehood when preceded by its quotation!" Achilles to
      > > Tortoise (in "Godel, Escher, Bach" by Douglas Hofstadter)
      > >
      > > On 05/31/2011 11:29 AM, codekiln wrote:
      > > > I found out about the govtrack database from my post [1] over at the
      > > google group for Sunlight Labs' API. As I mentioned there, I have a list of
      > > CRP ID / bioguide id / name triples. I would like to
      > > > assemble information on what committees each legislator was a member
      > > > of, and any special titles they held in that committee, such as
      > > > chairman or ranking member. Mr. Tauberer suggested I use govtrack's
      > > people.xml [2] and committees.xml [3].
      > > >
      > > > I am helping a professor assemble this data, so it is important for me to
      > > be able to explain the paper trail of the data. I have seen Govtrack's
      > > "Source Data" page [4], but I still have some questions.
      > > >
      > > > 1.) What is the source for committees.xml? Is there a page like
      > > people.xml's "Source Data" page for committees.xml on Govtrack?
      > > >
      > > > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the same
      > > data available for other years?
      > > >
      > > > 3.) The "Source Data" page says that people.xml "has been put together
      > > from a variety of sources and is maintained by hand." What are the sources?
      > > >
      > > > Thanks,
      > > > Peter
      > > >
      > > > [1]
      > > http://groups.google.com/group/sunlightlabs/browse_thread/thread/7f35405ebf184e0e
      > > >
      > > > [2] http://www.govtrack.us/data/us/people.xml
      > > >
      > > > [3] http://www.govtrack.us/data/us/112/committees.xml
      > > >
      > > > [4] http://www.govtrack.us/developers/data.xpd
      > > >
      > > >
      > > >
      > > > ------------------------------------
      > > >
      > > > Yahoo! Groups Links
      > > >
      > > >
      > > >
      > >
      > >
      >


    • codekiln
      Hi Darek, Thanks for your detailed response. Your record of ICPSRIDs would be a valuable to the project I am working on, but I have no idea if ICPSRIDs belong
      Message 2 of 8 , Jun 2, 2011
      • 0 Attachment
        Hi Darek,
        Thanks for your detailed response.

        Your record of ICPSRIDs would be a valuable to the project I am working on, but I have no idea if ICPSRIDs belong in the NYT Congress API. Netizens definitely have a need some kind of database id Rosetta Stone. Bioguide ID - ICPSRID isn't the only important connection. When I have assembled all the different corresponding ids I will ask the professor if I may publish the cross-references I have found.

        Thanks,
        Peter

        --- In govtrack@yahoogroups.com, Derek Willis <dwillis@...> wrote:
        >
        > Peter,
        >
        > We have ICPSRIDs for more than 2,600 members - if you think that would be a
        > valuable addition to the API, I'm happy to add that to the members response.
        > The committee names are tough because they can change from congress to
        > congress. So what was the House Banking Committee is now the House Financial
        > Services Committee. Depending on which party is in the majority, the House
        > Education and Labor Committee becomes the House Education and the Workforce
        > Committee, and so on. For my money, the official source is Congress itself,
        > which means looking up committee assignments in the Record, among other
        > methods. I haven't found phantom committees in Stewart's data, but rather
        > phantom assignments in which a member is recorded as a member of a committee
        > when official records do not reflect such an assignment.
        >
        > Derek
        >
        >
        >
        > On Tue, May 31, 2011 at 1:30 PM, codekiln <ptr.nore@...> wrote:
        >
        > >
        > >
        > >
        > >
        > > Darek and Josh,
        > > Thanks for the citations and the info. Using people.xml I was able to
        > > extract ICPSRID for 165 out of the 793 identities I am researching; I've put
        > > the results below for google [2].
        > >
        > > It seems as though the four or five data sources I have located are not
        > > always consistent on the committee names. Darek, what official sources did
        > > you use to double check Stewart's data? I had actually wondered whether
        > > there were phantom committees in Stewart's data because the committee name
        > > capitalization is inconsistent and the assignments lack subcommittees.
        > >
        > > Thanks for tipping me off to your work at NYT Congress API; I noticed the
        > > "data for earlier Congresses will be available soon" note in the committee
        > > call documentation [1].
        > >
        > > What a lively digital community there is around politician data; it seems
        > > like every few months a new API appears online.
        > >
        > > Thanks,
        > > Peter
        > >
        > > [1] http://developer.nytimes.com/docs/congress_api#h3-committees
        > > [2]
        > > OSID|CRPNAME|govtrack_icpsrid
        > > N00000010|"Burton, Dan"|15014
        > > N00000048|"Hefley, Joel"|15419
        > > N00000153|"Neal, Richard E"|15616
        > > N00000245|"Kerry, John"|14920
        > > N00000270|"Markey, Edward J"|14435
        > > N00000275|"Frank, Barney"|14824
        > > N00000308|"Kennedy, Edward M"|10808
        > > N00000444|"Gregg, Judd"|14826
        > > N00000480|"Snowe, Olympia J"|14661
        > > N00000534|"Jeffords, James M"|14240
        > > N00000561|"Johnson, Nancy L"|15028
        > > N00000581|"Dodd, Christopher J"|14213
        > > N00000616|"Lieberman, Joe"|15704
        > > N00000652|"Shays, Christopher"|15449
        > > N00000659|"Lautenberg, Frank R"|14914
        > > N00000716|"Payne, Donald M"|15619
        > > N00000781|"Pallone, Frank Jr"|15454
        > > N00000834|"Saxton, Jim"|15112
        > > N00000964|"Rangel, Charles B"|13035
        > > N00001003|"Engel, Eliot L"|15603
        > > N00001024|"Lowey, Nita M"|15612
        > > N00001082|"Towns, Edolphus"|15072
        > > N00001093|"Schumer, Charles E"|14858
        > > N00001143|"Ackerman, Gary"|15000
        > > N00001214|"McNulty, Michael R"|15614
        > > N00001261|"Walsh, James T"|15630
        > > N00001267|"Boehlert, Sherwood"|15007
        > > N00001311|"Slaughter, Louise M"|15444
        > > N00001329|"Houghton, Amo"|15423
        > > N00001408|"Murtha, John P"|14072
        > > N00001509|"Kanjorski, Paul E"|15104
        > > N00001535|"Weldon, Curt"|15447
        > > N00001604|"Specter, Arlen"|14910
        > > N00001669|"Biden, Joseph R Jr"|14101
        > > N00001685|"Rockefeller, Jay"|14922
        > > N00001691|"Levin, Carl"|14709
        > > N00001701|"Mineta, Norman Y."|14257
        > > N00001758|"Grassley, Chuck"|14226
        > > N00001762|"Inouye, Daniel K"|4812
        > > N00001764|"Lugar, Richard G"|14506
        > > N00001783|"Dingell, John D"|2605
        > > N00001806|"Oberstar, James L"|14265
        > > N00001811|"Smith, Lamar"|15445
        > > N00001813|"Serrano, Jose E"|29134
        > > N00001817|"Young, C W Bill"|13047
        > > N00001821|"Hoyer, Steny H"|14873
        > > N00001861|"Waxman, Henry A"|14280
        > > N00001945|"Mikulski, Barbara A"|14440
        > > N00001955|"Cardin, Ben"|15408
        > > N00001979|"Sarbanes, Paul S"|13039
        > > N00002061|"Warner, John W"|14712
        > > N00002073|"Wolf, Frank R"|14869
        > > N00002091|"Craig, Larry"|14809
        > > N00002171|"Boucher, Rick"|15010
        > > N00002198|"Rahall, Nick"|14448
        > > N00002200|"Byrd, Robert C"|1366
        > > N00002214|"Mollohan, Alan B"|15083
        > > N00002247|"Coble, Howard"|15092
        > > N00002260|"Price, David"|15438
        > > N00002377|"Ballenger, Cass"|15402
        > > N00002423|"Hollings, Fritz"|11204
        > > N00002492|"Spratt, John M Jr"|15064
        > > N00002577|"Lewis, John"|15431
        > > N00002742|"Graham, Bob"|15503
        > > N00002782|"Stearns, Cliff"|15627
        > > N00002858|"Ros-Lehtinen, Ileana"|15634
        > > N00002877|"Shaw, E Clay Jr"|14860
        > > N00002982|"Bilirakis, Michael"|15006
        > > N00003126|"Gordon, Bart"|15100
        > > N00003132|"Cooper, Jim"|15019
        > > N00003209|"Duncan, John J Jr"|15455
        > > N00003254|"Tanner, John"|15628
        > > N00003328|"Cochran, Thad"|14009
        > > N00003329|"Lott, Trent"|14031
        > > N00003350|"Taylor, Gene"|15637
        > > N00003389|"McConnell, Mitch"|14921
        > > N00003437|"Bunning, Jim"|15406
        > > N00003473|"Rogers, Hal"|14854
        > > N00003522|"Kaptur, Marcy"|15029
        > > N00003651|"Regula, Ralph"|14045
        > > N00003660|"Gillmor, Paul E"|15604
        > > N00003709|"DeWine, Mike"|15020
        > > N00003736|"Oxley, Michael G"|14875
        > > N00003813|"Visclosky, Pete"|15124
        > > N00003950|"Levin, Sander"|15033
        > > N00004029|"Conyers, John Jr"|10713
        > > N00004070|"Kildee, Dale E"|14430
        > > N00004133|"Upton, Fred"|15446
        > > N00004207|"Harkin, Tom"|14230
        > > N00004280|"Leach, Jim"|14432
        > > N00004291|"Sensenbrenner, F James Jr"|14657
        > > N00004309|"Kohl, Herb"|15703
        > > N00004330|"Kleczka, Jerry"|15082
        > > N00004394|"Obey, David R"|12036
        > > N00004426|"Petri, Tom"|14675
        > > N00004489|"Sabo, Martin Olav"|14656
        > > N00004583|"Daschle, Tom"|14617
        > > N00004613|"Conrad, Kent"|15502
        > > N00004615|"Dorgan, Byron L"|14812
        > > N00004638|"Burns, Conrad"|15701
        > > N00004643|"Baucus, Max"|14203
        > > N00004698|"Crane, Phil"|12041
        > > N00004702|"Hyde, Henry J"|14239
        > > N00004781|"Hastert, Dennis"|15417
        > > N00004856|"Lipinski, Bill"|15036
        > > N00004912|"Evans, Lane"|15023
        > > N00004956|"Costello, Jerry F"|15453
        > > N00004981|"Durbin, Dick"|15021
        > > N00005037|"Gephardt, Richard A"|14421
        > > N00005105|"Skelton, Ike"|14451
        > > N00005178|"Bond, Christopher S 'Kit'"|15501
        > > N00005285|"Roberts, Pat"|14852
        > > N00005331|"Bereuter, Doug"|14605
        > > N00005372|"Tauzin, Billy"|14679
        > > N00005385|"Breaux, John"|13056
        > > N00005407|"Baker, Richard"|15401
        > > N00005414|"McCrery, Jim"|15451
        > > N00005582|"Inhofe, James M"|15424
        > > N00005617|"Nickles, Don"|14908
        > > N00005645|"Hall, Ralph M"|14828
        > > N00005656|"Barton, Joe"|15085
        > > N00005677|"Frost, Martin"|14626
        > > N00005892|"DeLay, Tom"|15094
        > > N00005906|"Paul, Ron"|14290
        > > N00005998|"Ortiz, Solomon P"|15049
        > > N00006060|"Stenholm, Charles W"|14664
        > > N00006202|"Campbell, Ben Nighthorse"|15407
        > > N00006237|"Cheney, Dick"|14611
        > > N00006246|"Thomas, Craig"|15633
        > > N00006406|"Kyl, Jon"|15429
        > > N00006424|"McCain, John"|15039
        > > N00006486|"Kolbe, Jim"|15105
        > > N00006515|"Domenici, Pete V"|14103
        > > N00006518|"Bingaman, Jeff"|14912
        > > N00006692|"Boxer, Barbara"|15011
        > > N00006932|"Dreier, David"|14813
        > > N00006983|"Hunter, Duncan"|14835
        > > N00007087|"Lewis, Jerry"|14644
        > > N00007124|"Cox, Christopher"|15601
        > > N00007151|"Rohrabacher, Dana"|15621
        > > N00007231|"Gallegly, Elton"|15413
        > > N00007360|"Pelosi, Nancy"|15448
        > > N00007382|"Lantos, Tom"|14837
        > > N00007390|"Miller, George"|14256
        > > N00007397|"Stark, Pete"|14053
        > > N00007584|"Herger, Wally"|15420
        > > N00007653|"Akaka, Daniel K"|14400
        > > N00007665|"Abercrombie, Neil"|15245
        > > N00007724|"Wyden, Ron"|14871
        > > N00007781|"DeFazio, Peter"|15410
        > > N00007918|"Dicks, Norm"|14413
        > > N00007997|"Stevens, Ted"|12109
        > > N00007999|"Young, Don"|14066
        > > N00008094|"Berman, Howard L"|15005
        > > N00009816|"Smith, Chris"|14863
        > > N00009829|"McDermott, Jim"|15613
        > > N00009869|"Hatch, Orrin G"|14503
        > > N00009918|"Leahy, Patrick"|14307
        > > N00009920|"Shelby, Richard C"|14659
        > > N00009922|"Reid, Harry"|15054
        > > N00009926|"Nelson, Bill"|14651
        > > N00010084|"Johnson, Tim"|15425
        > > N00011971|"Lungren, Dan"|14647
        > > N00012508|"Carper, Tom"|15015
        > > N99999981|"Rumsfeld, Donald H."|10622
        > >
        > >
        > > --- In govtrack@yahoogroups.com, Derek Willis <dwillis@> wrote:
        > > >
        > > > In constructing committee membership data for the NYT Congress API, I
        > > used
        > > > data found via Charles Stewart's site:
        > > >
        > > > http://web.mit.edu/17.251/www/data_page.html#2
        > > >
        > > > As Josh says, older data contains errors in it - I recall incorrect party
        > > > affiliations and phantom committee assignments when I checked them
        > > against
        > > > official sources. So much so that our API only vouches for 110-112th
        > > > committee data...
        > > >
        > > > Derek
        > > >
        > > >
        > > >
        > > > On Tue, May 31, 2011 at 12:18 PM, Josh Tauberer <tauberer@>wrote:
        > >
        > > >
        > > > >
        > > > >
        > > > > Hi, Peter.
        > > > >
        > > > > > 1.) What is the source for committees.xml? Is there a page like
        > > > > people.xml's "Source Data" page for committees.xml on Govtrack?
        > > > >
        > > > > The committee data is automatically scraped from
        > > > >
        > > > >
        > > > >
        > > http://www.senate.gov/pagelayout/committees/b_three_sections_with_teasers/membership.htm
        > > > >
        > > > > and
        > > > >
        > > > > http://clerk.house.gov/committee_info/index.html
        > > > >
        > > > > (As I mentioned on the labs list, the file was last generated a few
        > > > > months ago- March 3. I can re-generate it to get the latest info.)
        > > > >
        > > > > > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the
        > > > > same data available for other years?
        > > > >
        > > > > No I wasn't collecting that info then, and archival data is not
        > > > > available from the House/Senate (at least in the form I scrape).
        > > > >
        > > > > > 3.) The "Source Data" page says that people.xml "has been put
        > > > > together from a variety of sources and is maintained by hand." What are
        > > > > the sources?
        > > > >
        > > > > A full list of data sources is at govtrack.us/credits.xpd. That said,
        > > > > the amount of data I used from each source and the quality of the
        > > > > sources varied a lot. For information from about 2003 and on, the info
        > > > > about Members of Congress has been entered by hand by me.
        > > > >
        > > > > As you go further back in time, the quality of party affiliations,
        > > > > district assignments, and links to other IDs (e.g. ICPSR) grows worse.
        > > > > But it's probably the best anyone has anyway.
        > > > >
        > > > > Good luck. If you manage to put together a database of additional
        > > > > information, I hope you'll share it.
        > > > >
        > > > > - Josh Tauberer
        > > > > - CivicImpulse
        > > > >
        > > > > http://razor.occams.info
        > > > > http://www.civicimpulse.com
        > > > >
        > > > > "Yields falsehood when preceded by its quotation! Yields
        > > > > falsehood when preceded by its quotation!" Achilles to
        > > > > Tortoise (in "Godel, Escher, Bach" by Douglas Hofstadter)
        > > > >
        > > > > On 05/31/2011 11:29 AM, codekiln wrote:
        > > > > > I found out about the govtrack database from my post [1] over at the
        > > > > google group for Sunlight Labs' API. As I mentioned there, I have a
        > > list of
        > > > > CRP ID / bioguide id / name triples. I would like to
        > > > > > assemble information on what committees each legislator was a member
        > > > > > of, and any special titles they held in that committee, such as
        > > > > > chairman or ranking member. Mr. Tauberer suggested I use govtrack's
        > > > > people.xml [2] and committees.xml [3].
        > > > > >
        > > > > > I am helping a professor assemble this data, so it is important for
        > > me to
        > > > > be able to explain the paper trail of the data. I have seen Govtrack's
        > > > > "Source Data" page [4], but I still have some questions.
        > > > > >
        > > > > > 1.) What is the source for committees.xml? Is there a page like
        > > > > people.xml's "Source Data" page for committees.xml on Govtrack?
        > > > > >
        > > > > > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the
        > > same
        > > > > data available for other years?
        > > > > >
        > > > > > 3.) The "Source Data" page says that people.xml "has been put
        > > together
        > > > > from a variety of sources and is maintained by hand." What are the
        > > sources?
        > > > > >
        > > > > > Thanks,
        > > > > > Peter
        > > > > >
        > > > > > [1]
        > > > >
        > > http://groups.google.com/group/sunlightlabs/browse_thread/thread/7f35405ebf184e0e
        > > > > >
        > > > > > [2] http://www.govtrack.us/data/us/people.xml
        > > > > >
        > > > > > [3] http://www.govtrack.us/data/us/112/committees.xml
        > > > > >
        > > > > > [4] http://www.govtrack.us/developers/data.xpd
        > > > > >
        > > > > >
        > > > > >
        > > > > > ------------------------------------
        > > > > >
        > > > > > Yahoo! Groups Links
        > > > > >
        > > > > >
        > > > > >
        > > > >
        > > > >
        > > >
        > >
        > >
        > >
        >
      • Derek Willis
        Hey Peter, I think we ll put them in anyway, but I think the bioguide ID should be the canonical reference. Derek
        Message 3 of 8 , Jun 2, 2011
        • 0 Attachment
          Hey Peter,

          I think we'll put them in anyway, but I think the bioguide ID should be the canonical reference.

          Derek

          On Thu, Jun 2, 2011 at 11:49 AM, codekiln <ptr.nore@...> wrote:
           



          Hi Darek,
          Thanks for your detailed response.

          Your record of ICPSRIDs would be a valuable to the project I am working on, but I have no idea if ICPSRIDs belong in the NYT Congress API. Netizens definitely have a need some kind of database id Rosetta Stone. Bioguide ID - ICPSRID isn't the only important connection. When I have assembled all the different corresponding ids I will ask the professor if I may publish the cross-references I have found.

          Thanks,
          Peter



          --- In govtrack@yahoogroups.com, Derek Willis <dwillis@...> wrote:
          >
          > Peter,
          >
          > We have ICPSRIDs for more than 2,600 members - if you think that would be a
          > valuable addition to the API, I'm happy to add that to the members response.
          > The committee names are tough because they can change from congress to
          > congress. So what was the House Banking Committee is now the House Financial
          > Services Committee. Depending on which party is in the majority, the House
          > Education and Labor Committee becomes the House Education and the Workforce
          > Committee, and so on. For my money, the official source is Congress itself,
          > which means looking up committee assignments in the Record, among other
          > methods. I haven't found phantom committees in Stewart's data, but rather
          > phantom assignments in which a member is recorded as a member of a committee
          > when official records do not reflect such an assignment.
          >
          > Derek
          >
          >
          >
          > On Tue, May 31, 2011 at 1:30 PM, codekiln <ptr.nore@...> wrote:
          >
          > >
          > >
          > >
          > >
          > > Darek and Josh,
          > > Thanks for the citations and the info. Using people.xml I was able to
          > > extract ICPSRID for 165 out of the 793 identities I am researching; I've put
          > > the results below for google [2].
          > >
          > > It seems as though the four or five data sources I have located are not
          > > always consistent on the committee names. Darek, what official sources did
          > > you use to double check Stewart's data? I had actually wondered whether
          > > there were phantom committees in Stewart's data because the committee name
          > > capitalization is inconsistent and the assignments lack subcommittees.
          > >
          > > Thanks for tipping me off to your work at NYT Congress API; I noticed the
          > > "data for earlier Congresses will be available soon" note in the committee
          > > call documentation [1].
          > >
          > > What a lively digital community there is around politician data; it seems
          > > like every few months a new API appears online.
          > >
          > > Thanks,
          > > Peter
          > >
          > > [1] http://developer.nytimes.com/docs/congress_api#h3-committees
          > > [2]
          > > OSID|CRPNAME|govtrack_icpsrid
          > > N00000010|"Burton, Dan"|15014
          > > N00000048|"Hefley, Joel"|15419
          > > N00000153|"Neal, Richard E"|15616
          > > N00000245|"Kerry, John"|14920
          > > N00000270|"Markey, Edward J"|14435
          > > N00000275|"Frank, Barney"|14824
          > > N00000308|"Kennedy, Edward M"|10808
          > > N00000444|"Gregg, Judd"|14826
          > > N00000480|"Snowe, Olympia J"|14661
          > > N00000534|"Jeffords, James M"|14240
          > > N00000561|"Johnson, Nancy L"|15028
          > > N00000581|"Dodd, Christopher J"|14213
          > > N00000616|"Lieberman, Joe"|15704
          > > N00000652|"Shays, Christopher"|15449
          > > N00000659|"Lautenberg, Frank R"|14914
          > > N00000716|"Payne, Donald M"|15619
          > > N00000781|"Pallone, Frank Jr"|15454
          > > N00000834|"Saxton, Jim"|15112
          > > N00000964|"Rangel, Charles B"|13035
          > > N00001003|"Engel, Eliot L"|15603
          > > N00001024|"Lowey, Nita M"|15612
          > > N00001082|"Towns, Edolphus"|15072
          > > N00001093|"Schumer, Charles E"|14858
          > > N00001143|"Ackerman, Gary"|15000
          > > N00001214|"McNulty, Michael R"|15614
          > > N00001261|"Walsh, James T"|15630
          > > N00001267|"Boehlert, Sherwood"|15007
          > > N00001311|"Slaughter, Louise M"|15444
          > > N00001329|"Houghton, Amo"|15423
          > > N00001408|"Murtha, John P"|14072
          > > N00001509|"Kanjorski, Paul E"|15104
          > > N00001535|"Weldon, Curt"|15447
          > > N00001604|"Specter, Arlen"|14910
          > > N00001669|"Biden, Joseph R Jr"|14101
          > > N00001685|"Rockefeller, Jay"|14922
          > > N00001691|"Levin, Carl"|14709
          > > N00001701|"Mineta, Norman Y."|14257
          > > N00001758|"Grassley, Chuck"|14226
          > > N00001762|"Inouye, Daniel K"|4812
          > > N00001764|"Lugar, Richard G"|14506
          > > N00001783|"Dingell, John D"|2605
          > > N00001806|"Oberstar, James L"|14265
          > > N00001811|"Smith, Lamar"|15445
          > > N00001813|"Serrano, Jose E"|29134
          > > N00001817|"Young, C W Bill"|13047
          > > N00001821|"Hoyer, Steny H"|14873
          > > N00001861|"Waxman, Henry A"|14280
          > > N00001945|"Mikulski, Barbara A"|14440
          > > N00001955|"Cardin, Ben"|15408
          > > N00001979|"Sarbanes, Paul S"|13039
          > > N00002061|"Warner, John W"|14712
          > > N00002073|"Wolf, Frank R"|14869
          > > N00002091|"Craig, Larry"|14809
          > > N00002171|"Boucher, Rick"|15010
          > > N00002198|"Rahall, Nick"|14448
          > > N00002200|"Byrd, Robert C"|1366
          > > N00002214|"Mollohan, Alan B"|15083
          > > N00002247|"Coble, Howard"|15092
          > > N00002260|"Price, David"|15438
          > > N00002377|"Ballenger, Cass"|15402
          > > N00002423|"Hollings, Fritz"|11204
          > > N00002492|"Spratt, John M Jr"|15064
          > > N00002577|"Lewis, John"|15431
          > > N00002742|"Graham, Bob"|15503
          > > N00002782|"Stearns, Cliff"|15627
          > > N00002858|"Ros-Lehtinen, Ileana"|15634
          > > N00002877|"Shaw, E Clay Jr"|14860
          > > N00002982|"Bilirakis, Michael"|15006
          > > N00003126|"Gordon, Bart"|15100
          > > N00003132|"Cooper, Jim"|15019
          > > N00003209|"Duncan, John J Jr"|15455
          > > N00003254|"Tanner, John"|15628
          > > N00003328|"Cochran, Thad"|14009
          > > N00003329|"Lott, Trent"|14031
          > > N00003350|"Taylor, Gene"|15637
          > > N00003389|"McConnell, Mitch"|14921
          > > N00003437|"Bunning, Jim"|15406
          > > N00003473|"Rogers, Hal"|14854
          > > N00003522|"Kaptur, Marcy"|15029
          > > N00003651|"Regula, Ralph"|14045
          > > N00003660|"Gillmor, Paul E"|15604
          > > N00003709|"DeWine, Mike"|15020
          > > N00003736|"Oxley, Michael G"|14875
          > > N00003813|"Visclosky, Pete"|15124
          > > N00003950|"Levin, Sander"|15033
          > > N00004029|"Conyers, John Jr"|10713
          > > N00004070|"Kildee, Dale E"|14430
          > > N00004133|"Upton, Fred"|15446
          > > N00004207|"Harkin, Tom"|14230
          > > N00004280|"Leach, Jim"|14432
          > > N00004291|"Sensenbrenner, F James Jr"|14657
          > > N00004309|"Kohl, Herb"|15703
          > > N00004330|"Kleczka, Jerry"|15082
          > > N00004394|"Obey, David R"|12036
          > > N00004426|"Petri, Tom"|14675
          > > N00004489|"Sabo, Martin Olav"|14656
          > > N00004583|"Daschle, Tom"|14617
          > > N00004613|"Conrad, Kent"|15502
          > > N00004615|"Dorgan, Byron L"|14812
          > > N00004638|"Burns, Conrad"|15701
          > > N00004643|"Baucus, Max"|14203
          > > N00004698|"Crane, Phil"|12041
          > > N00004702|"Hyde, Henry J"|14239
          > > N00004781|"Hastert, Dennis"|15417
          > > N00004856|"Lipinski, Bill"|15036
          > > N00004912|"Evans, Lane"|15023
          > > N00004956|"Costello, Jerry F"|15453
          > > N00004981|"Durbin, Dick"|15021
          > > N00005037|"Gephardt, Richard A"|14421
          > > N00005105|"Skelton, Ike"|14451
          > > N00005178|"Bond, Christopher S 'Kit'"|15501
          > > N00005285|"Roberts, Pat"|14852
          > > N00005331|"Bereuter, Doug"|14605
          > > N00005372|"Tauzin, Billy"|14679
          > > N00005385|"Breaux, John"|13056
          > > N00005407|"Baker, Richard"|15401
          > > N00005414|"McCrery, Jim"|15451
          > > N00005582|"Inhofe, James M"|15424
          > > N00005617|"Nickles, Don"|14908
          > > N00005645|"Hall, Ralph M"|14828
          > > N00005656|"Barton, Joe"|15085
          > > N00005677|"Frost, Martin"|14626
          > > N00005892|"DeLay, Tom"|15094
          > > N00005906|"Paul, Ron"|14290
          > > N00005998|"Ortiz, Solomon P"|15049
          > > N00006060|"Stenholm, Charles W"|14664
          > > N00006202|"Campbell, Ben Nighthorse"|15407
          > > N00006237|"Cheney, Dick"|14611
          > > N00006246|"Thomas, Craig"|15633
          > > N00006406|"Kyl, Jon"|15429
          > > N00006424|"McCain, John"|15039
          > > N00006486|"Kolbe, Jim"|15105
          > > N00006515|"Domenici, Pete V"|14103
          > > N00006518|"Bingaman, Jeff"|14912
          > > N00006692|"Boxer, Barbara"|15011
          > > N00006932|"Dreier, David"|14813
          > > N00006983|"Hunter, Duncan"|14835
          > > N00007087|"Lewis, Jerry"|14644
          > > N00007124|"Cox, Christopher"|15601
          > > N00007151|"Rohrabacher, Dana"|15621
          > > N00007231|"Gallegly, Elton"|15413
          > > N00007360|"Pelosi, Nancy"|15448
          > > N00007382|"Lantos, Tom"|14837
          > > N00007390|"Miller, George"|14256
          > > N00007397|"Stark, Pete"|14053
          > > N00007584|"Herger, Wally"|15420
          > > N00007653|"Akaka, Daniel K"|14400
          > > N00007665|"Abercrombie, Neil"|15245
          > > N00007724|"Wyden, Ron"|14871
          > > N00007781|"DeFazio, Peter"|15410
          > > N00007918|"Dicks, Norm"|14413
          > > N00007997|"Stevens, Ted"|12109
          > > N00007999|"Young, Don"|14066
          > > N00008094|"Berman, Howard L"|15005
          > > N00009816|"Smith, Chris"|14863
          > > N00009829|"McDermott, Jim"|15613
          > > N00009869|"Hatch, Orrin G"|14503
          > > N00009918|"Leahy, Patrick"|14307
          > > N00009920|"Shelby, Richard C"|14659
          > > N00009922|"Reid, Harry"|15054
          > > N00009926|"Nelson, Bill"|14651
          > > N00010084|"Johnson, Tim"|15425
          > > N00011971|"Lungren, Dan"|14647
          > > N00012508|"Carper, Tom"|15015
          > > N99999981|"Rumsfeld, Donald H."|10622
          > >
          > >
          > > --- In govtrack@yahoogroups.com, Derek Willis <dwillis@> wrote:
          > > >
          > > > In constructing committee membership data for the NYT Congress API, I
          > > used
          > > > data found via Charles Stewart's site:
          > > >
          > > > http://web.mit.edu/17.251/www/data_page.html#2
          > > >
          > > > As Josh says, older data contains errors in it - I recall incorrect party
          > > > affiliations and phantom committee assignments when I checked them
          > > against
          > > > official sources. So much so that our API only vouches for 110-112th
          > > > committee data...
          > > >
          > > > Derek
          > > >
          > > >
          > > >
          > > > On Tue, May 31, 2011 at 12:18 PM, Josh Tauberer <tauberer@>wrote:
          > >
          > > >
          > > > >
          > > > >
          > > > > Hi, Peter.
          > > > >
          > > > > > 1.) What is the source for committees.xml? Is there a page like
          > > > > people.xml's "Source Data" page for committees.xml on Govtrack?
          > > > >
          > > > > The committee data is automatically scraped from
          > > > >
          > > > >
          > > > >
          > > http://www.senate.gov/pagelayout/committees/b_three_sections_with_teasers/membership.htm
          > > > >
          > > > > and
          > > > >
          > > > > http://clerk.house.gov/committee_info/index.html
          > > > >
          > > > > (As I mentioned on the labs list, the file was last generated a few
          > > > > months ago- March 3. I can re-generate it to get the latest info.)
          > > > >
          > > > > > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the
          > > > > same data available for other years?
          > > > >
          > > > > No I wasn't collecting that info then, and archival data is not
          > > > > available from the House/Senate (at least in the form I scrape).
          > > > >
          > > > > > 3.) The "Source Data" page says that people.xml "has been put
          > > > > together from a variety of sources and is maintained by hand." What are
          > > > > the sources?
          > > > >
          > > > > A full list of data sources is at govtrack.us/credits.xpd. That said,
          > > > > the amount of data I used from each source and the quality of the
          > > > > sources varied a lot. For information from about 2003 and on, the info
          > > > > about Members of Congress has been entered by hand by me.
          > > > >
          > > > > As you go further back in time, the quality of party affiliations,
          > > > > district assignments, and links to other IDs (e.g. ICPSR) grows worse.
          > > > > But it's probably the best anyone has anyway.
          > > > >
          > > > > Good luck. If you manage to put together a database of additional
          > > > > information, I hope you'll share it.
          > > > >
          > > > > - Josh Tauberer
          > > > > - CivicImpulse
          > > > >
          > > > > http://razor.occams.info
          > > > > http://www.civicimpulse.com
          > > > >
          > > > > "Yields falsehood when preceded by its quotation! Yields
          > > > > falsehood when preceded by its quotation!" Achilles to
          > > > > Tortoise (in "Godel, Escher, Bach" by Douglas Hofstadter)
          > > > >
          > > > > On 05/31/2011 11:29 AM, codekiln wrote:
          > > > > > I found out about the govtrack database from my post [1] over at the
          > > > > google group for Sunlight Labs' API. As I mentioned there, I have a
          > > list of
          > > > > CRP ID / bioguide id / name triples. I would like to
          > > > > > assemble information on what committees each legislator was a member
          > > > > > of, and any special titles they held in that committee, such as
          > > > > > chairman or ranking member. Mr. Tauberer suggested I use govtrack's
          > > > > people.xml [2] and committees.xml [3].
          > > > > >
          > > > > > I am helping a professor assemble this data, so it is important for
          > > me to
          > > > > be able to explain the paper trail of the data. I have seen Govtrack's
          > > > > "Source Data" page [4], but I still have some questions.
          > > > > >
          > > > > > 1.) What is the source for committees.xml? Is there a page like
          > > > > people.xml's "Source Data" page for committees.xml on Govtrack?
          > > > > >
          > > > > > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the
          > > same
          > > > > data available for other years?
          > > > > >
          > > > > > 3.) The "Source Data" page says that people.xml "has been put
          > > together
          > > > > from a variety of sources and is maintained by hand." What are the
          > > sources?
          > > > > >
          > > > > > Thanks,
          > > > > > Peter
          > > > > >
          > > > > > [1]
          > > > >
          > > http://groups.google.com/group/sunlightlabs/browse_thread/thread/7f35405ebf184e0e
          > > > > >
          > > > > > [2] http://www.govtrack.us/data/us/people.xml
          > > > > >
          > > > > > [3] http://www.govtrack.us/data/us/112/committees.xml
          > > > > >
          > > > > > [4] http://www.govtrack.us/developers/data.xpd
          > > > > >
          > > > > >
          > > > > >
          > > > > > ------------------------------------
          > > > > >
          > > > > > Yahoo! Groups Links
          > > > > >
          > > > > >
          > > > > >
          > > > >
          > > > >
          > > >
          > >
          > >
          > >
          >


        • Derek Willis
          Just as an FYI, we ll probably make this official next week but the ICPSR IDs that we have are now in the Member response of the NYT Congress API.
          Message 4 of 8 , Jun 3, 2011
          • 0 Attachment
            Just as an FYI, we'll probably make this official next week but the ICPSR IDs that we have are now in the Member response of the NYT Congress API. 


            On Thu, Jun 2, 2011 at 11:49 AM, codekiln <ptr.nore@...> wrote:
             



            Hi Darek,
            Thanks for your detailed response.

            Your record of ICPSRIDs would be a valuable to the project I am working on, but I have no idea if ICPSRIDs belong in the NYT Congress API. Netizens definitely have a need some kind of database id Rosetta Stone. Bioguide ID - ICPSRID isn't the only important connection. When I have assembled all the different corresponding ids I will ask the professor if I may publish the cross-references I have found.

            Thanks,
            Peter



            --- In govtrack@yahoogroups.com, Derek Willis <dwillis@...> wrote:
            >
            > Peter,
            >
            > We have ICPSRIDs for more than 2,600 members - if you think that would be a
            > valuable addition to the API, I'm happy to add that to the members response.
            > The committee names are tough because they can change from congress to
            > congress. So what was the House Banking Committee is now the House Financial
            > Services Committee. Depending on which party is in the majority, the House
            > Education and Labor Committee becomes the House Education and the Workforce
            > Committee, and so on. For my money, the official source is Congress itself,
            > which means looking up committee assignments in the Record, among other
            > methods. I haven't found phantom committees in Stewart's data, but rather
            > phantom assignments in which a member is recorded as a member of a committee
            > when official records do not reflect such an assignment.
            >
            > Derek
            >
            >
            >
            > On Tue, May 31, 2011 at 1:30 PM, codekiln <ptr.nore@...> wrote:
            >
            > >
            > >
            > >
            > >
            > > Darek and Josh,
            > > Thanks for the citations and the info. Using people.xml I was able to
            > > extract ICPSRID for 165 out of the 793 identities I am researching; I've put
            > > the results below for google [2].
            > >
            > > It seems as though the four or five data sources I have located are not
            > > always consistent on the committee names. Darek, what official sources did
            > > you use to double check Stewart's data? I had actually wondered whether
            > > there were phantom committees in Stewart's data because the committee name
            > > capitalization is inconsistent and the assignments lack subcommittees.
            > >
            > > Thanks for tipping me off to your work at NYT Congress API; I noticed the
            > > "data for earlier Congresses will be available soon" note in the committee
            > > call documentation [1].
            > >
            > > What a lively digital community there is around politician data; it seems
            > > like every few months a new API appears online.
            > >
            > > Thanks,
            > > Peter
            > >
            > > [1] http://developer.nytimes.com/docs/congress_api#h3-committees
            > > [2]
            > > OSID|CRPNAME|govtrack_icpsrid
            > > N00000010|"Burton, Dan"|15014
            > > N00000048|"Hefley, Joel"|15419
            > > N00000153|"Neal, Richard E"|15616
            > > N00000245|"Kerry, John"|14920
            > > N00000270|"Markey, Edward J"|14435
            > > N00000275|"Frank, Barney"|14824
            > > N00000308|"Kennedy, Edward M"|10808
            > > N00000444|"Gregg, Judd"|14826
            > > N00000480|"Snowe, Olympia J"|14661
            > > N00000534|"Jeffords, James M"|14240
            > > N00000561|"Johnson, Nancy L"|15028
            > > N00000581|"Dodd, Christopher J"|14213
            > > N00000616|"Lieberman, Joe"|15704
            > > N00000652|"Shays, Christopher"|15449
            > > N00000659|"Lautenberg, Frank R"|14914
            > > N00000716|"Payne, Donald M"|15619
            > > N00000781|"Pallone, Frank Jr"|15454
            > > N00000834|"Saxton, Jim"|15112
            > > N00000964|"Rangel, Charles B"|13035
            > > N00001003|"Engel, Eliot L"|15603
            > > N00001024|"Lowey, Nita M"|15612
            > > N00001082|"Towns, Edolphus"|15072
            > > N00001093|"Schumer, Charles E"|14858
            > > N00001143|"Ackerman, Gary"|15000
            > > N00001214|"McNulty, Michael R"|15614
            > > N00001261|"Walsh, James T"|15630
            > > N00001267|"Boehlert, Sherwood"|15007
            > > N00001311|"Slaughter, Louise M"|15444
            > > N00001329|"Houghton, Amo"|15423
            > > N00001408|"Murtha, John P"|14072
            > > N00001509|"Kanjorski, Paul E"|15104
            > > N00001535|"Weldon, Curt"|15447
            > > N00001604|"Specter, Arlen"|14910
            > > N00001669|"Biden, Joseph R Jr"|14101
            > > N00001685|"Rockefeller, Jay"|14922
            > > N00001691|"Levin, Carl"|14709
            > > N00001701|"Mineta, Norman Y."|14257
            > > N00001758|"Grassley, Chuck"|14226
            > > N00001762|"Inouye, Daniel K"|4812
            > > N00001764|"Lugar, Richard G"|14506
            > > N00001783|"Dingell, John D"|2605
            > > N00001806|"Oberstar, James L"|14265
            > > N00001811|"Smith, Lamar"|15445
            > > N00001813|"Serrano, Jose E"|29134
            > > N00001817|"Young, C W Bill"|13047
            > > N00001821|"Hoyer, Steny H"|14873
            > > N00001861|"Waxman, Henry A"|14280
            > > N00001945|"Mikulski, Barbara A"|14440
            > > N00001955|"Cardin, Ben"|15408
            > > N00001979|"Sarbanes, Paul S"|13039
            > > N00002061|"Warner, John W"|14712
            > > N00002073|"Wolf, Frank R"|14869
            > > N00002091|"Craig, Larry"|14809
            > > N00002171|"Boucher, Rick"|15010
            > > N00002198|"Rahall, Nick"|14448
            > > N00002200|"Byrd, Robert C"|1366
            > > N00002214|"Mollohan, Alan B"|15083
            > > N00002247|"Coble, Howard"|15092
            > > N00002260|"Price, David"|15438
            > > N00002377|"Ballenger, Cass"|15402
            > > N00002423|"Hollings, Fritz"|11204
            > > N00002492|"Spratt, John M Jr"|15064
            > > N00002577|"Lewis, John"|15431
            > > N00002742|"Graham, Bob"|15503
            > > N00002782|"Stearns, Cliff"|15627
            > > N00002858|"Ros-Lehtinen, Ileana"|15634
            > > N00002877|"Shaw, E Clay Jr"|14860
            > > N00002982|"Bilirakis, Michael"|15006
            > > N00003126|"Gordon, Bart"|15100
            > > N00003132|"Cooper, Jim"|15019
            > > N00003209|"Duncan, John J Jr"|15455
            > > N00003254|"Tanner, John"|15628
            > > N00003328|"Cochran, Thad"|14009
            > > N00003329|"Lott, Trent"|14031
            > > N00003350|"Taylor, Gene"|15637
            > > N00003389|"McConnell, Mitch"|14921
            > > N00003437|"Bunning, Jim"|15406
            > > N00003473|"Rogers, Hal"|14854
            > > N00003522|"Kaptur, Marcy"|15029
            > > N00003651|"Regula, Ralph"|14045
            > > N00003660|"Gillmor, Paul E"|15604
            > > N00003709|"DeWine, Mike"|15020
            > > N00003736|"Oxley, Michael G"|14875
            > > N00003813|"Visclosky, Pete"|15124
            > > N00003950|"Levin, Sander"|15033
            > > N00004029|"Conyers, John Jr"|10713
            > > N00004070|"Kildee, Dale E"|14430
            > > N00004133|"Upton, Fred"|15446
            > > N00004207|"Harkin, Tom"|14230
            > > N00004280|"Leach, Jim"|14432
            > > N00004291|"Sensenbrenner, F James Jr"|14657
            > > N00004309|"Kohl, Herb"|15703
            > > N00004330|"Kleczka, Jerry"|15082
            > > N00004394|"Obey, David R"|12036
            > > N00004426|"Petri, Tom"|14675
            > > N00004489|"Sabo, Martin Olav"|14656
            > > N00004583|"Daschle, Tom"|14617
            > > N00004613|"Conrad, Kent"|15502
            > > N00004615|"Dorgan, Byron L"|14812
            > > N00004638|"Burns, Conrad"|15701
            > > N00004643|"Baucus, Max"|14203
            > > N00004698|"Crane, Phil"|12041
            > > N00004702|"Hyde, Henry J"|14239
            > > N00004781|"Hastert, Dennis"|15417
            > > N00004856|"Lipinski, Bill"|15036
            > > N00004912|"Evans, Lane"|15023
            > > N00004956|"Costello, Jerry F"|15453
            > > N00004981|"Durbin, Dick"|15021
            > > N00005037|"Gephardt, Richard A"|14421
            > > N00005105|"Skelton, Ike"|14451
            > > N00005178|"Bond, Christopher S 'Kit'"|15501
            > > N00005285|"Roberts, Pat"|14852
            > > N00005331|"Bereuter, Doug"|14605
            > > N00005372|"Tauzin, Billy"|14679
            > > N00005385|"Breaux, John"|13056
            > > N00005407|"Baker, Richard"|15401
            > > N00005414|"McCrery, Jim"|15451
            > > N00005582|"Inhofe, James M"|15424
            > > N00005617|"Nickles, Don"|14908
            > > N00005645|"Hall, Ralph M"|14828
            > > N00005656|"Barton, Joe"|15085
            > > N00005677|"Frost, Martin"|14626
            > > N00005892|"DeLay, Tom"|15094
            > > N00005906|"Paul, Ron"|14290
            > > N00005998|"Ortiz, Solomon P"|15049
            > > N00006060|"Stenholm, Charles W"|14664
            > > N00006202|"Campbell, Ben Nighthorse"|15407
            > > N00006237|"Cheney, Dick"|14611
            > > N00006246|"Thomas, Craig"|15633
            > > N00006406|"Kyl, Jon"|15429
            > > N00006424|"McCain, John"|15039
            > > N00006486|"Kolbe, Jim"|15105
            > > N00006515|"Domenici, Pete V"|14103
            > > N00006518|"Bingaman, Jeff"|14912
            > > N00006692|"Boxer, Barbara"|15011
            > > N00006932|"Dreier, David"|14813
            > > N00006983|"Hunter, Duncan"|14835
            > > N00007087|"Lewis, Jerry"|14644
            > > N00007124|"Cox, Christopher"|15601
            > > N00007151|"Rohrabacher, Dana"|15621
            > > N00007231|"Gallegly, Elton"|15413
            > > N00007360|"Pelosi, Nancy"|15448
            > > N00007382|"Lantos, Tom"|14837
            > > N00007390|"Miller, George"|14256
            > > N00007397|"Stark, Pete"|14053
            > > N00007584|"Herger, Wally"|15420
            > > N00007653|"Akaka, Daniel K"|14400
            > > N00007665|"Abercrombie, Neil"|15245
            > > N00007724|"Wyden, Ron"|14871
            > > N00007781|"DeFazio, Peter"|15410
            > > N00007918|"Dicks, Norm"|14413
            > > N00007997|"Stevens, Ted"|12109
            > > N00007999|"Young, Don"|14066
            > > N00008094|"Berman, Howard L"|15005
            > > N00009816|"Smith, Chris"|14863
            > > N00009829|"McDermott, Jim"|15613
            > > N00009869|"Hatch, Orrin G"|14503
            > > N00009918|"Leahy, Patrick"|14307
            > > N00009920|"Shelby, Richard C"|14659
            > > N00009922|"Reid, Harry"|15054
            > > N00009926|"Nelson, Bill"|14651
            > > N00010084|"Johnson, Tim"|15425
            > > N00011971|"Lungren, Dan"|14647
            > > N00012508|"Carper, Tom"|15015
            > > N99999981|"Rumsfeld, Donald H."|10622
            > >
            > >
            > > --- In govtrack@yahoogroups.com, Derek Willis <dwillis@> wrote:
            > > >
            > > > In constructing committee membership data for the NYT Congress API, I
            > > used
            > > > data found via Charles Stewart's site:
            > > >
            > > > http://web.mit.edu/17.251/www/data_page.html#2
            > > >
            > > > As Josh says, older data contains errors in it - I recall incorrect party
            > > > affiliations and phantom committee assignments when I checked them
            > > against
            > > > official sources. So much so that our API only vouches for 110-112th
            > > > committee data...
            > > >
            > > > Derek
            > > >
            > > >
            > > >
            > > > On Tue, May 31, 2011 at 12:18 PM, Josh Tauberer <tauberer@>wrote:
            > >
            > > >
            > > > >
            > > > >
            > > > > Hi, Peter.
            > > > >
            > > > > > 1.) What is the source for committees.xml? Is there a page like
            > > > > people.xml's "Source Data" page for committees.xml on Govtrack?
            > > > >
            > > > > The committee data is automatically scraped from
            > > > >
            > > > >
            > > > >
            > > http://www.senate.gov/pagelayout/committees/b_three_sections_with_teasers/membership.htm
            > > > >
            > > > > and
            > > > >
            > > > > http://clerk.house.gov/committee_info/index.html
            > > > >
            > > > > (As I mentioned on the labs list, the file was last generated a few
            > > > > months ago- March 3. I can re-generate it to get the latest info.)
            > > > >
            > > > > > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the
            > > > > same data available for other years?
            > > > >
            > > > > No I wasn't collecting that info then, and archival data is not
            > > > > available from the House/Senate (at least in the form I scrape).
            > > > >
            > > > > > 3.) The "Source Data" page says that people.xml "has been put
            > > > > together from a variety of sources and is maintained by hand." What are
            > > > > the sources?
            > > > >
            > > > > A full list of data sources is at govtrack.us/credits.xpd. That said,
            > > > > the amount of data I used from each source and the quality of the
            > > > > sources varied a lot. For information from about 2003 and on, the info
            > > > > about Members of Congress has been entered by hand by me.
            > > > >
            > > > > As you go further back in time, the quality of party affiliations,
            > > > > district assignments, and links to other IDs (e.g. ICPSR) grows worse.
            > > > > But it's probably the best anyone has anyway.
            > > > >
            > > > > Good luck. If you manage to put together a database of additional
            > > > > information, I hope you'll share it.
            > > > >
            > > > > - Josh Tauberer
            > > > > - CivicImpulse
            > > > >
            > > > > http://razor.occams.info
            > > > > http://www.civicimpulse.com
            > > > >
            > > > > "Yields falsehood when preceded by its quotation! Yields
            > > > > falsehood when preceded by its quotation!" Achilles to
            > > > > Tortoise (in "Godel, Escher, Bach" by Douglas Hofstadter)
            > > > >
            > > > > On 05/31/2011 11:29 AM, codekiln wrote:
            > > > > > I found out about the govtrack database from my post [1] over at the
            > > > > google group for Sunlight Labs' API. As I mentioned there, I have a
            > > list of
            > > > > CRP ID / bioguide id / name triples. I would like to
            > > > > > assemble information on what committees each legislator was a member
            > > > > > of, and any special titles they held in that committee, such as
            > > > > > chairman or ranking member. Mr. Tauberer suggested I use govtrack's
            > > > > people.xml [2] and committees.xml [3].
            > > > > >
            > > > > > I am helping a professor assemble this data, so it is important for
            > > me to
            > > > > be able to explain the paper trail of the data. I have seen Govtrack's
            > > > > "Source Data" page [4], but I still have some questions.
            > > > > >
            > > > > > 1.) What is the source for committees.xml? Is there a page like
            > > > > people.xml's "Source Data" page for committees.xml on Govtrack?
            > > > > >
            > > > > > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the
            > > same
            > > > > data available for other years?
            > > > > >
            > > > > > 3.) The "Source Data" page says that people.xml "has been put
            > > together
            > > > > from a variety of sources and is maintained by hand." What are the
            > > sources?
            > > > > >
            > > > > > Thanks,
            > > > > > Peter
            > > > > >
            > > > > > [1]
            > > > >
            > > http://groups.google.com/group/sunlightlabs/browse_thread/thread/7f35405ebf184e0e
            > > > > >
            > > > > > [2] http://www.govtrack.us/data/us/people.xml
            > > > > >
            > > > > > [3] http://www.govtrack.us/data/us/112/committees.xml
            > > > > >
            > > > > > [4] http://www.govtrack.us/developers/data.xpd
            > > > > >
            > > > > >
            > > > > >
            > > > > > ------------------------------------
            > > > > >
            > > > > > Yahoo! Groups Links
            > > > > >
            > > > > >
            > > > > >
            > > > >
            > > > >
            > > >
            > >
            > >
            > >
            >


          Your message has been successfully submitted and would be delivered to recipients shortly.