Loading ...
Sorry, an error occurred while loading the content.
 

Re: [govtrack] Data sources and people.xml/people/person/role@district

Expand Messages
  • Josh Tauberer
    Hi, Peter. ... people.xml s Source Data page for committees.xml on Govtrack? The committee data is automatically scraped from
    Message 1 of 8 , May 31, 2011
      Hi, Peter.

      > 1.) What is the source for committees.xml? Is there a page like
      people.xml's "Source Data" page for committees.xml on Govtrack?

      The committee data is automatically scraped from

      http://www.senate.gov/pagelayout/committees/b_three_sections_with_teasers/membership.htm

      and

      http://clerk.house.gov/committee_info/index.html

      (As I mentioned on the labs list, the file was last generated a few
      months ago- March 3. I can re-generate it to get the latest info.)

      > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the
      same data available for other years?

      No I wasn't collecting that info then, and archival data is not
      available from the House/Senate (at least in the form I scrape).

      > 3.) The "Source Data" page says that people.xml "has been put
      together from a variety of sources and is maintained by hand." What are
      the sources?

      A full list of data sources is at govtrack.us/credits.xpd. That said,
      the amount of data I used from each source and the quality of the
      sources varied a lot. For information from about 2003 and on, the info
      about Members of Congress has been entered by hand by me.

      As you go further back in time, the quality of party affiliations,
      district assignments, and links to other IDs (e.g. ICPSR) grows worse.
      But it's probably the best anyone has anyway.

      Good luck. If you manage to put together a database of additional
      information, I hope you'll share it.


      - Josh Tauberer
      - CivicImpulse

      http://razor.occams.info
      http://www.civicimpulse.com

      "Yields falsehood when preceded by its quotation! Yields
      falsehood when preceded by its quotation!" Achilles to
      Tortoise (in "Godel, Escher, Bach" by Douglas Hofstadter)

      On 05/31/2011 11:29 AM, codekiln wrote:
      > I found out about the govtrack database from my post [1] over at the google group for Sunlight Labs' API. As I mentioned there, I have a list of CRP ID / bioguide id / name triples. I would like to
      > assemble information on what committees each legislator was a member
      > of, and any special titles they held in that committee, such as
      > chairman or ranking member. Mr. Tauberer suggested I use govtrack's people.xml [2] and committees.xml [3].
      >
      > I am helping a professor assemble this data, so it is important for me to be able to explain the paper trail of the data. I have seen Govtrack's "Source Data" page [4], but I still have some questions.
      >
      > 1.) What is the source for committees.xml? Is there a page like people.xml's "Source Data" page for committees.xml on Govtrack?
      >
      > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the same data available for other years?
      >
      > 3.) The "Source Data" page says that people.xml "has been put together from a variety of sources and is maintained by hand." What are the sources?
      >
      > Thanks,
      > Peter
      >
      > [1] http://groups.google.com/group/sunlightlabs/browse_thread/thread/7f35405ebf184e0e
      >
      > [2] http://www.govtrack.us/data/us/people.xml
      >
      > [3] http://www.govtrack.us/data/us/112/committees.xml
      >
      > [4] http://www.govtrack.us/developers/data.xpd
      >
      >
      >
      > ------------------------------------
      >
      > Yahoo! Groups Links
      >
      >
      >
    • Derek Willis
      In constructing committee membership data for the NYT Congress API, I used data found via Charles Stewart s site:
      Message 2 of 8 , May 31, 2011
        In constructing committee membership data for the NYT Congress API, I used data found via Charles Stewart's site:


        As Josh says, older data contains errors in it - I recall incorrect party affiliations and phantom committee assignments when I checked them against official sources. So much so that our API only vouches for 110-112th committee data...

        Derek



        On Tue, May 31, 2011 at 12:18 PM, Josh Tauberer <tauberer@...> wrote:
         

        Hi, Peter.

        > 1.) What is the source for committees.xml? Is there a page like
        people.xml's "Source Data" page for committees.xml on Govtrack?

        The committee data is automatically scraped from

        http://www.senate.gov/pagelayout/committees/b_three_sections_with_teasers/membership.htm

        and

        http://clerk.house.gov/committee_info/index.html

        (As I mentioned on the labs list, the file was last generated a few
        months ago- March 3. I can re-generate it to get the latest info.)

        > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the
        same data available for other years?

        No I wasn't collecting that info then, and archival data is not
        available from the House/Senate (at least in the form I scrape).

        > 3.) The "Source Data" page says that people.xml "has been put
        together from a variety of sources and is maintained by hand." What are
        the sources?

        A full list of data sources is at govtrack.us/credits.xpd. That said,
        the amount of data I used from each source and the quality of the
        sources varied a lot. For information from about 2003 and on, the info
        about Members of Congress has been entered by hand by me.

        As you go further back in time, the quality of party affiliations,
        district assignments, and links to other IDs (e.g. ICPSR) grows worse.
        But it's probably the best anyone has anyway.

        Good luck. If you manage to put together a database of additional
        information, I hope you'll share it.

        - Josh Tauberer
        - CivicImpulse

        http://razor.occams.info
        http://www.civicimpulse.com

        "Yields falsehood when preceded by its quotation! Yields
        falsehood when preceded by its quotation!" Achilles to
        Tortoise (in "Godel, Escher, Bach" by Douglas Hofstadter)

        On 05/31/2011 11:29 AM, codekiln wrote:
        > I found out about the govtrack database from my post [1] over at the google group for Sunlight Labs' API. As I mentioned there, I have a list of CRP ID / bioguide id / name triples. I would like to
        > assemble information on what committees each legislator was a member
        > of, and any special titles they held in that committee, such as
        > chairman or ranking member. Mr. Tauberer suggested I use govtrack's people.xml [2] and committees.xml [3].
        >
        > I am helping a professor assemble this data, so it is important for me to be able to explain the paper trail of the data. I have seen Govtrack's "Source Data" page [4], but I still have some questions.
        >
        > 1.) What is the source for committees.xml? Is there a page like people.xml's "Source Data" page for committees.xml on Govtrack?
        >
        > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the same data available for other years?
        >
        > 3.) The "Source Data" page says that people.xml "has been put together from a variety of sources and is maintained by hand." What are the sources?
        >
        > Thanks,
        > Peter
        >
        > [1] http://groups.google.com/group/sunlightlabs/browse_thread/thread/7f35405ebf184e0e
        >
        > [2] http://www.govtrack.us/data/us/people.xml
        >
        > [3] http://www.govtrack.us/data/us/112/committees.xml
        >
        > [4] http://www.govtrack.us/developers/data.xpd
        >
        >
        >
        > ------------------------------------
        >
        > Yahoo! Groups Links
        >
        >
        >


      • codekiln
        Darek and Josh, Thanks for the citations and the info. Using people.xml I was able to extract ICPSRID for 165 out of the 793 identities I am researching; I ve
        Message 3 of 8 , May 31, 2011
          Darek and Josh,
          Thanks for the citations and the info. Using people.xml I was able to extract ICPSRID for 165 out of the 793 identities I am researching; I've put the results below for google [2].

          It seems as though the four or five data sources I have located are not always consistent on the committee names. Darek, what official sources did you use to double check Stewart's data? I had actually wondered whether there were phantom committees in Stewart's data because the committee name capitalization is inconsistent and the assignments lack subcommittees.

          Thanks for tipping me off to your work at NYT Congress API; I noticed the "data for earlier Congresses will be available soon" note in the committee call documentation [1].

          What a lively digital community there is around politician data; it seems like every few months a new API appears online.

          Thanks,
          Peter

          [1] http://developer.nytimes.com/docs/congress_api#h3-committees
          [2]
          OSID|CRPNAME|govtrack_icpsrid
          N00000010|"Burton, Dan"|15014
          N00000048|"Hefley, Joel"|15419
          N00000153|"Neal, Richard E"|15616
          N00000245|"Kerry, John"|14920
          N00000270|"Markey, Edward J"|14435
          N00000275|"Frank, Barney"|14824
          N00000308|"Kennedy, Edward M"|10808
          N00000444|"Gregg, Judd"|14826
          N00000480|"Snowe, Olympia J"|14661
          N00000534|"Jeffords, James M"|14240
          N00000561|"Johnson, Nancy L"|15028
          N00000581|"Dodd, Christopher J"|14213
          N00000616|"Lieberman, Joe"|15704
          N00000652|"Shays, Christopher"|15449
          N00000659|"Lautenberg, Frank R"|14914
          N00000716|"Payne, Donald M"|15619
          N00000781|"Pallone, Frank Jr"|15454
          N00000834|"Saxton, Jim"|15112
          N00000964|"Rangel, Charles B"|13035
          N00001003|"Engel, Eliot L"|15603
          N00001024|"Lowey, Nita M"|15612
          N00001082|"Towns, Edolphus"|15072
          N00001093|"Schumer, Charles E"|14858
          N00001143|"Ackerman, Gary"|15000
          N00001214|"McNulty, Michael R"|15614
          N00001261|"Walsh, James T"|15630
          N00001267|"Boehlert, Sherwood"|15007
          N00001311|"Slaughter, Louise M"|15444
          N00001329|"Houghton, Amo"|15423
          N00001408|"Murtha, John P"|14072
          N00001509|"Kanjorski, Paul E"|15104
          N00001535|"Weldon, Curt"|15447
          N00001604|"Specter, Arlen"|14910
          N00001669|"Biden, Joseph R Jr"|14101
          N00001685|"Rockefeller, Jay"|14922
          N00001691|"Levin, Carl"|14709
          N00001701|"Mineta, Norman Y."|14257
          N00001758|"Grassley, Chuck"|14226
          N00001762|"Inouye, Daniel K"|4812
          N00001764|"Lugar, Richard G"|14506
          N00001783|"Dingell, John D"|2605
          N00001806|"Oberstar, James L"|14265
          N00001811|"Smith, Lamar"|15445
          N00001813|"Serrano, Jose E"|29134
          N00001817|"Young, C W Bill"|13047
          N00001821|"Hoyer, Steny H"|14873
          N00001861|"Waxman, Henry A"|14280
          N00001945|"Mikulski, Barbara A"|14440
          N00001955|"Cardin, Ben"|15408
          N00001979|"Sarbanes, Paul S"|13039
          N00002061|"Warner, John W"|14712
          N00002073|"Wolf, Frank R"|14869
          N00002091|"Craig, Larry"|14809
          N00002171|"Boucher, Rick"|15010
          N00002198|"Rahall, Nick"|14448
          N00002200|"Byrd, Robert C"|1366
          N00002214|"Mollohan, Alan B"|15083
          N00002247|"Coble, Howard"|15092
          N00002260|"Price, David"|15438
          N00002377|"Ballenger, Cass"|15402
          N00002423|"Hollings, Fritz"|11204
          N00002492|"Spratt, John M Jr"|15064
          N00002577|"Lewis, John"|15431
          N00002742|"Graham, Bob"|15503
          N00002782|"Stearns, Cliff"|15627
          N00002858|"Ros-Lehtinen, Ileana"|15634
          N00002877|"Shaw, E Clay Jr"|14860
          N00002982|"Bilirakis, Michael"|15006
          N00003126|"Gordon, Bart"|15100
          N00003132|"Cooper, Jim"|15019
          N00003209|"Duncan, John J Jr"|15455
          N00003254|"Tanner, John"|15628
          N00003328|"Cochran, Thad"|14009
          N00003329|"Lott, Trent"|14031
          N00003350|"Taylor, Gene"|15637
          N00003389|"McConnell, Mitch"|14921
          N00003437|"Bunning, Jim"|15406
          N00003473|"Rogers, Hal"|14854
          N00003522|"Kaptur, Marcy"|15029
          N00003651|"Regula, Ralph"|14045
          N00003660|"Gillmor, Paul E"|15604
          N00003709|"DeWine, Mike"|15020
          N00003736|"Oxley, Michael G"|14875
          N00003813|"Visclosky, Pete"|15124
          N00003950|"Levin, Sander"|15033
          N00004029|"Conyers, John Jr"|10713
          N00004070|"Kildee, Dale E"|14430
          N00004133|"Upton, Fred"|15446
          N00004207|"Harkin, Tom"|14230
          N00004280|"Leach, Jim"|14432
          N00004291|"Sensenbrenner, F James Jr"|14657
          N00004309|"Kohl, Herb"|15703
          N00004330|"Kleczka, Jerry"|15082
          N00004394|"Obey, David R"|12036
          N00004426|"Petri, Tom"|14675
          N00004489|"Sabo, Martin Olav"|14656
          N00004583|"Daschle, Tom"|14617
          N00004613|"Conrad, Kent"|15502
          N00004615|"Dorgan, Byron L"|14812
          N00004638|"Burns, Conrad"|15701
          N00004643|"Baucus, Max"|14203
          N00004698|"Crane, Phil"|12041
          N00004702|"Hyde, Henry J"|14239
          N00004781|"Hastert, Dennis"|15417
          N00004856|"Lipinski, Bill"|15036
          N00004912|"Evans, Lane"|15023
          N00004956|"Costello, Jerry F"|15453
          N00004981|"Durbin, Dick"|15021
          N00005037|"Gephardt, Richard A"|14421
          N00005105|"Skelton, Ike"|14451
          N00005178|"Bond, Christopher S 'Kit'"|15501
          N00005285|"Roberts, Pat"|14852
          N00005331|"Bereuter, Doug"|14605
          N00005372|"Tauzin, Billy"|14679
          N00005385|"Breaux, John"|13056
          N00005407|"Baker, Richard"|15401
          N00005414|"McCrery, Jim"|15451
          N00005582|"Inhofe, James M"|15424
          N00005617|"Nickles, Don"|14908
          N00005645|"Hall, Ralph M"|14828
          N00005656|"Barton, Joe"|15085
          N00005677|"Frost, Martin"|14626
          N00005892|"DeLay, Tom"|15094
          N00005906|"Paul, Ron"|14290
          N00005998|"Ortiz, Solomon P"|15049
          N00006060|"Stenholm, Charles W"|14664
          N00006202|"Campbell, Ben Nighthorse"|15407
          N00006237|"Cheney, Dick"|14611
          N00006246|"Thomas, Craig"|15633
          N00006406|"Kyl, Jon"|15429
          N00006424|"McCain, John"|15039
          N00006486|"Kolbe, Jim"|15105
          N00006515|"Domenici, Pete V"|14103
          N00006518|"Bingaman, Jeff"|14912
          N00006692|"Boxer, Barbara"|15011
          N00006932|"Dreier, David"|14813
          N00006983|"Hunter, Duncan"|14835
          N00007087|"Lewis, Jerry"|14644
          N00007124|"Cox, Christopher"|15601
          N00007151|"Rohrabacher, Dana"|15621
          N00007231|"Gallegly, Elton"|15413
          N00007360|"Pelosi, Nancy"|15448
          N00007382|"Lantos, Tom"|14837
          N00007390|"Miller, George"|14256
          N00007397|"Stark, Pete"|14053
          N00007584|"Herger, Wally"|15420
          N00007653|"Akaka, Daniel K"|14400
          N00007665|"Abercrombie, Neil"|15245
          N00007724|"Wyden, Ron"|14871
          N00007781|"DeFazio, Peter"|15410
          N00007918|"Dicks, Norm"|14413
          N00007997|"Stevens, Ted"|12109
          N00007999|"Young, Don"|14066
          N00008094|"Berman, Howard L"|15005
          N00009816|"Smith, Chris"|14863
          N00009829|"McDermott, Jim"|15613
          N00009869|"Hatch, Orrin G"|14503
          N00009918|"Leahy, Patrick"|14307
          N00009920|"Shelby, Richard C"|14659
          N00009922|"Reid, Harry"|15054
          N00009926|"Nelson, Bill"|14651
          N00010084|"Johnson, Tim"|15425
          N00011971|"Lungren, Dan"|14647
          N00012508|"Carper, Tom"|15015
          N99999981|"Rumsfeld, Donald H."|10622

          --- In govtrack@yahoogroups.com, Derek Willis <dwillis@...> wrote:
          >
          > In constructing committee membership data for the NYT Congress API, I used
          > data found via Charles Stewart's site:
          >
          > http://web.mit.edu/17.251/www/data_page.html#2
          >
          > As Josh says, older data contains errors in it - I recall incorrect party
          > affiliations and phantom committee assignments when I checked them against
          > official sources. So much so that our API only vouches for 110-112th
          > committee data...
          >
          > Derek
          >
          >
          >
          > On Tue, May 31, 2011 at 12:18 PM, Josh Tauberer <tauberer@...>wrote:
          >
          > >
          > >
          > > Hi, Peter.
          > >
          > > > 1.) What is the source for committees.xml? Is there a page like
          > > people.xml's "Source Data" page for committees.xml on Govtrack?
          > >
          > > The committee data is automatically scraped from
          > >
          > >
          > > http://www.senate.gov/pagelayout/committees/b_three_sections_with_teasers/membership.htm
          > >
          > > and
          > >
          > > http://clerk.house.gov/committee_info/index.html
          > >
          > > (As I mentioned on the labs list, the file was last generated a few
          > > months ago- March 3. I can re-generate it to get the latest info.)
          > >
          > > > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the
          > > same data available for other years?
          > >
          > > No I wasn't collecting that info then, and archival data is not
          > > available from the House/Senate (at least in the form I scrape).
          > >
          > > > 3.) The "Source Data" page says that people.xml "has been put
          > > together from a variety of sources and is maintained by hand." What are
          > > the sources?
          > >
          > > A full list of data sources is at govtrack.us/credits.xpd. That said,
          > > the amount of data I used from each source and the quality of the
          > > sources varied a lot. For information from about 2003 and on, the info
          > > about Members of Congress has been entered by hand by me.
          > >
          > > As you go further back in time, the quality of party affiliations,
          > > district assignments, and links to other IDs (e.g. ICPSR) grows worse.
          > > But it's probably the best anyone has anyway.
          > >
          > > Good luck. If you manage to put together a database of additional
          > > information, I hope you'll share it.
          > >
          > > - Josh Tauberer
          > > - CivicImpulse
          > >
          > > http://razor.occams.info
          > > http://www.civicimpulse.com
          > >
          > > "Yields falsehood when preceded by its quotation! Yields
          > > falsehood when preceded by its quotation!" Achilles to
          > > Tortoise (in "Godel, Escher, Bach" by Douglas Hofstadter)
          > >
          > > On 05/31/2011 11:29 AM, codekiln wrote:
          > > > I found out about the govtrack database from my post [1] over at the
          > > google group for Sunlight Labs' API. As I mentioned there, I have a list of
          > > CRP ID / bioguide id / name triples. I would like to
          > > > assemble information on what committees each legislator was a member
          > > > of, and any special titles they held in that committee, such as
          > > > chairman or ranking member. Mr. Tauberer suggested I use govtrack's
          > > people.xml [2] and committees.xml [3].
          > > >
          > > > I am helping a professor assemble this data, so it is important for me to
          > > be able to explain the paper trail of the data. I have seen Govtrack's
          > > "Source Data" page [4], but I still have some questions.
          > > >
          > > > 1.) What is the source for committees.xml? Is there a page like
          > > people.xml's "Source Data" page for committees.xml on Govtrack?
          > > >
          > > > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the same
          > > data available for other years?
          > > >
          > > > 3.) The "Source Data" page says that people.xml "has been put together
          > > from a variety of sources and is maintained by hand." What are the sources?
          > > >
          > > > Thanks,
          > > > Peter
          > > >
          > > > [1]
          > > http://groups.google.com/group/sunlightlabs/browse_thread/thread/7f35405ebf184e0e
          > > >
          > > > [2] http://www.govtrack.us/data/us/people.xml
          > > >
          > > > [3] http://www.govtrack.us/data/us/112/committees.xml
          > > >
          > > > [4] http://www.govtrack.us/developers/data.xpd
          > > >
          > > >
          > > >
          > > > ------------------------------------
          > > >
          > > > Yahoo! Groups Links
          > > >
          > > >
          > > >
          > >
          > >
          >
        • Derek Willis
          Peter, We have ICPSRIDs for more than 2,600 members - if you think that would be a valuable addition to the API, I m happy to add that to the members response.
          Message 4 of 8 , May 31, 2011
            Peter,

            We have ICPSRIDs for more than 2,600 members - if you think that would be a valuable addition to the API, I'm happy to add that to the members response. The committee names are tough because they can change from congress to congress. So what was the House Banking Committee is now the House Financial Services Committee. Depending on which party is in the majority, the House Education and Labor Committee becomes the House Education and the Workforce Committee, and so on. For my money, the official source is Congress itself, which means looking up committee assignments in the Record, among other methods. I haven't found phantom committees in Stewart's data, but rather phantom assignments in which a member is recorded as a member of a committee when official records do not reflect such an assignment.

            Derek



            On Tue, May 31, 2011 at 1:30 PM, codekiln <ptr.nore@...> wrote:
             



            Darek and Josh,
            Thanks for the citations and the info. Using people.xml I was able to extract ICPSRID for 165 out of the 793 identities I am researching; I've put the results below for google [2].

            It seems as though the four or five data sources I have located are not always consistent on the committee names. Darek, what official sources did you use to double check Stewart's data? I had actually wondered whether there were phantom committees in Stewart's data because the committee name capitalization is inconsistent and the assignments lack subcommittees.

            Thanks for tipping me off to your work at NYT Congress API; I noticed the "data for earlier Congresses will be available soon" note in the committee call documentation [1].

            What a lively digital community there is around politician data; it seems like every few months a new API appears online.

            Thanks,
            Peter

            [1] http://developer.nytimes.com/docs/congress_api#h3-committees
            [2]
            OSID|CRPNAME|govtrack_icpsrid
            N00000010|"Burton, Dan"|15014
            N00000048|"Hefley, Joel"|15419
            N00000153|"Neal, Richard E"|15616
            N00000245|"Kerry, John"|14920
            N00000270|"Markey, Edward J"|14435
            N00000275|"Frank, Barney"|14824
            N00000308|"Kennedy, Edward M"|10808
            N00000444|"Gregg, Judd"|14826
            N00000480|"Snowe, Olympia J"|14661
            N00000534|"Jeffords, James M"|14240
            N00000561|"Johnson, Nancy L"|15028
            N00000581|"Dodd, Christopher J"|14213
            N00000616|"Lieberman, Joe"|15704
            N00000652|"Shays, Christopher"|15449
            N00000659|"Lautenberg, Frank R"|14914
            N00000716|"Payne, Donald M"|15619
            N00000781|"Pallone, Frank Jr"|15454
            N00000834|"Saxton, Jim"|15112
            N00000964|"Rangel, Charles B"|13035
            N00001003|"Engel, Eliot L"|15603
            N00001024|"Lowey, Nita M"|15612
            N00001082|"Towns, Edolphus"|15072
            N00001093|"Schumer, Charles E"|14858
            N00001143|"Ackerman, Gary"|15000
            N00001214|"McNulty, Michael R"|15614
            N00001261|"Walsh, James T"|15630
            N00001267|"Boehlert, Sherwood"|15007
            N00001311|"Slaughter, Louise M"|15444
            N00001329|"Houghton, Amo"|15423
            N00001408|"Murtha, John P"|14072
            N00001509|"Kanjorski, Paul E"|15104
            N00001535|"Weldon, Curt"|15447
            N00001604|"Specter, Arlen"|14910
            N00001669|"Biden, Joseph R Jr"|14101
            N00001685|"Rockefeller, Jay"|14922
            N00001691|"Levin, Carl"|14709
            N00001701|"Mineta, Norman Y."|14257
            N00001758|"Grassley, Chuck"|14226
            N00001762|"Inouye, Daniel K"|4812
            N00001764|"Lugar, Richard G"|14506
            N00001783|"Dingell, John D"|2605
            N00001806|"Oberstar, James L"|14265
            N00001811|"Smith, Lamar"|15445
            N00001813|"Serrano, Jose E"|29134
            N00001817|"Young, C W Bill"|13047
            N00001821|"Hoyer, Steny H"|14873
            N00001861|"Waxman, Henry A"|14280
            N00001945|"Mikulski, Barbara A"|14440
            N00001955|"Cardin, Ben"|15408
            N00001979|"Sarbanes, Paul S"|13039
            N00002061|"Warner, John W"|14712
            N00002073|"Wolf, Frank R"|14869
            N00002091|"Craig, Larry"|14809
            N00002171|"Boucher, Rick"|15010
            N00002198|"Rahall, Nick"|14448
            N00002200|"Byrd, Robert C"|1366
            N00002214|"Mollohan, Alan B"|15083
            N00002247|"Coble, Howard"|15092
            N00002260|"Price, David"|15438
            N00002377|"Ballenger, Cass"|15402
            N00002423|"Hollings, Fritz"|11204
            N00002492|"Spratt, John M Jr"|15064
            N00002577|"Lewis, John"|15431
            N00002742|"Graham, Bob"|15503
            N00002782|"Stearns, Cliff"|15627
            N00002858|"Ros-Lehtinen, Ileana"|15634
            N00002877|"Shaw, E Clay Jr"|14860
            N00002982|"Bilirakis, Michael"|15006
            N00003126|"Gordon, Bart"|15100
            N00003132|"Cooper, Jim"|15019
            N00003209|"Duncan, John J Jr"|15455
            N00003254|"Tanner, John"|15628
            N00003328|"Cochran, Thad"|14009
            N00003329|"Lott, Trent"|14031
            N00003350|"Taylor, Gene"|15637
            N00003389|"McConnell, Mitch"|14921
            N00003437|"Bunning, Jim"|15406
            N00003473|"Rogers, Hal"|14854
            N00003522|"Kaptur, Marcy"|15029
            N00003651|"Regula, Ralph"|14045
            N00003660|"Gillmor, Paul E"|15604
            N00003709|"DeWine, Mike"|15020
            N00003736|"Oxley, Michael G"|14875
            N00003813|"Visclosky, Pete"|15124
            N00003950|"Levin, Sander"|15033
            N00004029|"Conyers, John Jr"|10713
            N00004070|"Kildee, Dale E"|14430
            N00004133|"Upton, Fred"|15446
            N00004207|"Harkin, Tom"|14230
            N00004280|"Leach, Jim"|14432
            N00004291|"Sensenbrenner, F James Jr"|14657
            N00004309|"Kohl, Herb"|15703
            N00004330|"Kleczka, Jerry"|15082
            N00004394|"Obey, David R"|12036
            N00004426|"Petri, Tom"|14675
            N00004489|"Sabo, Martin Olav"|14656
            N00004583|"Daschle, Tom"|14617
            N00004613|"Conrad, Kent"|15502
            N00004615|"Dorgan, Byron L"|14812
            N00004638|"Burns, Conrad"|15701
            N00004643|"Baucus, Max"|14203
            N00004698|"Crane, Phil"|12041
            N00004702|"Hyde, Henry J"|14239
            N00004781|"Hastert, Dennis"|15417
            N00004856|"Lipinski, Bill"|15036
            N00004912|"Evans, Lane"|15023
            N00004956|"Costello, Jerry F"|15453
            N00004981|"Durbin, Dick"|15021
            N00005037|"Gephardt, Richard A"|14421
            N00005105|"Skelton, Ike"|14451
            N00005178|"Bond, Christopher S 'Kit'"|15501
            N00005285|"Roberts, Pat"|14852
            N00005331|"Bereuter, Doug"|14605
            N00005372|"Tauzin, Billy"|14679
            N00005385|"Breaux, John"|13056
            N00005407|"Baker, Richard"|15401
            N00005414|"McCrery, Jim"|15451
            N00005582|"Inhofe, James M"|15424
            N00005617|"Nickles, Don"|14908
            N00005645|"Hall, Ralph M"|14828
            N00005656|"Barton, Joe"|15085
            N00005677|"Frost, Martin"|14626
            N00005892|"DeLay, Tom"|15094
            N00005906|"Paul, Ron"|14290
            N00005998|"Ortiz, Solomon P"|15049
            N00006060|"Stenholm, Charles W"|14664
            N00006202|"Campbell, Ben Nighthorse"|15407
            N00006237|"Cheney, Dick"|14611
            N00006246|"Thomas, Craig"|15633
            N00006406|"Kyl, Jon"|15429
            N00006424|"McCain, John"|15039
            N00006486|"Kolbe, Jim"|15105
            N00006515|"Domenici, Pete V"|14103
            N00006518|"Bingaman, Jeff"|14912
            N00006692|"Boxer, Barbara"|15011
            N00006932|"Dreier, David"|14813
            N00006983|"Hunter, Duncan"|14835
            N00007087|"Lewis, Jerry"|14644
            N00007124|"Cox, Christopher"|15601
            N00007151|"Rohrabacher, Dana"|15621
            N00007231|"Gallegly, Elton"|15413
            N00007360|"Pelosi, Nancy"|15448
            N00007382|"Lantos, Tom"|14837
            N00007390|"Miller, George"|14256
            N00007397|"Stark, Pete"|14053
            N00007584|"Herger, Wally"|15420
            N00007653|"Akaka, Daniel K"|14400
            N00007665|"Abercrombie, Neil"|15245
            N00007724|"Wyden, Ron"|14871
            N00007781|"DeFazio, Peter"|15410
            N00007918|"Dicks, Norm"|14413
            N00007997|"Stevens, Ted"|12109
            N00007999|"Young, Don"|14066
            N00008094|"Berman, Howard L"|15005
            N00009816|"Smith, Chris"|14863
            N00009829|"McDermott, Jim"|15613
            N00009869|"Hatch, Orrin G"|14503
            N00009918|"Leahy, Patrick"|14307
            N00009920|"Shelby, Richard C"|14659
            N00009922|"Reid, Harry"|15054
            N00009926|"Nelson, Bill"|14651
            N00010084|"Johnson, Tim"|15425
            N00011971|"Lungren, Dan"|14647
            N00012508|"Carper, Tom"|15015
            N99999981|"Rumsfeld, Donald H."|10622



            --- In govtrack@yahoogroups.com, Derek Willis <dwillis@...> wrote:
            >
            > In constructing committee membership data for the NYT Congress API, I used
            > data found via Charles Stewart's site:
            >
            > http://web.mit.edu/17.251/www/data_page.html#2
            >
            > As Josh says, older data contains errors in it - I recall incorrect party
            > affiliations and phantom committee assignments when I checked them against
            > official sources. So much so that our API only vouches for 110-112th
            > committee data...
            >
            > Derek
            >
            >
            >
            > On Tue, May 31, 2011 at 12:18 PM, Josh Tauberer <tauberer@...>wrote:

            >
            > >
            > >
            > > Hi, Peter.
            > >
            > > > 1.) What is the source for committees.xml? Is there a page like
            > > people.xml's "Source Data" page for committees.xml on Govtrack?
            > >
            > > The committee data is automatically scraped from
            > >
            > >
            > > http://www.senate.gov/pagelayout/committees/b_three_sections_with_teasers/membership.htm
            > >
            > > and
            > >
            > > http://clerk.house.gov/committee_info/index.html
            > >
            > > (As I mentioned on the labs list, the file was last generated a few
            > > months ago- March 3. I can re-generate it to get the latest info.)
            > >
            > > > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the
            > > same data available for other years?
            > >
            > > No I wasn't collecting that info then, and archival data is not
            > > available from the House/Senate (at least in the form I scrape).
            > >
            > > > 3.) The "Source Data" page says that people.xml "has been put
            > > together from a variety of sources and is maintained by hand." What are
            > > the sources?
            > >
            > > A full list of data sources is at govtrack.us/credits.xpd. That said,
            > > the amount of data I used from each source and the quality of the
            > > sources varied a lot. For information from about 2003 and on, the info
            > > about Members of Congress has been entered by hand by me.
            > >
            > > As you go further back in time, the quality of party affiliations,
            > > district assignments, and links to other IDs (e.g. ICPSR) grows worse.
            > > But it's probably the best anyone has anyway.
            > >
            > > Good luck. If you manage to put together a database of additional
            > > information, I hope you'll share it.
            > >
            > > - Josh Tauberer
            > > - CivicImpulse
            > >
            > > http://razor.occams.info
            > > http://www.civicimpulse.com
            > >
            > > "Yields falsehood when preceded by its quotation! Yields
            > > falsehood when preceded by its quotation!" Achilles to
            > > Tortoise (in "Godel, Escher, Bach" by Douglas Hofstadter)
            > >
            > > On 05/31/2011 11:29 AM, codekiln wrote:
            > > > I found out about the govtrack database from my post [1] over at the
            > > google group for Sunlight Labs' API. As I mentioned there, I have a list of
            > > CRP ID / bioguide id / name triples. I would like to
            > > > assemble information on what committees each legislator was a member
            > > > of, and any special titles they held in that committee, such as
            > > > chairman or ranking member. Mr. Tauberer suggested I use govtrack's
            > > people.xml [2] and committees.xml [3].
            > > >
            > > > I am helping a professor assemble this data, so it is important for me to
            > > be able to explain the paper trail of the data. I have seen Govtrack's
            > > "Source Data" page [4], but I still have some questions.
            > > >
            > > > 1.) What is the source for committees.xml? Is there a page like
            > > people.xml's "Source Data" page for committees.xml on Govtrack?
            > > >
            > > > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the same
            > > data available for other years?
            > > >
            > > > 3.) The "Source Data" page says that people.xml "has been put together
            > > from a variety of sources and is maintained by hand." What are the sources?
            > > >
            > > > Thanks,
            > > > Peter
            > > >
            > > > [1]
            > > http://groups.google.com/group/sunlightlabs/browse_thread/thread/7f35405ebf184e0e
            > > >
            > > > [2] http://www.govtrack.us/data/us/people.xml
            > > >
            > > > [3] http://www.govtrack.us/data/us/112/committees.xml
            > > >
            > > > [4] http://www.govtrack.us/developers/data.xpd
            > > >
            > > >
            > > >
            > > > ------------------------------------
            > > >
            > > > Yahoo! Groups Links
            > > >
            > > >
            > > >
            > >
            > >
            >


          • codekiln
            Hi Darek, Thanks for your detailed response. Your record of ICPSRIDs would be a valuable to the project I am working on, but I have no idea if ICPSRIDs belong
            Message 5 of 8 , Jun 2, 2011
              Hi Darek,
              Thanks for your detailed response.

              Your record of ICPSRIDs would be a valuable to the project I am working on, but I have no idea if ICPSRIDs belong in the NYT Congress API. Netizens definitely have a need some kind of database id Rosetta Stone. Bioguide ID - ICPSRID isn't the only important connection. When I have assembled all the different corresponding ids I will ask the professor if I may publish the cross-references I have found.

              Thanks,
              Peter

              --- In govtrack@yahoogroups.com, Derek Willis <dwillis@...> wrote:
              >
              > Peter,
              >
              > We have ICPSRIDs for more than 2,600 members - if you think that would be a
              > valuable addition to the API, I'm happy to add that to the members response.
              > The committee names are tough because they can change from congress to
              > congress. So what was the House Banking Committee is now the House Financial
              > Services Committee. Depending on which party is in the majority, the House
              > Education and Labor Committee becomes the House Education and the Workforce
              > Committee, and so on. For my money, the official source is Congress itself,
              > which means looking up committee assignments in the Record, among other
              > methods. I haven't found phantom committees in Stewart's data, but rather
              > phantom assignments in which a member is recorded as a member of a committee
              > when official records do not reflect such an assignment.
              >
              > Derek
              >
              >
              >
              > On Tue, May 31, 2011 at 1:30 PM, codekiln <ptr.nore@...> wrote:
              >
              > >
              > >
              > >
              > >
              > > Darek and Josh,
              > > Thanks for the citations and the info. Using people.xml I was able to
              > > extract ICPSRID for 165 out of the 793 identities I am researching; I've put
              > > the results below for google [2].
              > >
              > > It seems as though the four or five data sources I have located are not
              > > always consistent on the committee names. Darek, what official sources did
              > > you use to double check Stewart's data? I had actually wondered whether
              > > there were phantom committees in Stewart's data because the committee name
              > > capitalization is inconsistent and the assignments lack subcommittees.
              > >
              > > Thanks for tipping me off to your work at NYT Congress API; I noticed the
              > > "data for earlier Congresses will be available soon" note in the committee
              > > call documentation [1].
              > >
              > > What a lively digital community there is around politician data; it seems
              > > like every few months a new API appears online.
              > >
              > > Thanks,
              > > Peter
              > >
              > > [1] http://developer.nytimes.com/docs/congress_api#h3-committees
              > > [2]
              > > OSID|CRPNAME|govtrack_icpsrid
              > > N00000010|"Burton, Dan"|15014
              > > N00000048|"Hefley, Joel"|15419
              > > N00000153|"Neal, Richard E"|15616
              > > N00000245|"Kerry, John"|14920
              > > N00000270|"Markey, Edward J"|14435
              > > N00000275|"Frank, Barney"|14824
              > > N00000308|"Kennedy, Edward M"|10808
              > > N00000444|"Gregg, Judd"|14826
              > > N00000480|"Snowe, Olympia J"|14661
              > > N00000534|"Jeffords, James M"|14240
              > > N00000561|"Johnson, Nancy L"|15028
              > > N00000581|"Dodd, Christopher J"|14213
              > > N00000616|"Lieberman, Joe"|15704
              > > N00000652|"Shays, Christopher"|15449
              > > N00000659|"Lautenberg, Frank R"|14914
              > > N00000716|"Payne, Donald M"|15619
              > > N00000781|"Pallone, Frank Jr"|15454
              > > N00000834|"Saxton, Jim"|15112
              > > N00000964|"Rangel, Charles B"|13035
              > > N00001003|"Engel, Eliot L"|15603
              > > N00001024|"Lowey, Nita M"|15612
              > > N00001082|"Towns, Edolphus"|15072
              > > N00001093|"Schumer, Charles E"|14858
              > > N00001143|"Ackerman, Gary"|15000
              > > N00001214|"McNulty, Michael R"|15614
              > > N00001261|"Walsh, James T"|15630
              > > N00001267|"Boehlert, Sherwood"|15007
              > > N00001311|"Slaughter, Louise M"|15444
              > > N00001329|"Houghton, Amo"|15423
              > > N00001408|"Murtha, John P"|14072
              > > N00001509|"Kanjorski, Paul E"|15104
              > > N00001535|"Weldon, Curt"|15447
              > > N00001604|"Specter, Arlen"|14910
              > > N00001669|"Biden, Joseph R Jr"|14101
              > > N00001685|"Rockefeller, Jay"|14922
              > > N00001691|"Levin, Carl"|14709
              > > N00001701|"Mineta, Norman Y."|14257
              > > N00001758|"Grassley, Chuck"|14226
              > > N00001762|"Inouye, Daniel K"|4812
              > > N00001764|"Lugar, Richard G"|14506
              > > N00001783|"Dingell, John D"|2605
              > > N00001806|"Oberstar, James L"|14265
              > > N00001811|"Smith, Lamar"|15445
              > > N00001813|"Serrano, Jose E"|29134
              > > N00001817|"Young, C W Bill"|13047
              > > N00001821|"Hoyer, Steny H"|14873
              > > N00001861|"Waxman, Henry A"|14280
              > > N00001945|"Mikulski, Barbara A"|14440
              > > N00001955|"Cardin, Ben"|15408
              > > N00001979|"Sarbanes, Paul S"|13039
              > > N00002061|"Warner, John W"|14712
              > > N00002073|"Wolf, Frank R"|14869
              > > N00002091|"Craig, Larry"|14809
              > > N00002171|"Boucher, Rick"|15010
              > > N00002198|"Rahall, Nick"|14448
              > > N00002200|"Byrd, Robert C"|1366
              > > N00002214|"Mollohan, Alan B"|15083
              > > N00002247|"Coble, Howard"|15092
              > > N00002260|"Price, David"|15438
              > > N00002377|"Ballenger, Cass"|15402
              > > N00002423|"Hollings, Fritz"|11204
              > > N00002492|"Spratt, John M Jr"|15064
              > > N00002577|"Lewis, John"|15431
              > > N00002742|"Graham, Bob"|15503
              > > N00002782|"Stearns, Cliff"|15627
              > > N00002858|"Ros-Lehtinen, Ileana"|15634
              > > N00002877|"Shaw, E Clay Jr"|14860
              > > N00002982|"Bilirakis, Michael"|15006
              > > N00003126|"Gordon, Bart"|15100
              > > N00003132|"Cooper, Jim"|15019
              > > N00003209|"Duncan, John J Jr"|15455
              > > N00003254|"Tanner, John"|15628
              > > N00003328|"Cochran, Thad"|14009
              > > N00003329|"Lott, Trent"|14031
              > > N00003350|"Taylor, Gene"|15637
              > > N00003389|"McConnell, Mitch"|14921
              > > N00003437|"Bunning, Jim"|15406
              > > N00003473|"Rogers, Hal"|14854
              > > N00003522|"Kaptur, Marcy"|15029
              > > N00003651|"Regula, Ralph"|14045
              > > N00003660|"Gillmor, Paul E"|15604
              > > N00003709|"DeWine, Mike"|15020
              > > N00003736|"Oxley, Michael G"|14875
              > > N00003813|"Visclosky, Pete"|15124
              > > N00003950|"Levin, Sander"|15033
              > > N00004029|"Conyers, John Jr"|10713
              > > N00004070|"Kildee, Dale E"|14430
              > > N00004133|"Upton, Fred"|15446
              > > N00004207|"Harkin, Tom"|14230
              > > N00004280|"Leach, Jim"|14432
              > > N00004291|"Sensenbrenner, F James Jr"|14657
              > > N00004309|"Kohl, Herb"|15703
              > > N00004330|"Kleczka, Jerry"|15082
              > > N00004394|"Obey, David R"|12036
              > > N00004426|"Petri, Tom"|14675
              > > N00004489|"Sabo, Martin Olav"|14656
              > > N00004583|"Daschle, Tom"|14617
              > > N00004613|"Conrad, Kent"|15502
              > > N00004615|"Dorgan, Byron L"|14812
              > > N00004638|"Burns, Conrad"|15701
              > > N00004643|"Baucus, Max"|14203
              > > N00004698|"Crane, Phil"|12041
              > > N00004702|"Hyde, Henry J"|14239
              > > N00004781|"Hastert, Dennis"|15417
              > > N00004856|"Lipinski, Bill"|15036
              > > N00004912|"Evans, Lane"|15023
              > > N00004956|"Costello, Jerry F"|15453
              > > N00004981|"Durbin, Dick"|15021
              > > N00005037|"Gephardt, Richard A"|14421
              > > N00005105|"Skelton, Ike"|14451
              > > N00005178|"Bond, Christopher S 'Kit'"|15501
              > > N00005285|"Roberts, Pat"|14852
              > > N00005331|"Bereuter, Doug"|14605
              > > N00005372|"Tauzin, Billy"|14679
              > > N00005385|"Breaux, John"|13056
              > > N00005407|"Baker, Richard"|15401
              > > N00005414|"McCrery, Jim"|15451
              > > N00005582|"Inhofe, James M"|15424
              > > N00005617|"Nickles, Don"|14908
              > > N00005645|"Hall, Ralph M"|14828
              > > N00005656|"Barton, Joe"|15085
              > > N00005677|"Frost, Martin"|14626
              > > N00005892|"DeLay, Tom"|15094
              > > N00005906|"Paul, Ron"|14290
              > > N00005998|"Ortiz, Solomon P"|15049
              > > N00006060|"Stenholm, Charles W"|14664
              > > N00006202|"Campbell, Ben Nighthorse"|15407
              > > N00006237|"Cheney, Dick"|14611
              > > N00006246|"Thomas, Craig"|15633
              > > N00006406|"Kyl, Jon"|15429
              > > N00006424|"McCain, John"|15039
              > > N00006486|"Kolbe, Jim"|15105
              > > N00006515|"Domenici, Pete V"|14103
              > > N00006518|"Bingaman, Jeff"|14912
              > > N00006692|"Boxer, Barbara"|15011
              > > N00006932|"Dreier, David"|14813
              > > N00006983|"Hunter, Duncan"|14835
              > > N00007087|"Lewis, Jerry"|14644
              > > N00007124|"Cox, Christopher"|15601
              > > N00007151|"Rohrabacher, Dana"|15621
              > > N00007231|"Gallegly, Elton"|15413
              > > N00007360|"Pelosi, Nancy"|15448
              > > N00007382|"Lantos, Tom"|14837
              > > N00007390|"Miller, George"|14256
              > > N00007397|"Stark, Pete"|14053
              > > N00007584|"Herger, Wally"|15420
              > > N00007653|"Akaka, Daniel K"|14400
              > > N00007665|"Abercrombie, Neil"|15245
              > > N00007724|"Wyden, Ron"|14871
              > > N00007781|"DeFazio, Peter"|15410
              > > N00007918|"Dicks, Norm"|14413
              > > N00007997|"Stevens, Ted"|12109
              > > N00007999|"Young, Don"|14066
              > > N00008094|"Berman, Howard L"|15005
              > > N00009816|"Smith, Chris"|14863
              > > N00009829|"McDermott, Jim"|15613
              > > N00009869|"Hatch, Orrin G"|14503
              > > N00009918|"Leahy, Patrick"|14307
              > > N00009920|"Shelby, Richard C"|14659
              > > N00009922|"Reid, Harry"|15054
              > > N00009926|"Nelson, Bill"|14651
              > > N00010084|"Johnson, Tim"|15425
              > > N00011971|"Lungren, Dan"|14647
              > > N00012508|"Carper, Tom"|15015
              > > N99999981|"Rumsfeld, Donald H."|10622
              > >
              > >
              > > --- In govtrack@yahoogroups.com, Derek Willis <dwillis@> wrote:
              > > >
              > > > In constructing committee membership data for the NYT Congress API, I
              > > used
              > > > data found via Charles Stewart's site:
              > > >
              > > > http://web.mit.edu/17.251/www/data_page.html#2
              > > >
              > > > As Josh says, older data contains errors in it - I recall incorrect party
              > > > affiliations and phantom committee assignments when I checked them
              > > against
              > > > official sources. So much so that our API only vouches for 110-112th
              > > > committee data...
              > > >
              > > > Derek
              > > >
              > > >
              > > >
              > > > On Tue, May 31, 2011 at 12:18 PM, Josh Tauberer <tauberer@>wrote:
              > >
              > > >
              > > > >
              > > > >
              > > > > Hi, Peter.
              > > > >
              > > > > > 1.) What is the source for committees.xml? Is there a page like
              > > > > people.xml's "Source Data" page for committees.xml on Govtrack?
              > > > >
              > > > > The committee data is automatically scraped from
              > > > >
              > > > >
              > > > >
              > > http://www.senate.gov/pagelayout/committees/b_three_sections_with_teasers/membership.htm
              > > > >
              > > > > and
              > > > >
              > > > > http://clerk.house.gov/committee_info/index.html
              > > > >
              > > > > (As I mentioned on the labs list, the file was last generated a few
              > > > > months ago- March 3. I can re-generate it to get the latest info.)
              > > > >
              > > > > > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the
              > > > > same data available for other years?
              > > > >
              > > > > No I wasn't collecting that info then, and archival data is not
              > > > > available from the House/Senate (at least in the form I scrape).
              > > > >
              > > > > > 3.) The "Source Data" page says that people.xml "has been put
              > > > > together from a variety of sources and is maintained by hand." What are
              > > > > the sources?
              > > > >
              > > > > A full list of data sources is at govtrack.us/credits.xpd. That said,
              > > > > the amount of data I used from each source and the quality of the
              > > > > sources varied a lot. For information from about 2003 and on, the info
              > > > > about Members of Congress has been entered by hand by me.
              > > > >
              > > > > As you go further back in time, the quality of party affiliations,
              > > > > district assignments, and links to other IDs (e.g. ICPSR) grows worse.
              > > > > But it's probably the best anyone has anyway.
              > > > >
              > > > > Good luck. If you manage to put together a database of additional
              > > > > information, I hope you'll share it.
              > > > >
              > > > > - Josh Tauberer
              > > > > - CivicImpulse
              > > > >
              > > > > http://razor.occams.info
              > > > > http://www.civicimpulse.com
              > > > >
              > > > > "Yields falsehood when preceded by its quotation! Yields
              > > > > falsehood when preceded by its quotation!" Achilles to
              > > > > Tortoise (in "Godel, Escher, Bach" by Douglas Hofstadter)
              > > > >
              > > > > On 05/31/2011 11:29 AM, codekiln wrote:
              > > > > > I found out about the govtrack database from my post [1] over at the
              > > > > google group for Sunlight Labs' API. As I mentioned there, I have a
              > > list of
              > > > > CRP ID / bioguide id / name triples. I would like to
              > > > > > assemble information on what committees each legislator was a member
              > > > > > of, and any special titles they held in that committee, such as
              > > > > > chairman or ranking member. Mr. Tauberer suggested I use govtrack's
              > > > > people.xml [2] and committees.xml [3].
              > > > > >
              > > > > > I am helping a professor assemble this data, so it is important for
              > > me to
              > > > > be able to explain the paper trail of the data. I have seen Govtrack's
              > > > > "Source Data" page [4], but I still have some questions.
              > > > > >
              > > > > > 1.) What is the source for committees.xml? Is there a page like
              > > > > people.xml's "Source Data" page for committees.xml on Govtrack?
              > > > > >
              > > > > > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the
              > > same
              > > > > data available for other years?
              > > > > >
              > > > > > 3.) The "Source Data" page says that people.xml "has been put
              > > together
              > > > > from a variety of sources and is maintained by hand." What are the
              > > sources?
              > > > > >
              > > > > > Thanks,
              > > > > > Peter
              > > > > >
              > > > > > [1]
              > > > >
              > > http://groups.google.com/group/sunlightlabs/browse_thread/thread/7f35405ebf184e0e
              > > > > >
              > > > > > [2] http://www.govtrack.us/data/us/people.xml
              > > > > >
              > > > > > [3] http://www.govtrack.us/data/us/112/committees.xml
              > > > > >
              > > > > > [4] http://www.govtrack.us/developers/data.xpd
              > > > > >
              > > > > >
              > > > > >
              > > > > > ------------------------------------
              > > > > >
              > > > > > Yahoo! Groups Links
              > > > > >
              > > > > >
              > > > > >
              > > > >
              > > > >
              > > >
              > >
              > >
              > >
              >
            • Derek Willis
              Hey Peter, I think we ll put them in anyway, but I think the bioguide ID should be the canonical reference. Derek
              Message 6 of 8 , Jun 2, 2011
                Hey Peter,

                I think we'll put them in anyway, but I think the bioguide ID should be the canonical reference.

                Derek

                On Thu, Jun 2, 2011 at 11:49 AM, codekiln <ptr.nore@...> wrote:
                 



                Hi Darek,
                Thanks for your detailed response.

                Your record of ICPSRIDs would be a valuable to the project I am working on, but I have no idea if ICPSRIDs belong in the NYT Congress API. Netizens definitely have a need some kind of database id Rosetta Stone. Bioguide ID - ICPSRID isn't the only important connection. When I have assembled all the different corresponding ids I will ask the professor if I may publish the cross-references I have found.

                Thanks,
                Peter



                --- In govtrack@yahoogroups.com, Derek Willis <dwillis@...> wrote:
                >
                > Peter,
                >
                > We have ICPSRIDs for more than 2,600 members - if you think that would be a
                > valuable addition to the API, I'm happy to add that to the members response.
                > The committee names are tough because they can change from congress to
                > congress. So what was the House Banking Committee is now the House Financial
                > Services Committee. Depending on which party is in the majority, the House
                > Education and Labor Committee becomes the House Education and the Workforce
                > Committee, and so on. For my money, the official source is Congress itself,
                > which means looking up committee assignments in the Record, among other
                > methods. I haven't found phantom committees in Stewart's data, but rather
                > phantom assignments in which a member is recorded as a member of a committee
                > when official records do not reflect such an assignment.
                >
                > Derek
                >
                >
                >
                > On Tue, May 31, 2011 at 1:30 PM, codekiln <ptr.nore@...> wrote:
                >
                > >
                > >
                > >
                > >
                > > Darek and Josh,
                > > Thanks for the citations and the info. Using people.xml I was able to
                > > extract ICPSRID for 165 out of the 793 identities I am researching; I've put
                > > the results below for google [2].
                > >
                > > It seems as though the four or five data sources I have located are not
                > > always consistent on the committee names. Darek, what official sources did
                > > you use to double check Stewart's data? I had actually wondered whether
                > > there were phantom committees in Stewart's data because the committee name
                > > capitalization is inconsistent and the assignments lack subcommittees.
                > >
                > > Thanks for tipping me off to your work at NYT Congress API; I noticed the
                > > "data for earlier Congresses will be available soon" note in the committee
                > > call documentation [1].
                > >
                > > What a lively digital community there is around politician data; it seems
                > > like every few months a new API appears online.
                > >
                > > Thanks,
                > > Peter
                > >
                > > [1] http://developer.nytimes.com/docs/congress_api#h3-committees
                > > [2]
                > > OSID|CRPNAME|govtrack_icpsrid
                > > N00000010|"Burton, Dan"|15014
                > > N00000048|"Hefley, Joel"|15419
                > > N00000153|"Neal, Richard E"|15616
                > > N00000245|"Kerry, John"|14920
                > > N00000270|"Markey, Edward J"|14435
                > > N00000275|"Frank, Barney"|14824
                > > N00000308|"Kennedy, Edward M"|10808
                > > N00000444|"Gregg, Judd"|14826
                > > N00000480|"Snowe, Olympia J"|14661
                > > N00000534|"Jeffords, James M"|14240
                > > N00000561|"Johnson, Nancy L"|15028
                > > N00000581|"Dodd, Christopher J"|14213
                > > N00000616|"Lieberman, Joe"|15704
                > > N00000652|"Shays, Christopher"|15449
                > > N00000659|"Lautenberg, Frank R"|14914
                > > N00000716|"Payne, Donald M"|15619
                > > N00000781|"Pallone, Frank Jr"|15454
                > > N00000834|"Saxton, Jim"|15112
                > > N00000964|"Rangel, Charles B"|13035
                > > N00001003|"Engel, Eliot L"|15603
                > > N00001024|"Lowey, Nita M"|15612
                > > N00001082|"Towns, Edolphus"|15072
                > > N00001093|"Schumer, Charles E"|14858
                > > N00001143|"Ackerman, Gary"|15000
                > > N00001214|"McNulty, Michael R"|15614
                > > N00001261|"Walsh, James T"|15630
                > > N00001267|"Boehlert, Sherwood"|15007
                > > N00001311|"Slaughter, Louise M"|15444
                > > N00001329|"Houghton, Amo"|15423
                > > N00001408|"Murtha, John P"|14072
                > > N00001509|"Kanjorski, Paul E"|15104
                > > N00001535|"Weldon, Curt"|15447
                > > N00001604|"Specter, Arlen"|14910
                > > N00001669|"Biden, Joseph R Jr"|14101
                > > N00001685|"Rockefeller, Jay"|14922
                > > N00001691|"Levin, Carl"|14709
                > > N00001701|"Mineta, Norman Y."|14257
                > > N00001758|"Grassley, Chuck"|14226
                > > N00001762|"Inouye, Daniel K"|4812
                > > N00001764|"Lugar, Richard G"|14506
                > > N00001783|"Dingell, John D"|2605
                > > N00001806|"Oberstar, James L"|14265
                > > N00001811|"Smith, Lamar"|15445
                > > N00001813|"Serrano, Jose E"|29134
                > > N00001817|"Young, C W Bill"|13047
                > > N00001821|"Hoyer, Steny H"|14873
                > > N00001861|"Waxman, Henry A"|14280
                > > N00001945|"Mikulski, Barbara A"|14440
                > > N00001955|"Cardin, Ben"|15408
                > > N00001979|"Sarbanes, Paul S"|13039
                > > N00002061|"Warner, John W"|14712
                > > N00002073|"Wolf, Frank R"|14869
                > > N00002091|"Craig, Larry"|14809
                > > N00002171|"Boucher, Rick"|15010
                > > N00002198|"Rahall, Nick"|14448
                > > N00002200|"Byrd, Robert C"|1366
                > > N00002214|"Mollohan, Alan B"|15083
                > > N00002247|"Coble, Howard"|15092
                > > N00002260|"Price, David"|15438
                > > N00002377|"Ballenger, Cass"|15402
                > > N00002423|"Hollings, Fritz"|11204
                > > N00002492|"Spratt, John M Jr"|15064
                > > N00002577|"Lewis, John"|15431
                > > N00002742|"Graham, Bob"|15503
                > > N00002782|"Stearns, Cliff"|15627
                > > N00002858|"Ros-Lehtinen, Ileana"|15634
                > > N00002877|"Shaw, E Clay Jr"|14860
                > > N00002982|"Bilirakis, Michael"|15006
                > > N00003126|"Gordon, Bart"|15100
                > > N00003132|"Cooper, Jim"|15019
                > > N00003209|"Duncan, John J Jr"|15455
                > > N00003254|"Tanner, John"|15628
                > > N00003328|"Cochran, Thad"|14009
                > > N00003329|"Lott, Trent"|14031
                > > N00003350|"Taylor, Gene"|15637
                > > N00003389|"McConnell, Mitch"|14921
                > > N00003437|"Bunning, Jim"|15406
                > > N00003473|"Rogers, Hal"|14854
                > > N00003522|"Kaptur, Marcy"|15029
                > > N00003651|"Regula, Ralph"|14045
                > > N00003660|"Gillmor, Paul E"|15604
                > > N00003709|"DeWine, Mike"|15020
                > > N00003736|"Oxley, Michael G"|14875
                > > N00003813|"Visclosky, Pete"|15124
                > > N00003950|"Levin, Sander"|15033
                > > N00004029|"Conyers, John Jr"|10713
                > > N00004070|"Kildee, Dale E"|14430
                > > N00004133|"Upton, Fred"|15446
                > > N00004207|"Harkin, Tom"|14230
                > > N00004280|"Leach, Jim"|14432
                > > N00004291|"Sensenbrenner, F James Jr"|14657
                > > N00004309|"Kohl, Herb"|15703
                > > N00004330|"Kleczka, Jerry"|15082
                > > N00004394|"Obey, David R"|12036
                > > N00004426|"Petri, Tom"|14675
                > > N00004489|"Sabo, Martin Olav"|14656
                > > N00004583|"Daschle, Tom"|14617
                > > N00004613|"Conrad, Kent"|15502
                > > N00004615|"Dorgan, Byron L"|14812
                > > N00004638|"Burns, Conrad"|15701
                > > N00004643|"Baucus, Max"|14203
                > > N00004698|"Crane, Phil"|12041
                > > N00004702|"Hyde, Henry J"|14239
                > > N00004781|"Hastert, Dennis"|15417
                > > N00004856|"Lipinski, Bill"|15036
                > > N00004912|"Evans, Lane"|15023
                > > N00004956|"Costello, Jerry F"|15453
                > > N00004981|"Durbin, Dick"|15021
                > > N00005037|"Gephardt, Richard A"|14421
                > > N00005105|"Skelton, Ike"|14451
                > > N00005178|"Bond, Christopher S 'Kit'"|15501
                > > N00005285|"Roberts, Pat"|14852
                > > N00005331|"Bereuter, Doug"|14605
                > > N00005372|"Tauzin, Billy"|14679
                > > N00005385|"Breaux, John"|13056
                > > N00005407|"Baker, Richard"|15401
                > > N00005414|"McCrery, Jim"|15451
                > > N00005582|"Inhofe, James M"|15424
                > > N00005617|"Nickles, Don"|14908
                > > N00005645|"Hall, Ralph M"|14828
                > > N00005656|"Barton, Joe"|15085
                > > N00005677|"Frost, Martin"|14626
                > > N00005892|"DeLay, Tom"|15094
                > > N00005906|"Paul, Ron"|14290
                > > N00005998|"Ortiz, Solomon P"|15049
                > > N00006060|"Stenholm, Charles W"|14664
                > > N00006202|"Campbell, Ben Nighthorse"|15407
                > > N00006237|"Cheney, Dick"|14611
                > > N00006246|"Thomas, Craig"|15633
                > > N00006406|"Kyl, Jon"|15429
                > > N00006424|"McCain, John"|15039
                > > N00006486|"Kolbe, Jim"|15105
                > > N00006515|"Domenici, Pete V"|14103
                > > N00006518|"Bingaman, Jeff"|14912
                > > N00006692|"Boxer, Barbara"|15011
                > > N00006932|"Dreier, David"|14813
                > > N00006983|"Hunter, Duncan"|14835
                > > N00007087|"Lewis, Jerry"|14644
                > > N00007124|"Cox, Christopher"|15601
                > > N00007151|"Rohrabacher, Dana"|15621
                > > N00007231|"Gallegly, Elton"|15413
                > > N00007360|"Pelosi, Nancy"|15448
                > > N00007382|"Lantos, Tom"|14837
                > > N00007390|"Miller, George"|14256
                > > N00007397|"Stark, Pete"|14053
                > > N00007584|"Herger, Wally"|15420
                > > N00007653|"Akaka, Daniel K"|14400
                > > N00007665|"Abercrombie, Neil"|15245
                > > N00007724|"Wyden, Ron"|14871
                > > N00007781|"DeFazio, Peter"|15410
                > > N00007918|"Dicks, Norm"|14413
                > > N00007997|"Stevens, Ted"|12109
                > > N00007999|"Young, Don"|14066
                > > N00008094|"Berman, Howard L"|15005
                > > N00009816|"Smith, Chris"|14863
                > > N00009829|"McDermott, Jim"|15613
                > > N00009869|"Hatch, Orrin G"|14503
                > > N00009918|"Leahy, Patrick"|14307
                > > N00009920|"Shelby, Richard C"|14659
                > > N00009922|"Reid, Harry"|15054
                > > N00009926|"Nelson, Bill"|14651
                > > N00010084|"Johnson, Tim"|15425
                > > N00011971|"Lungren, Dan"|14647
                > > N00012508|"Carper, Tom"|15015
                > > N99999981|"Rumsfeld, Donald H."|10622
                > >
                > >
                > > --- In govtrack@yahoogroups.com, Derek Willis <dwillis@> wrote:
                > > >
                > > > In constructing committee membership data for the NYT Congress API, I
                > > used
                > > > data found via Charles Stewart's site:
                > > >
                > > > http://web.mit.edu/17.251/www/data_page.html#2
                > > >
                > > > As Josh says, older data contains errors in it - I recall incorrect party
                > > > affiliations and phantom committee assignments when I checked them
                > > against
                > > > official sources. So much so that our API only vouches for 110-112th
                > > > committee data...
                > > >
                > > > Derek
                > > >
                > > >
                > > >
                > > > On Tue, May 31, 2011 at 12:18 PM, Josh Tauberer <tauberer@>wrote:
                > >
                > > >
                > > > >
                > > > >
                > > > > Hi, Peter.
                > > > >
                > > > > > 1.) What is the source for committees.xml? Is there a page like
                > > > > people.xml's "Source Data" page for committees.xml on Govtrack?
                > > > >
                > > > > The committee data is automatically scraped from
                > > > >
                > > > >
                > > > >
                > > http://www.senate.gov/pagelayout/committees/b_three_sections_with_teasers/membership.htm
                > > > >
                > > > > and
                > > > >
                > > > > http://clerk.house.gov/committee_info/index.html
                > > > >
                > > > > (As I mentioned on the labs list, the file was last generated a few
                > > > > months ago- March 3. I can re-generate it to get the latest info.)
                > > > >
                > > > > > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the
                > > > > same data available for other years?
                > > > >
                > > > > No I wasn't collecting that info then, and archival data is not
                > > > > available from the House/Senate (at least in the form I scrape).
                > > > >
                > > > > > 3.) The "Source Data" page says that people.xml "has been put
                > > > > together from a variety of sources and is maintained by hand." What are
                > > > > the sources?
                > > > >
                > > > > A full list of data sources is at govtrack.us/credits.xpd. That said,
                > > > > the amount of data I used from each source and the quality of the
                > > > > sources varied a lot. For information from about 2003 and on, the info
                > > > > about Members of Congress has been entered by hand by me.
                > > > >
                > > > > As you go further back in time, the quality of party affiliations,
                > > > > district assignments, and links to other IDs (e.g. ICPSR) grows worse.
                > > > > But it's probably the best anyone has anyway.
                > > > >
                > > > > Good luck. If you manage to put together a database of additional
                > > > > information, I hope you'll share it.
                > > > >
                > > > > - Josh Tauberer
                > > > > - CivicImpulse
                > > > >
                > > > > http://razor.occams.info
                > > > > http://www.civicimpulse.com
                > > > >
                > > > > "Yields falsehood when preceded by its quotation! Yields
                > > > > falsehood when preceded by its quotation!" Achilles to
                > > > > Tortoise (in "Godel, Escher, Bach" by Douglas Hofstadter)
                > > > >
                > > > > On 05/31/2011 11:29 AM, codekiln wrote:
                > > > > > I found out about the govtrack database from my post [1] over at the
                > > > > google group for Sunlight Labs' API. As I mentioned there, I have a
                > > list of
                > > > > CRP ID / bioguide id / name triples. I would like to
                > > > > > assemble information on what committees each legislator was a member
                > > > > > of, and any special titles they held in that committee, such as
                > > > > > chairman or ranking member. Mr. Tauberer suggested I use govtrack's
                > > > > people.xml [2] and committees.xml [3].
                > > > > >
                > > > > > I am helping a professor assemble this data, so it is important for
                > > me to
                > > > > be able to explain the paper trail of the data. I have seen Govtrack's
                > > > > "Source Data" page [4], but I still have some questions.
                > > > > >
                > > > > > 1.) What is the source for committees.xml? Is there a page like
                > > > > people.xml's "Source Data" page for committees.xml on Govtrack?
                > > > > >
                > > > > > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the
                > > same
                > > > > data available for other years?
                > > > > >
                > > > > > 3.) The "Source Data" page says that people.xml "has been put
                > > together
                > > > > from a variety of sources and is maintained by hand." What are the
                > > sources?
                > > > > >
                > > > > > Thanks,
                > > > > > Peter
                > > > > >
                > > > > > [1]
                > > > >
                > > http://groups.google.com/group/sunlightlabs/browse_thread/thread/7f35405ebf184e0e
                > > > > >
                > > > > > [2] http://www.govtrack.us/data/us/people.xml
                > > > > >
                > > > > > [3] http://www.govtrack.us/data/us/112/committees.xml
                > > > > >
                > > > > > [4] http://www.govtrack.us/developers/data.xpd
                > > > > >
                > > > > >
                > > > > >
                > > > > > ------------------------------------
                > > > > >
                > > > > > Yahoo! Groups Links
                > > > > >
                > > > > >
                > > > > >
                > > > >
                > > > >
                > > >
                > >
                > >
                > >
                >


              • Derek Willis
                Just as an FYI, we ll probably make this official next week but the ICPSR IDs that we have are now in the Member response of the NYT Congress API.
                Message 7 of 8 , Jun 3, 2011
                  Just as an FYI, we'll probably make this official next week but the ICPSR IDs that we have are now in the Member response of the NYT Congress API. 


                  On Thu, Jun 2, 2011 at 11:49 AM, codekiln <ptr.nore@...> wrote:
                   



                  Hi Darek,
                  Thanks for your detailed response.

                  Your record of ICPSRIDs would be a valuable to the project I am working on, but I have no idea if ICPSRIDs belong in the NYT Congress API. Netizens definitely have a need some kind of database id Rosetta Stone. Bioguide ID - ICPSRID isn't the only important connection. When I have assembled all the different corresponding ids I will ask the professor if I may publish the cross-references I have found.

                  Thanks,
                  Peter



                  --- In govtrack@yahoogroups.com, Derek Willis <dwillis@...> wrote:
                  >
                  > Peter,
                  >
                  > We have ICPSRIDs for more than 2,600 members - if you think that would be a
                  > valuable addition to the API, I'm happy to add that to the members response.
                  > The committee names are tough because they can change from congress to
                  > congress. So what was the House Banking Committee is now the House Financial
                  > Services Committee. Depending on which party is in the majority, the House
                  > Education and Labor Committee becomes the House Education and the Workforce
                  > Committee, and so on. For my money, the official source is Congress itself,
                  > which means looking up committee assignments in the Record, among other
                  > methods. I haven't found phantom committees in Stewart's data, but rather
                  > phantom assignments in which a member is recorded as a member of a committee
                  > when official records do not reflect such an assignment.
                  >
                  > Derek
                  >
                  >
                  >
                  > On Tue, May 31, 2011 at 1:30 PM, codekiln <ptr.nore@...> wrote:
                  >
                  > >
                  > >
                  > >
                  > >
                  > > Darek and Josh,
                  > > Thanks for the citations and the info. Using people.xml I was able to
                  > > extract ICPSRID for 165 out of the 793 identities I am researching; I've put
                  > > the results below for google [2].
                  > >
                  > > It seems as though the four or five data sources I have located are not
                  > > always consistent on the committee names. Darek, what official sources did
                  > > you use to double check Stewart's data? I had actually wondered whether
                  > > there were phantom committees in Stewart's data because the committee name
                  > > capitalization is inconsistent and the assignments lack subcommittees.
                  > >
                  > > Thanks for tipping me off to your work at NYT Congress API; I noticed the
                  > > "data for earlier Congresses will be available soon" note in the committee
                  > > call documentation [1].
                  > >
                  > > What a lively digital community there is around politician data; it seems
                  > > like every few months a new API appears online.
                  > >
                  > > Thanks,
                  > > Peter
                  > >
                  > > [1] http://developer.nytimes.com/docs/congress_api#h3-committees
                  > > [2]
                  > > OSID|CRPNAME|govtrack_icpsrid
                  > > N00000010|"Burton, Dan"|15014
                  > > N00000048|"Hefley, Joel"|15419
                  > > N00000153|"Neal, Richard E"|15616
                  > > N00000245|"Kerry, John"|14920
                  > > N00000270|"Markey, Edward J"|14435
                  > > N00000275|"Frank, Barney"|14824
                  > > N00000308|"Kennedy, Edward M"|10808
                  > > N00000444|"Gregg, Judd"|14826
                  > > N00000480|"Snowe, Olympia J"|14661
                  > > N00000534|"Jeffords, James M"|14240
                  > > N00000561|"Johnson, Nancy L"|15028
                  > > N00000581|"Dodd, Christopher J"|14213
                  > > N00000616|"Lieberman, Joe"|15704
                  > > N00000652|"Shays, Christopher"|15449
                  > > N00000659|"Lautenberg, Frank R"|14914
                  > > N00000716|"Payne, Donald M"|15619
                  > > N00000781|"Pallone, Frank Jr"|15454
                  > > N00000834|"Saxton, Jim"|15112
                  > > N00000964|"Rangel, Charles B"|13035
                  > > N00001003|"Engel, Eliot L"|15603
                  > > N00001024|"Lowey, Nita M"|15612
                  > > N00001082|"Towns, Edolphus"|15072
                  > > N00001093|"Schumer, Charles E"|14858
                  > > N00001143|"Ackerman, Gary"|15000
                  > > N00001214|"McNulty, Michael R"|15614
                  > > N00001261|"Walsh, James T"|15630
                  > > N00001267|"Boehlert, Sherwood"|15007
                  > > N00001311|"Slaughter, Louise M"|15444
                  > > N00001329|"Houghton, Amo"|15423
                  > > N00001408|"Murtha, John P"|14072
                  > > N00001509|"Kanjorski, Paul E"|15104
                  > > N00001535|"Weldon, Curt"|15447
                  > > N00001604|"Specter, Arlen"|14910
                  > > N00001669|"Biden, Joseph R Jr"|14101
                  > > N00001685|"Rockefeller, Jay"|14922
                  > > N00001691|"Levin, Carl"|14709
                  > > N00001701|"Mineta, Norman Y."|14257
                  > > N00001758|"Grassley, Chuck"|14226
                  > > N00001762|"Inouye, Daniel K"|4812
                  > > N00001764|"Lugar, Richard G"|14506
                  > > N00001783|"Dingell, John D"|2605
                  > > N00001806|"Oberstar, James L"|14265
                  > > N00001811|"Smith, Lamar"|15445
                  > > N00001813|"Serrano, Jose E"|29134
                  > > N00001817|"Young, C W Bill"|13047
                  > > N00001821|"Hoyer, Steny H"|14873
                  > > N00001861|"Waxman, Henry A"|14280
                  > > N00001945|"Mikulski, Barbara A"|14440
                  > > N00001955|"Cardin, Ben"|15408
                  > > N00001979|"Sarbanes, Paul S"|13039
                  > > N00002061|"Warner, John W"|14712
                  > > N00002073|"Wolf, Frank R"|14869
                  > > N00002091|"Craig, Larry"|14809
                  > > N00002171|"Boucher, Rick"|15010
                  > > N00002198|"Rahall, Nick"|14448
                  > > N00002200|"Byrd, Robert C"|1366
                  > > N00002214|"Mollohan, Alan B"|15083
                  > > N00002247|"Coble, Howard"|15092
                  > > N00002260|"Price, David"|15438
                  > > N00002377|"Ballenger, Cass"|15402
                  > > N00002423|"Hollings, Fritz"|11204
                  > > N00002492|"Spratt, John M Jr"|15064
                  > > N00002577|"Lewis, John"|15431
                  > > N00002742|"Graham, Bob"|15503
                  > > N00002782|"Stearns, Cliff"|15627
                  > > N00002858|"Ros-Lehtinen, Ileana"|15634
                  > > N00002877|"Shaw, E Clay Jr"|14860
                  > > N00002982|"Bilirakis, Michael"|15006
                  > > N00003126|"Gordon, Bart"|15100
                  > > N00003132|"Cooper, Jim"|15019
                  > > N00003209|"Duncan, John J Jr"|15455
                  > > N00003254|"Tanner, John"|15628
                  > > N00003328|"Cochran, Thad"|14009
                  > > N00003329|"Lott, Trent"|14031
                  > > N00003350|"Taylor, Gene"|15637
                  > > N00003389|"McConnell, Mitch"|14921
                  > > N00003437|"Bunning, Jim"|15406
                  > > N00003473|"Rogers, Hal"|14854
                  > > N00003522|"Kaptur, Marcy"|15029
                  > > N00003651|"Regula, Ralph"|14045
                  > > N00003660|"Gillmor, Paul E"|15604
                  > > N00003709|"DeWine, Mike"|15020
                  > > N00003736|"Oxley, Michael G"|14875
                  > > N00003813|"Visclosky, Pete"|15124
                  > > N00003950|"Levin, Sander"|15033
                  > > N00004029|"Conyers, John Jr"|10713
                  > > N00004070|"Kildee, Dale E"|14430
                  > > N00004133|"Upton, Fred"|15446
                  > > N00004207|"Harkin, Tom"|14230
                  > > N00004280|"Leach, Jim"|14432
                  > > N00004291|"Sensenbrenner, F James Jr"|14657
                  > > N00004309|"Kohl, Herb"|15703
                  > > N00004330|"Kleczka, Jerry"|15082
                  > > N00004394|"Obey, David R"|12036
                  > > N00004426|"Petri, Tom"|14675
                  > > N00004489|"Sabo, Martin Olav"|14656
                  > > N00004583|"Daschle, Tom"|14617
                  > > N00004613|"Conrad, Kent"|15502
                  > > N00004615|"Dorgan, Byron L"|14812
                  > > N00004638|"Burns, Conrad"|15701
                  > > N00004643|"Baucus, Max"|14203
                  > > N00004698|"Crane, Phil"|12041
                  > > N00004702|"Hyde, Henry J"|14239
                  > > N00004781|"Hastert, Dennis"|15417
                  > > N00004856|"Lipinski, Bill"|15036
                  > > N00004912|"Evans, Lane"|15023
                  > > N00004956|"Costello, Jerry F"|15453
                  > > N00004981|"Durbin, Dick"|15021
                  > > N00005037|"Gephardt, Richard A"|14421
                  > > N00005105|"Skelton, Ike"|14451
                  > > N00005178|"Bond, Christopher S 'Kit'"|15501
                  > > N00005285|"Roberts, Pat"|14852
                  > > N00005331|"Bereuter, Doug"|14605
                  > > N00005372|"Tauzin, Billy"|14679
                  > > N00005385|"Breaux, John"|13056
                  > > N00005407|"Baker, Richard"|15401
                  > > N00005414|"McCrery, Jim"|15451
                  > > N00005582|"Inhofe, James M"|15424
                  > > N00005617|"Nickles, Don"|14908
                  > > N00005645|"Hall, Ralph M"|14828
                  > > N00005656|"Barton, Joe"|15085
                  > > N00005677|"Frost, Martin"|14626
                  > > N00005892|"DeLay, Tom"|15094
                  > > N00005906|"Paul, Ron"|14290
                  > > N00005998|"Ortiz, Solomon P"|15049
                  > > N00006060|"Stenholm, Charles W"|14664
                  > > N00006202|"Campbell, Ben Nighthorse"|15407
                  > > N00006237|"Cheney, Dick"|14611
                  > > N00006246|"Thomas, Craig"|15633
                  > > N00006406|"Kyl, Jon"|15429
                  > > N00006424|"McCain, John"|15039
                  > > N00006486|"Kolbe, Jim"|15105
                  > > N00006515|"Domenici, Pete V"|14103
                  > > N00006518|"Bingaman, Jeff"|14912
                  > > N00006692|"Boxer, Barbara"|15011
                  > > N00006932|"Dreier, David"|14813
                  > > N00006983|"Hunter, Duncan"|14835
                  > > N00007087|"Lewis, Jerry"|14644
                  > > N00007124|"Cox, Christopher"|15601
                  > > N00007151|"Rohrabacher, Dana"|15621
                  > > N00007231|"Gallegly, Elton"|15413
                  > > N00007360|"Pelosi, Nancy"|15448
                  > > N00007382|"Lantos, Tom"|14837
                  > > N00007390|"Miller, George"|14256
                  > > N00007397|"Stark, Pete"|14053
                  > > N00007584|"Herger, Wally"|15420
                  > > N00007653|"Akaka, Daniel K"|14400
                  > > N00007665|"Abercrombie, Neil"|15245
                  > > N00007724|"Wyden, Ron"|14871
                  > > N00007781|"DeFazio, Peter"|15410
                  > > N00007918|"Dicks, Norm"|14413
                  > > N00007997|"Stevens, Ted"|12109
                  > > N00007999|"Young, Don"|14066
                  > > N00008094|"Berman, Howard L"|15005
                  > > N00009816|"Smith, Chris"|14863
                  > > N00009829|"McDermott, Jim"|15613
                  > > N00009869|"Hatch, Orrin G"|14503
                  > > N00009918|"Leahy, Patrick"|14307
                  > > N00009920|"Shelby, Richard C"|14659
                  > > N00009922|"Reid, Harry"|15054
                  > > N00009926|"Nelson, Bill"|14651
                  > > N00010084|"Johnson, Tim"|15425
                  > > N00011971|"Lungren, Dan"|14647
                  > > N00012508|"Carper, Tom"|15015
                  > > N99999981|"Rumsfeld, Donald H."|10622
                  > >
                  > >
                  > > --- In govtrack@yahoogroups.com, Derek Willis <dwillis@> wrote:
                  > > >
                  > > > In constructing committee membership data for the NYT Congress API, I
                  > > used
                  > > > data found via Charles Stewart's site:
                  > > >
                  > > > http://web.mit.edu/17.251/www/data_page.html#2
                  > > >
                  > > > As Josh says, older data contains errors in it - I recall incorrect party
                  > > > affiliations and phantom committee assignments when I checked them
                  > > against
                  > > > official sources. So much so that our API only vouches for 110-112th
                  > > > committee data...
                  > > >
                  > > > Derek
                  > > >
                  > > >
                  > > >
                  > > > On Tue, May 31, 2011 at 12:18 PM, Josh Tauberer <tauberer@>wrote:
                  > >
                  > > >
                  > > > >
                  > > > >
                  > > > > Hi, Peter.
                  > > > >
                  > > > > > 1.) What is the source for committees.xml? Is there a page like
                  > > > > people.xml's "Source Data" page for committees.xml on Govtrack?
                  > > > >
                  > > > > The committee data is automatically scraped from
                  > > > >
                  > > > >
                  > > > >
                  > > http://www.senate.gov/pagelayout/committees/b_three_sections_with_teasers/membership.htm
                  > > > >
                  > > > > and
                  > > > >
                  > > > > http://clerk.house.gov/committee_info/index.html
                  > > > >
                  > > > > (As I mentioned on the labs list, the file was last generated a few
                  > > > > months ago- March 3. I can re-generate it to get the latest info.)
                  > > > >
                  > > > > > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the
                  > > > > same data available for other years?
                  > > > >
                  > > > > No I wasn't collecting that info then, and archival data is not
                  > > > > available from the House/Senate (at least in the form I scrape).
                  > > > >
                  > > > > > 3.) The "Source Data" page says that people.xml "has been put
                  > > > > together from a variety of sources and is maintained by hand." What are
                  > > > > the sources?
                  > > > >
                  > > > > A full list of data sources is at govtrack.us/credits.xpd. That said,
                  > > > > the amount of data I used from each source and the quality of the
                  > > > > sources varied a lot. For information from about 2003 and on, the info
                  > > > > about Members of Congress has been entered by hand by me.
                  > > > >
                  > > > > As you go further back in time, the quality of party affiliations,
                  > > > > district assignments, and links to other IDs (e.g. ICPSR) grows worse.
                  > > > > But it's probably the best anyone has anyway.
                  > > > >
                  > > > > Good luck. If you manage to put together a database of additional
                  > > > > information, I hope you'll share it.
                  > > > >
                  > > > > - Josh Tauberer
                  > > > > - CivicImpulse
                  > > > >
                  > > > > http://razor.occams.info
                  > > > > http://www.civicimpulse.com
                  > > > >
                  > > > > "Yields falsehood when preceded by its quotation! Yields
                  > > > > falsehood when preceded by its quotation!" Achilles to
                  > > > > Tortoise (in "Godel, Escher, Bach" by Douglas Hofstadter)
                  > > > >
                  > > > > On 05/31/2011 11:29 AM, codekiln wrote:
                  > > > > > I found out about the govtrack database from my post [1] over at the
                  > > > > google group for Sunlight Labs' API. As I mentioned there, I have a
                  > > list of
                  > > > > CRP ID / bioguide id / name triples. I would like to
                  > > > > > assemble information on what committees each legislator was a member
                  > > > > > of, and any special titles they held in that committee, such as
                  > > > > > chairman or ranking member. Mr. Tauberer suggested I use govtrack's
                  > > > > people.xml [2] and committees.xml [3].
                  > > > > >
                  > > > > > I am helping a professor assemble this data, so it is important for
                  > > me to
                  > > > > be able to explain the paper trail of the data. I have seen Govtrack's
                  > > > > "Source Data" page [4], but I still have some questions.
                  > > > > >
                  > > > > > 1.) What is the source for committees.xml? Is there a page like
                  > > > > people.xml's "Source Data" page for committees.xml on Govtrack?
                  > > > > >
                  > > > > > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the
                  > > same
                  > > > > data available for other years?
                  > > > > >
                  > > > > > 3.) The "Source Data" page says that people.xml "has been put
                  > > together
                  > > > > from a variety of sources and is maintained by hand." What are the
                  > > sources?
                  > > > > >
                  > > > > > Thanks,
                  > > > > > Peter
                  > > > > >
                  > > > > > [1]
                  > > > >
                  > > http://groups.google.com/group/sunlightlabs/browse_thread/thread/7f35405ebf184e0e
                  > > > > >
                  > > > > > [2] http://www.govtrack.us/data/us/people.xml
                  > > > > >
                  > > > > > [3] http://www.govtrack.us/data/us/112/committees.xml
                  > > > > >
                  > > > > > [4] http://www.govtrack.us/developers/data.xpd
                  > > > > >
                  > > > > >
                  > > > > >
                  > > > > > ------------------------------------
                  > > > > >
                  > > > > > Yahoo! Groups Links
                  > > > > >
                  > > > > >
                  > > > > >
                  > > > >
                  > > > >
                  > > >
                  > >
                  > >
                  > >
                  >


                Your message has been successfully submitted and would be delivered to recipients shortly.