Loading ...
Sorry, an error occurred while loading the content.
 

Data sources and people.xml/people/person/role@district

Expand Messages
  • codekiln
    I found out about the govtrack database from my post [1] over at the google group for Sunlight Labs API. As I mentioned there, I have a list of CRP ID /
    Message 1 of 8 , May 31, 2011
      I found out about the govtrack database from my post [1] over at the google group for Sunlight Labs' API. As I mentioned there, I have a list of CRP ID / bioguide id / name triples. I would like to
      assemble information on what committees each legislator was a member
      of, and any special titles they held in that committee, such as
      chairman or ranking member. Mr. Tauberer suggested I use govtrack's people.xml [2] and committees.xml [3].

      I am helping a professor assemble this data, so it is important for me to be able to explain the paper trail of the data. I have seen Govtrack's "Source Data" page [4], but I still have some questions.

      1.) What is the source for committees.xml? Is there a page like people.xml's "Source Data" page for committees.xml on Govtrack?

      2.) I need data for 103rd to 110th congresses (2003-2009). Is the same data available for other years?

      3.) The "Source Data" page says that people.xml "has been put together from a variety of sources and is maintained by hand." What are the sources?

      Thanks,
      Peter

      [1] http://groups.google.com/group/sunlightlabs/browse_thread/thread/7f35405ebf184e0e

      [2] http://www.govtrack.us/data/us/people.xml

      [3] http://www.govtrack.us/data/us/112/committees.xml

      [4] http://www.govtrack.us/developers/data.xpd
    • Josh Tauberer
      Hi, Peter. ... people.xml s Source Data page for committees.xml on Govtrack? The committee data is automatically scraped from
      Message 2 of 8 , May 31, 2011
        Hi, Peter.

        > 1.) What is the source for committees.xml? Is there a page like
        people.xml's "Source Data" page for committees.xml on Govtrack?

        The committee data is automatically scraped from

        http://www.senate.gov/pagelayout/committees/b_three_sections_with_teasers/membership.htm

        and

        http://clerk.house.gov/committee_info/index.html

        (As I mentioned on the labs list, the file was last generated a few
        months ago- March 3. I can re-generate it to get the latest info.)

        > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the
        same data available for other years?

        No I wasn't collecting that info then, and archival data is not
        available from the House/Senate (at least in the form I scrape).

        > 3.) The "Source Data" page says that people.xml "has been put
        together from a variety of sources and is maintained by hand." What are
        the sources?

        A full list of data sources is at govtrack.us/credits.xpd. That said,
        the amount of data I used from each source and the quality of the
        sources varied a lot. For information from about 2003 and on, the info
        about Members of Congress has been entered by hand by me.

        As you go further back in time, the quality of party affiliations,
        district assignments, and links to other IDs (e.g. ICPSR) grows worse.
        But it's probably the best anyone has anyway.

        Good luck. If you manage to put together a database of additional
        information, I hope you'll share it.


        - Josh Tauberer
        - CivicImpulse

        http://razor.occams.info
        http://www.civicimpulse.com

        "Yields falsehood when preceded by its quotation! Yields
        falsehood when preceded by its quotation!" Achilles to
        Tortoise (in "Godel, Escher, Bach" by Douglas Hofstadter)

        On 05/31/2011 11:29 AM, codekiln wrote:
        > I found out about the govtrack database from my post [1] over at the google group for Sunlight Labs' API. As I mentioned there, I have a list of CRP ID / bioguide id / name triples. I would like to
        > assemble information on what committees each legislator was a member
        > of, and any special titles they held in that committee, such as
        > chairman or ranking member. Mr. Tauberer suggested I use govtrack's people.xml [2] and committees.xml [3].
        >
        > I am helping a professor assemble this data, so it is important for me to be able to explain the paper trail of the data. I have seen Govtrack's "Source Data" page [4], but I still have some questions.
        >
        > 1.) What is the source for committees.xml? Is there a page like people.xml's "Source Data" page for committees.xml on Govtrack?
        >
        > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the same data available for other years?
        >
        > 3.) The "Source Data" page says that people.xml "has been put together from a variety of sources and is maintained by hand." What are the sources?
        >
        > Thanks,
        > Peter
        >
        > [1] http://groups.google.com/group/sunlightlabs/browse_thread/thread/7f35405ebf184e0e
        >
        > [2] http://www.govtrack.us/data/us/people.xml
        >
        > [3] http://www.govtrack.us/data/us/112/committees.xml
        >
        > [4] http://www.govtrack.us/developers/data.xpd
        >
        >
        >
        > ------------------------------------
        >
        > Yahoo! Groups Links
        >
        >
        >
      • Derek Willis
        In constructing committee membership data for the NYT Congress API, I used data found via Charles Stewart s site:
        Message 3 of 8 , May 31, 2011
          In constructing committee membership data for the NYT Congress API, I used data found via Charles Stewart's site:


          As Josh says, older data contains errors in it - I recall incorrect party affiliations and phantom committee assignments when I checked them against official sources. So much so that our API only vouches for 110-112th committee data...

          Derek



          On Tue, May 31, 2011 at 12:18 PM, Josh Tauberer <tauberer@...> wrote:
           

          Hi, Peter.

          > 1.) What is the source for committees.xml? Is there a page like
          people.xml's "Source Data" page for committees.xml on Govtrack?

          The committee data is automatically scraped from

          http://www.senate.gov/pagelayout/committees/b_three_sections_with_teasers/membership.htm

          and

          http://clerk.house.gov/committee_info/index.html

          (As I mentioned on the labs list, the file was last generated a few
          months ago- March 3. I can re-generate it to get the latest info.)

          > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the
          same data available for other years?

          No I wasn't collecting that info then, and archival data is not
          available from the House/Senate (at least in the form I scrape).

          > 3.) The "Source Data" page says that people.xml "has been put
          together from a variety of sources and is maintained by hand." What are
          the sources?

          A full list of data sources is at govtrack.us/credits.xpd. That said,
          the amount of data I used from each source and the quality of the
          sources varied a lot. For information from about 2003 and on, the info
          about Members of Congress has been entered by hand by me.

          As you go further back in time, the quality of party affiliations,
          district assignments, and links to other IDs (e.g. ICPSR) grows worse.
          But it's probably the best anyone has anyway.

          Good luck. If you manage to put together a database of additional
          information, I hope you'll share it.

          - Josh Tauberer
          - CivicImpulse

          http://razor.occams.info
          http://www.civicimpulse.com

          "Yields falsehood when preceded by its quotation! Yields
          falsehood when preceded by its quotation!" Achilles to
          Tortoise (in "Godel, Escher, Bach" by Douglas Hofstadter)

          On 05/31/2011 11:29 AM, codekiln wrote:
          > I found out about the govtrack database from my post [1] over at the google group for Sunlight Labs' API. As I mentioned there, I have a list of CRP ID / bioguide id / name triples. I would like to
          > assemble information on what committees each legislator was a member
          > of, and any special titles they held in that committee, such as
          > chairman or ranking member. Mr. Tauberer suggested I use govtrack's people.xml [2] and committees.xml [3].
          >
          > I am helping a professor assemble this data, so it is important for me to be able to explain the paper trail of the data. I have seen Govtrack's "Source Data" page [4], but I still have some questions.
          >
          > 1.) What is the source for committees.xml? Is there a page like people.xml's "Source Data" page for committees.xml on Govtrack?
          >
          > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the same data available for other years?
          >
          > 3.) The "Source Data" page says that people.xml "has been put together from a variety of sources and is maintained by hand." What are the sources?
          >
          > Thanks,
          > Peter
          >
          > [1] http://groups.google.com/group/sunlightlabs/browse_thread/thread/7f35405ebf184e0e
          >
          > [2] http://www.govtrack.us/data/us/people.xml
          >
          > [3] http://www.govtrack.us/data/us/112/committees.xml
          >
          > [4] http://www.govtrack.us/developers/data.xpd
          >
          >
          >
          > ------------------------------------
          >
          > Yahoo! Groups Links
          >
          >
          >


        • codekiln
          Darek and Josh, Thanks for the citations and the info. Using people.xml I was able to extract ICPSRID for 165 out of the 793 identities I am researching; I ve
          Message 4 of 8 , May 31, 2011
            Darek and Josh,
            Thanks for the citations and the info. Using people.xml I was able to extract ICPSRID for 165 out of the 793 identities I am researching; I've put the results below for google [2].

            It seems as though the four or five data sources I have located are not always consistent on the committee names. Darek, what official sources did you use to double check Stewart's data? I had actually wondered whether there were phantom committees in Stewart's data because the committee name capitalization is inconsistent and the assignments lack subcommittees.

            Thanks for tipping me off to your work at NYT Congress API; I noticed the "data for earlier Congresses will be available soon" note in the committee call documentation [1].

            What a lively digital community there is around politician data; it seems like every few months a new API appears online.

            Thanks,
            Peter

            [1] http://developer.nytimes.com/docs/congress_api#h3-committees
            [2]
            OSID|CRPNAME|govtrack_icpsrid
            N00000010|"Burton, Dan"|15014
            N00000048|"Hefley, Joel"|15419
            N00000153|"Neal, Richard E"|15616
            N00000245|"Kerry, John"|14920
            N00000270|"Markey, Edward J"|14435
            N00000275|"Frank, Barney"|14824
            N00000308|"Kennedy, Edward M"|10808
            N00000444|"Gregg, Judd"|14826
            N00000480|"Snowe, Olympia J"|14661
            N00000534|"Jeffords, James M"|14240
            N00000561|"Johnson, Nancy L"|15028
            N00000581|"Dodd, Christopher J"|14213
            N00000616|"Lieberman, Joe"|15704
            N00000652|"Shays, Christopher"|15449
            N00000659|"Lautenberg, Frank R"|14914
            N00000716|"Payne, Donald M"|15619
            N00000781|"Pallone, Frank Jr"|15454
            N00000834|"Saxton, Jim"|15112
            N00000964|"Rangel, Charles B"|13035
            N00001003|"Engel, Eliot L"|15603
            N00001024|"Lowey, Nita M"|15612
            N00001082|"Towns, Edolphus"|15072
            N00001093|"Schumer, Charles E"|14858
            N00001143|"Ackerman, Gary"|15000
            N00001214|"McNulty, Michael R"|15614
            N00001261|"Walsh, James T"|15630
            N00001267|"Boehlert, Sherwood"|15007
            N00001311|"Slaughter, Louise M"|15444
            N00001329|"Houghton, Amo"|15423
            N00001408|"Murtha, John P"|14072
            N00001509|"Kanjorski, Paul E"|15104
            N00001535|"Weldon, Curt"|15447
            N00001604|"Specter, Arlen"|14910
            N00001669|"Biden, Joseph R Jr"|14101
            N00001685|"Rockefeller, Jay"|14922
            N00001691|"Levin, Carl"|14709
            N00001701|"Mineta, Norman Y."|14257
            N00001758|"Grassley, Chuck"|14226
            N00001762|"Inouye, Daniel K"|4812
            N00001764|"Lugar, Richard G"|14506
            N00001783|"Dingell, John D"|2605
            N00001806|"Oberstar, James L"|14265
            N00001811|"Smith, Lamar"|15445
            N00001813|"Serrano, Jose E"|29134
            N00001817|"Young, C W Bill"|13047
            N00001821|"Hoyer, Steny H"|14873
            N00001861|"Waxman, Henry A"|14280
            N00001945|"Mikulski, Barbara A"|14440
            N00001955|"Cardin, Ben"|15408
            N00001979|"Sarbanes, Paul S"|13039
            N00002061|"Warner, John W"|14712
            N00002073|"Wolf, Frank R"|14869
            N00002091|"Craig, Larry"|14809
            N00002171|"Boucher, Rick"|15010
            N00002198|"Rahall, Nick"|14448
            N00002200|"Byrd, Robert C"|1366
            N00002214|"Mollohan, Alan B"|15083
            N00002247|"Coble, Howard"|15092
            N00002260|"Price, David"|15438
            N00002377|"Ballenger, Cass"|15402
            N00002423|"Hollings, Fritz"|11204
            N00002492|"Spratt, John M Jr"|15064
            N00002577|"Lewis, John"|15431
            N00002742|"Graham, Bob"|15503
            N00002782|"Stearns, Cliff"|15627
            N00002858|"Ros-Lehtinen, Ileana"|15634
            N00002877|"Shaw, E Clay Jr"|14860
            N00002982|"Bilirakis, Michael"|15006
            N00003126|"Gordon, Bart"|15100
            N00003132|"Cooper, Jim"|15019
            N00003209|"Duncan, John J Jr"|15455
            N00003254|"Tanner, John"|15628
            N00003328|"Cochran, Thad"|14009
            N00003329|"Lott, Trent"|14031
            N00003350|"Taylor, Gene"|15637
            N00003389|"McConnell, Mitch"|14921
            N00003437|"Bunning, Jim"|15406
            N00003473|"Rogers, Hal"|14854
            N00003522|"Kaptur, Marcy"|15029
            N00003651|"Regula, Ralph"|14045
            N00003660|"Gillmor, Paul E"|15604
            N00003709|"DeWine, Mike"|15020
            N00003736|"Oxley, Michael G"|14875
            N00003813|"Visclosky, Pete"|15124
            N00003950|"Levin, Sander"|15033
            N00004029|"Conyers, John Jr"|10713
            N00004070|"Kildee, Dale E"|14430
            N00004133|"Upton, Fred"|15446
            N00004207|"Harkin, Tom"|14230
            N00004280|"Leach, Jim"|14432
            N00004291|"Sensenbrenner, F James Jr"|14657
            N00004309|"Kohl, Herb"|15703
            N00004330|"Kleczka, Jerry"|15082
            N00004394|"Obey, David R"|12036
            N00004426|"Petri, Tom"|14675
            N00004489|"Sabo, Martin Olav"|14656
            N00004583|"Daschle, Tom"|14617
            N00004613|"Conrad, Kent"|15502
            N00004615|"Dorgan, Byron L"|14812
            N00004638|"Burns, Conrad"|15701
            N00004643|"Baucus, Max"|14203
            N00004698|"Crane, Phil"|12041
            N00004702|"Hyde, Henry J"|14239
            N00004781|"Hastert, Dennis"|15417
            N00004856|"Lipinski, Bill"|15036
            N00004912|"Evans, Lane"|15023
            N00004956|"Costello, Jerry F"|15453
            N00004981|"Durbin, Dick"|15021
            N00005037|"Gephardt, Richard A"|14421
            N00005105|"Skelton, Ike"|14451
            N00005178|"Bond, Christopher S 'Kit'"|15501
            N00005285|"Roberts, Pat"|14852
            N00005331|"Bereuter, Doug"|14605
            N00005372|"Tauzin, Billy"|14679
            N00005385|"Breaux, John"|13056
            N00005407|"Baker, Richard"|15401
            N00005414|"McCrery, Jim"|15451
            N00005582|"Inhofe, James M"|15424
            N00005617|"Nickles, Don"|14908
            N00005645|"Hall, Ralph M"|14828
            N00005656|"Barton, Joe"|15085
            N00005677|"Frost, Martin"|14626
            N00005892|"DeLay, Tom"|15094
            N00005906|"Paul, Ron"|14290
            N00005998|"Ortiz, Solomon P"|15049
            N00006060|"Stenholm, Charles W"|14664
            N00006202|"Campbell, Ben Nighthorse"|15407
            N00006237|"Cheney, Dick"|14611
            N00006246|"Thomas, Craig"|15633
            N00006406|"Kyl, Jon"|15429
            N00006424|"McCain, John"|15039
            N00006486|"Kolbe, Jim"|15105
            N00006515|"Domenici, Pete V"|14103
            N00006518|"Bingaman, Jeff"|14912
            N00006692|"Boxer, Barbara"|15011
            N00006932|"Dreier, David"|14813
            N00006983|"Hunter, Duncan"|14835
            N00007087|"Lewis, Jerry"|14644
            N00007124|"Cox, Christopher"|15601
            N00007151|"Rohrabacher, Dana"|15621
            N00007231|"Gallegly, Elton"|15413
            N00007360|"Pelosi, Nancy"|15448
            N00007382|"Lantos, Tom"|14837
            N00007390|"Miller, George"|14256
            N00007397|"Stark, Pete"|14053
            N00007584|"Herger, Wally"|15420
            N00007653|"Akaka, Daniel K"|14400
            N00007665|"Abercrombie, Neil"|15245
            N00007724|"Wyden, Ron"|14871
            N00007781|"DeFazio, Peter"|15410
            N00007918|"Dicks, Norm"|14413
            N00007997|"Stevens, Ted"|12109
            N00007999|"Young, Don"|14066
            N00008094|"Berman, Howard L"|15005
            N00009816|"Smith, Chris"|14863
            N00009829|"McDermott, Jim"|15613
            N00009869|"Hatch, Orrin G"|14503
            N00009918|"Leahy, Patrick"|14307
            N00009920|"Shelby, Richard C"|14659
            N00009922|"Reid, Harry"|15054
            N00009926|"Nelson, Bill"|14651
            N00010084|"Johnson, Tim"|15425
            N00011971|"Lungren, Dan"|14647
            N00012508|"Carper, Tom"|15015
            N99999981|"Rumsfeld, Donald H."|10622

            --- In govtrack@yahoogroups.com, Derek Willis <dwillis@...> wrote:
            >
            > In constructing committee membership data for the NYT Congress API, I used
            > data found via Charles Stewart's site:
            >
            > http://web.mit.edu/17.251/www/data_page.html#2
            >
            > As Josh says, older data contains errors in it - I recall incorrect party
            > affiliations and phantom committee assignments when I checked them against
            > official sources. So much so that our API only vouches for 110-112th
            > committee data...
            >
            > Derek
            >
            >
            >
            > On Tue, May 31, 2011 at 12:18 PM, Josh Tauberer <tauberer@...>wrote:
            >
            > >
            > >
            > > Hi, Peter.
            > >
            > > > 1.) What is the source for committees.xml? Is there a page like
            > > people.xml's "Source Data" page for committees.xml on Govtrack?
            > >
            > > The committee data is automatically scraped from
            > >
            > >
            > > http://www.senate.gov/pagelayout/committees/b_three_sections_with_teasers/membership.htm
            > >
            > > and
            > >
            > > http://clerk.house.gov/committee_info/index.html
            > >
            > > (As I mentioned on the labs list, the file was last generated a few
            > > months ago- March 3. I can re-generate it to get the latest info.)
            > >
            > > > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the
            > > same data available for other years?
            > >
            > > No I wasn't collecting that info then, and archival data is not
            > > available from the House/Senate (at least in the form I scrape).
            > >
            > > > 3.) The "Source Data" page says that people.xml "has been put
            > > together from a variety of sources and is maintained by hand." What are
            > > the sources?
            > >
            > > A full list of data sources is at govtrack.us/credits.xpd. That said,
            > > the amount of data I used from each source and the quality of the
            > > sources varied a lot. For information from about 2003 and on, the info
            > > about Members of Congress has been entered by hand by me.
            > >
            > > As you go further back in time, the quality of party affiliations,
            > > district assignments, and links to other IDs (e.g. ICPSR) grows worse.
            > > But it's probably the best anyone has anyway.
            > >
            > > Good luck. If you manage to put together a database of additional
            > > information, I hope you'll share it.
            > >
            > > - Josh Tauberer
            > > - CivicImpulse
            > >
            > > http://razor.occams.info
            > > http://www.civicimpulse.com
            > >
            > > "Yields falsehood when preceded by its quotation! Yields
            > > falsehood when preceded by its quotation!" Achilles to
            > > Tortoise (in "Godel, Escher, Bach" by Douglas Hofstadter)
            > >
            > > On 05/31/2011 11:29 AM, codekiln wrote:
            > > > I found out about the govtrack database from my post [1] over at the
            > > google group for Sunlight Labs' API. As I mentioned there, I have a list of
            > > CRP ID / bioguide id / name triples. I would like to
            > > > assemble information on what committees each legislator was a member
            > > > of, and any special titles they held in that committee, such as
            > > > chairman or ranking member. Mr. Tauberer suggested I use govtrack's
            > > people.xml [2] and committees.xml [3].
            > > >
            > > > I am helping a professor assemble this data, so it is important for me to
            > > be able to explain the paper trail of the data. I have seen Govtrack's
            > > "Source Data" page [4], but I still have some questions.
            > > >
            > > > 1.) What is the source for committees.xml? Is there a page like
            > > people.xml's "Source Data" page for committees.xml on Govtrack?
            > > >
            > > > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the same
            > > data available for other years?
            > > >
            > > > 3.) The "Source Data" page says that people.xml "has been put together
            > > from a variety of sources and is maintained by hand." What are the sources?
            > > >
            > > > Thanks,
            > > > Peter
            > > >
            > > > [1]
            > > http://groups.google.com/group/sunlightlabs/browse_thread/thread/7f35405ebf184e0e
            > > >
            > > > [2] http://www.govtrack.us/data/us/people.xml
            > > >
            > > > [3] http://www.govtrack.us/data/us/112/committees.xml
            > > >
            > > > [4] http://www.govtrack.us/developers/data.xpd
            > > >
            > > >
            > > >
            > > > ------------------------------------
            > > >
            > > > Yahoo! Groups Links
            > > >
            > > >
            > > >
            > >
            > >
            >
          • Derek Willis
            Peter, We have ICPSRIDs for more than 2,600 members - if you think that would be a valuable addition to the API, I m happy to add that to the members response.
            Message 5 of 8 , May 31, 2011
              Peter,

              We have ICPSRIDs for more than 2,600 members - if you think that would be a valuable addition to the API, I'm happy to add that to the members response. The committee names are tough because they can change from congress to congress. So what was the House Banking Committee is now the House Financial Services Committee. Depending on which party is in the majority, the House Education and Labor Committee becomes the House Education and the Workforce Committee, and so on. For my money, the official source is Congress itself, which means looking up committee assignments in the Record, among other methods. I haven't found phantom committees in Stewart's data, but rather phantom assignments in which a member is recorded as a member of a committee when official records do not reflect such an assignment.

              Derek



              On Tue, May 31, 2011 at 1:30 PM, codekiln <ptr.nore@...> wrote:
               



              Darek and Josh,
              Thanks for the citations and the info. Using people.xml I was able to extract ICPSRID for 165 out of the 793 identities I am researching; I've put the results below for google [2].

              It seems as though the four or five data sources I have located are not always consistent on the committee names. Darek, what official sources did you use to double check Stewart's data? I had actually wondered whether there were phantom committees in Stewart's data because the committee name capitalization is inconsistent and the assignments lack subcommittees.

              Thanks for tipping me off to your work at NYT Congress API; I noticed the "data for earlier Congresses will be available soon" note in the committee call documentation [1].

              What a lively digital community there is around politician data; it seems like every few months a new API appears online.

              Thanks,
              Peter

              [1] http://developer.nytimes.com/docs/congress_api#h3-committees
              [2]
              OSID|CRPNAME|govtrack_icpsrid
              N00000010|"Burton, Dan"|15014
              N00000048|"Hefley, Joel"|15419
              N00000153|"Neal, Richard E"|15616
              N00000245|"Kerry, John"|14920
              N00000270|"Markey, Edward J"|14435
              N00000275|"Frank, Barney"|14824
              N00000308|"Kennedy, Edward M"|10808
              N00000444|"Gregg, Judd"|14826
              N00000480|"Snowe, Olympia J"|14661
              N00000534|"Jeffords, James M"|14240
              N00000561|"Johnson, Nancy L"|15028
              N00000581|"Dodd, Christopher J"|14213
              N00000616|"Lieberman, Joe"|15704
              N00000652|"Shays, Christopher"|15449
              N00000659|"Lautenberg, Frank R"|14914
              N00000716|"Payne, Donald M"|15619
              N00000781|"Pallone, Frank Jr"|15454
              N00000834|"Saxton, Jim"|15112
              N00000964|"Rangel, Charles B"|13035
              N00001003|"Engel, Eliot L"|15603
              N00001024|"Lowey, Nita M"|15612
              N00001082|"Towns, Edolphus"|15072
              N00001093|"Schumer, Charles E"|14858
              N00001143|"Ackerman, Gary"|15000
              N00001214|"McNulty, Michael R"|15614
              N00001261|"Walsh, James T"|15630
              N00001267|"Boehlert, Sherwood"|15007
              N00001311|"Slaughter, Louise M"|15444
              N00001329|"Houghton, Amo"|15423
              N00001408|"Murtha, John P"|14072
              N00001509|"Kanjorski, Paul E"|15104
              N00001535|"Weldon, Curt"|15447
              N00001604|"Specter, Arlen"|14910
              N00001669|"Biden, Joseph R Jr"|14101
              N00001685|"Rockefeller, Jay"|14922
              N00001691|"Levin, Carl"|14709
              N00001701|"Mineta, Norman Y."|14257
              N00001758|"Grassley, Chuck"|14226
              N00001762|"Inouye, Daniel K"|4812
              N00001764|"Lugar, Richard G"|14506
              N00001783|"Dingell, John D"|2605
              N00001806|"Oberstar, James L"|14265
              N00001811|"Smith, Lamar"|15445
              N00001813|"Serrano, Jose E"|29134
              N00001817|"Young, C W Bill"|13047
              N00001821|"Hoyer, Steny H"|14873
              N00001861|"Waxman, Henry A"|14280
              N00001945|"Mikulski, Barbara A"|14440
              N00001955|"Cardin, Ben"|15408
              N00001979|"Sarbanes, Paul S"|13039
              N00002061|"Warner, John W"|14712
              N00002073|"Wolf, Frank R"|14869
              N00002091|"Craig, Larry"|14809
              N00002171|"Boucher, Rick"|15010
              N00002198|"Rahall, Nick"|14448
              N00002200|"Byrd, Robert C"|1366
              N00002214|"Mollohan, Alan B"|15083
              N00002247|"Coble, Howard"|15092
              N00002260|"Price, David"|15438
              N00002377|"Ballenger, Cass"|15402
              N00002423|"Hollings, Fritz"|11204
              N00002492|"Spratt, John M Jr"|15064
              N00002577|"Lewis, John"|15431
              N00002742|"Graham, Bob"|15503
              N00002782|"Stearns, Cliff"|15627
              N00002858|"Ros-Lehtinen, Ileana"|15634
              N00002877|"Shaw, E Clay Jr"|14860
              N00002982|"Bilirakis, Michael"|15006
              N00003126|"Gordon, Bart"|15100
              N00003132|"Cooper, Jim"|15019
              N00003209|"Duncan, John J Jr"|15455
              N00003254|"Tanner, John"|15628
              N00003328|"Cochran, Thad"|14009
              N00003329|"Lott, Trent"|14031
              N00003350|"Taylor, Gene"|15637
              N00003389|"McConnell, Mitch"|14921
              N00003437|"Bunning, Jim"|15406
              N00003473|"Rogers, Hal"|14854
              N00003522|"Kaptur, Marcy"|15029
              N00003651|"Regula, Ralph"|14045
              N00003660|"Gillmor, Paul E"|15604
              N00003709|"DeWine, Mike"|15020
              N00003736|"Oxley, Michael G"|14875
              N00003813|"Visclosky, Pete"|15124
              N00003950|"Levin, Sander"|15033
              N00004029|"Conyers, John Jr"|10713
              N00004070|"Kildee, Dale E"|14430
              N00004133|"Upton, Fred"|15446
              N00004207|"Harkin, Tom"|14230
              N00004280|"Leach, Jim"|14432
              N00004291|"Sensenbrenner, F James Jr"|14657
              N00004309|"Kohl, Herb"|15703
              N00004330|"Kleczka, Jerry"|15082
              N00004394|"Obey, David R"|12036
              N00004426|"Petri, Tom"|14675
              N00004489|"Sabo, Martin Olav"|14656
              N00004583|"Daschle, Tom"|14617
              N00004613|"Conrad, Kent"|15502
              N00004615|"Dorgan, Byron L"|14812
              N00004638|"Burns, Conrad"|15701
              N00004643|"Baucus, Max"|14203
              N00004698|"Crane, Phil"|12041
              N00004702|"Hyde, Henry J"|14239
              N00004781|"Hastert, Dennis"|15417
              N00004856|"Lipinski, Bill"|15036
              N00004912|"Evans, Lane"|15023
              N00004956|"Costello, Jerry F"|15453
              N00004981|"Durbin, Dick"|15021
              N00005037|"Gephardt, Richard A"|14421
              N00005105|"Skelton, Ike"|14451
              N00005178|"Bond, Christopher S 'Kit'"|15501
              N00005285|"Roberts, Pat"|14852
              N00005331|"Bereuter, Doug"|14605
              N00005372|"Tauzin, Billy"|14679
              N00005385|"Breaux, John"|13056
              N00005407|"Baker, Richard"|15401
              N00005414|"McCrery, Jim"|15451
              N00005582|"Inhofe, James M"|15424
              N00005617|"Nickles, Don"|14908
              N00005645|"Hall, Ralph M"|14828
              N00005656|"Barton, Joe"|15085
              N00005677|"Frost, Martin"|14626
              N00005892|"DeLay, Tom"|15094
              N00005906|"Paul, Ron"|14290
              N00005998|"Ortiz, Solomon P"|15049
              N00006060|"Stenholm, Charles W"|14664
              N00006202|"Campbell, Ben Nighthorse"|15407
              N00006237|"Cheney, Dick"|14611
              N00006246|"Thomas, Craig"|15633
              N00006406|"Kyl, Jon"|15429
              N00006424|"McCain, John"|15039
              N00006486|"Kolbe, Jim"|15105
              N00006515|"Domenici, Pete V"|14103
              N00006518|"Bingaman, Jeff"|14912
              N00006692|"Boxer, Barbara"|15011
              N00006932|"Dreier, David"|14813
              N00006983|"Hunter, Duncan"|14835
              N00007087|"Lewis, Jerry"|14644
              N00007124|"Cox, Christopher"|15601
              N00007151|"Rohrabacher, Dana"|15621
              N00007231|"Gallegly, Elton"|15413
              N00007360|"Pelosi, Nancy"|15448
              N00007382|"Lantos, Tom"|14837
              N00007390|"Miller, George"|14256
              N00007397|"Stark, Pete"|14053
              N00007584|"Herger, Wally"|15420
              N00007653|"Akaka, Daniel K"|14400
              N00007665|"Abercrombie, Neil"|15245
              N00007724|"Wyden, Ron"|14871
              N00007781|"DeFazio, Peter"|15410
              N00007918|"Dicks, Norm"|14413
              N00007997|"Stevens, Ted"|12109
              N00007999|"Young, Don"|14066
              N00008094|"Berman, Howard L"|15005
              N00009816|"Smith, Chris"|14863
              N00009829|"McDermott, Jim"|15613
              N00009869|"Hatch, Orrin G"|14503
              N00009918|"Leahy, Patrick"|14307
              N00009920|"Shelby, Richard C"|14659
              N00009922|"Reid, Harry"|15054
              N00009926|"Nelson, Bill"|14651
              N00010084|"Johnson, Tim"|15425
              N00011971|"Lungren, Dan"|14647
              N00012508|"Carper, Tom"|15015
              N99999981|"Rumsfeld, Donald H."|10622



              --- In govtrack@yahoogroups.com, Derek Willis <dwillis@...> wrote:
              >
              > In constructing committee membership data for the NYT Congress API, I used
              > data found via Charles Stewart's site:
              >
              > http://web.mit.edu/17.251/www/data_page.html#2
              >
              > As Josh says, older data contains errors in it - I recall incorrect party
              > affiliations and phantom committee assignments when I checked them against
              > official sources. So much so that our API only vouches for 110-112th
              > committee data...
              >
              > Derek
              >
              >
              >
              > On Tue, May 31, 2011 at 12:18 PM, Josh Tauberer <tauberer@...>wrote:

              >
              > >
              > >
              > > Hi, Peter.
              > >
              > > > 1.) What is the source for committees.xml? Is there a page like
              > > people.xml's "Source Data" page for committees.xml on Govtrack?
              > >
              > > The committee data is automatically scraped from
              > >
              > >
              > > http://www.senate.gov/pagelayout/committees/b_three_sections_with_teasers/membership.htm
              > >
              > > and
              > >
              > > http://clerk.house.gov/committee_info/index.html
              > >
              > > (As I mentioned on the labs list, the file was last generated a few
              > > months ago- March 3. I can re-generate it to get the latest info.)
              > >
              > > > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the
              > > same data available for other years?
              > >
              > > No I wasn't collecting that info then, and archival data is not
              > > available from the House/Senate (at least in the form I scrape).
              > >
              > > > 3.) The "Source Data" page says that people.xml "has been put
              > > together from a variety of sources and is maintained by hand." What are
              > > the sources?
              > >
              > > A full list of data sources is at govtrack.us/credits.xpd. That said,
              > > the amount of data I used from each source and the quality of the
              > > sources varied a lot. For information from about 2003 and on, the info
              > > about Members of Congress has been entered by hand by me.
              > >
              > > As you go further back in time, the quality of party affiliations,
              > > district assignments, and links to other IDs (e.g. ICPSR) grows worse.
              > > But it's probably the best anyone has anyway.
              > >
              > > Good luck. If you manage to put together a database of additional
              > > information, I hope you'll share it.
              > >
              > > - Josh Tauberer
              > > - CivicImpulse
              > >
              > > http://razor.occams.info
              > > http://www.civicimpulse.com
              > >
              > > "Yields falsehood when preceded by its quotation! Yields
              > > falsehood when preceded by its quotation!" Achilles to
              > > Tortoise (in "Godel, Escher, Bach" by Douglas Hofstadter)
              > >
              > > On 05/31/2011 11:29 AM, codekiln wrote:
              > > > I found out about the govtrack database from my post [1] over at the
              > > google group for Sunlight Labs' API. As I mentioned there, I have a list of
              > > CRP ID / bioguide id / name triples. I would like to
              > > > assemble information on what committees each legislator was a member
              > > > of, and any special titles they held in that committee, such as
              > > > chairman or ranking member. Mr. Tauberer suggested I use govtrack's
              > > people.xml [2] and committees.xml [3].
              > > >
              > > > I am helping a professor assemble this data, so it is important for me to
              > > be able to explain the paper trail of the data. I have seen Govtrack's
              > > "Source Data" page [4], but I still have some questions.
              > > >
              > > > 1.) What is the source for committees.xml? Is there a page like
              > > people.xml's "Source Data" page for committees.xml on Govtrack?
              > > >
              > > > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the same
              > > data available for other years?
              > > >
              > > > 3.) The "Source Data" page says that people.xml "has been put together
              > > from a variety of sources and is maintained by hand." What are the sources?
              > > >
              > > > Thanks,
              > > > Peter
              > > >
              > > > [1]
              > > http://groups.google.com/group/sunlightlabs/browse_thread/thread/7f35405ebf184e0e
              > > >
              > > > [2] http://www.govtrack.us/data/us/people.xml
              > > >
              > > > [3] http://www.govtrack.us/data/us/112/committees.xml
              > > >
              > > > [4] http://www.govtrack.us/developers/data.xpd
              > > >
              > > >
              > > >
              > > > ------------------------------------
              > > >
              > > > Yahoo! Groups Links
              > > >
              > > >
              > > >
              > >
              > >
              >


            • codekiln
              Hi Darek, Thanks for your detailed response. Your record of ICPSRIDs would be a valuable to the project I am working on, but I have no idea if ICPSRIDs belong
              Message 6 of 8 , Jun 2, 2011
                Hi Darek,
                Thanks for your detailed response.

                Your record of ICPSRIDs would be a valuable to the project I am working on, but I have no idea if ICPSRIDs belong in the NYT Congress API. Netizens definitely have a need some kind of database id Rosetta Stone. Bioguide ID - ICPSRID isn't the only important connection. When I have assembled all the different corresponding ids I will ask the professor if I may publish the cross-references I have found.

                Thanks,
                Peter

                --- In govtrack@yahoogroups.com, Derek Willis <dwillis@...> wrote:
                >
                > Peter,
                >
                > We have ICPSRIDs for more than 2,600 members - if you think that would be a
                > valuable addition to the API, I'm happy to add that to the members response.
                > The committee names are tough because they can change from congress to
                > congress. So what was the House Banking Committee is now the House Financial
                > Services Committee. Depending on which party is in the majority, the House
                > Education and Labor Committee becomes the House Education and the Workforce
                > Committee, and so on. For my money, the official source is Congress itself,
                > which means looking up committee assignments in the Record, among other
                > methods. I haven't found phantom committees in Stewart's data, but rather
                > phantom assignments in which a member is recorded as a member of a committee
                > when official records do not reflect such an assignment.
                >
                > Derek
                >
                >
                >
                > On Tue, May 31, 2011 at 1:30 PM, codekiln <ptr.nore@...> wrote:
                >
                > >
                > >
                > >
                > >
                > > Darek and Josh,
                > > Thanks for the citations and the info. Using people.xml I was able to
                > > extract ICPSRID for 165 out of the 793 identities I am researching; I've put
                > > the results below for google [2].
                > >
                > > It seems as though the four or five data sources I have located are not
                > > always consistent on the committee names. Darek, what official sources did
                > > you use to double check Stewart's data? I had actually wondered whether
                > > there were phantom committees in Stewart's data because the committee name
                > > capitalization is inconsistent and the assignments lack subcommittees.
                > >
                > > Thanks for tipping me off to your work at NYT Congress API; I noticed the
                > > "data for earlier Congresses will be available soon" note in the committee
                > > call documentation [1].
                > >
                > > What a lively digital community there is around politician data; it seems
                > > like every few months a new API appears online.
                > >
                > > Thanks,
                > > Peter
                > >
                > > [1] http://developer.nytimes.com/docs/congress_api#h3-committees
                > > [2]
                > > OSID|CRPNAME|govtrack_icpsrid
                > > N00000010|"Burton, Dan"|15014
                > > N00000048|"Hefley, Joel"|15419
                > > N00000153|"Neal, Richard E"|15616
                > > N00000245|"Kerry, John"|14920
                > > N00000270|"Markey, Edward J"|14435
                > > N00000275|"Frank, Barney"|14824
                > > N00000308|"Kennedy, Edward M"|10808
                > > N00000444|"Gregg, Judd"|14826
                > > N00000480|"Snowe, Olympia J"|14661
                > > N00000534|"Jeffords, James M"|14240
                > > N00000561|"Johnson, Nancy L"|15028
                > > N00000581|"Dodd, Christopher J"|14213
                > > N00000616|"Lieberman, Joe"|15704
                > > N00000652|"Shays, Christopher"|15449
                > > N00000659|"Lautenberg, Frank R"|14914
                > > N00000716|"Payne, Donald M"|15619
                > > N00000781|"Pallone, Frank Jr"|15454
                > > N00000834|"Saxton, Jim"|15112
                > > N00000964|"Rangel, Charles B"|13035
                > > N00001003|"Engel, Eliot L"|15603
                > > N00001024|"Lowey, Nita M"|15612
                > > N00001082|"Towns, Edolphus"|15072
                > > N00001093|"Schumer, Charles E"|14858
                > > N00001143|"Ackerman, Gary"|15000
                > > N00001214|"McNulty, Michael R"|15614
                > > N00001261|"Walsh, James T"|15630
                > > N00001267|"Boehlert, Sherwood"|15007
                > > N00001311|"Slaughter, Louise M"|15444
                > > N00001329|"Houghton, Amo"|15423
                > > N00001408|"Murtha, John P"|14072
                > > N00001509|"Kanjorski, Paul E"|15104
                > > N00001535|"Weldon, Curt"|15447
                > > N00001604|"Specter, Arlen"|14910
                > > N00001669|"Biden, Joseph R Jr"|14101
                > > N00001685|"Rockefeller, Jay"|14922
                > > N00001691|"Levin, Carl"|14709
                > > N00001701|"Mineta, Norman Y."|14257
                > > N00001758|"Grassley, Chuck"|14226
                > > N00001762|"Inouye, Daniel K"|4812
                > > N00001764|"Lugar, Richard G"|14506
                > > N00001783|"Dingell, John D"|2605
                > > N00001806|"Oberstar, James L"|14265
                > > N00001811|"Smith, Lamar"|15445
                > > N00001813|"Serrano, Jose E"|29134
                > > N00001817|"Young, C W Bill"|13047
                > > N00001821|"Hoyer, Steny H"|14873
                > > N00001861|"Waxman, Henry A"|14280
                > > N00001945|"Mikulski, Barbara A"|14440
                > > N00001955|"Cardin, Ben"|15408
                > > N00001979|"Sarbanes, Paul S"|13039
                > > N00002061|"Warner, John W"|14712
                > > N00002073|"Wolf, Frank R"|14869
                > > N00002091|"Craig, Larry"|14809
                > > N00002171|"Boucher, Rick"|15010
                > > N00002198|"Rahall, Nick"|14448
                > > N00002200|"Byrd, Robert C"|1366
                > > N00002214|"Mollohan, Alan B"|15083
                > > N00002247|"Coble, Howard"|15092
                > > N00002260|"Price, David"|15438
                > > N00002377|"Ballenger, Cass"|15402
                > > N00002423|"Hollings, Fritz"|11204
                > > N00002492|"Spratt, John M Jr"|15064
                > > N00002577|"Lewis, John"|15431
                > > N00002742|"Graham, Bob"|15503
                > > N00002782|"Stearns, Cliff"|15627
                > > N00002858|"Ros-Lehtinen, Ileana"|15634
                > > N00002877|"Shaw, E Clay Jr"|14860
                > > N00002982|"Bilirakis, Michael"|15006
                > > N00003126|"Gordon, Bart"|15100
                > > N00003132|"Cooper, Jim"|15019
                > > N00003209|"Duncan, John J Jr"|15455
                > > N00003254|"Tanner, John"|15628
                > > N00003328|"Cochran, Thad"|14009
                > > N00003329|"Lott, Trent"|14031
                > > N00003350|"Taylor, Gene"|15637
                > > N00003389|"McConnell, Mitch"|14921
                > > N00003437|"Bunning, Jim"|15406
                > > N00003473|"Rogers, Hal"|14854
                > > N00003522|"Kaptur, Marcy"|15029
                > > N00003651|"Regula, Ralph"|14045
                > > N00003660|"Gillmor, Paul E"|15604
                > > N00003709|"DeWine, Mike"|15020
                > > N00003736|"Oxley, Michael G"|14875
                > > N00003813|"Visclosky, Pete"|15124
                > > N00003950|"Levin, Sander"|15033
                > > N00004029|"Conyers, John Jr"|10713
                > > N00004070|"Kildee, Dale E"|14430
                > > N00004133|"Upton, Fred"|15446
                > > N00004207|"Harkin, Tom"|14230
                > > N00004280|"Leach, Jim"|14432
                > > N00004291|"Sensenbrenner, F James Jr"|14657
                > > N00004309|"Kohl, Herb"|15703
                > > N00004330|"Kleczka, Jerry"|15082
                > > N00004394|"Obey, David R"|12036
                > > N00004426|"Petri, Tom"|14675
                > > N00004489|"Sabo, Martin Olav"|14656
                > > N00004583|"Daschle, Tom"|14617
                > > N00004613|"Conrad, Kent"|15502
                > > N00004615|"Dorgan, Byron L"|14812
                > > N00004638|"Burns, Conrad"|15701
                > > N00004643|"Baucus, Max"|14203
                > > N00004698|"Crane, Phil"|12041
                > > N00004702|"Hyde, Henry J"|14239
                > > N00004781|"Hastert, Dennis"|15417
                > > N00004856|"Lipinski, Bill"|15036
                > > N00004912|"Evans, Lane"|15023
                > > N00004956|"Costello, Jerry F"|15453
                > > N00004981|"Durbin, Dick"|15021
                > > N00005037|"Gephardt, Richard A"|14421
                > > N00005105|"Skelton, Ike"|14451
                > > N00005178|"Bond, Christopher S 'Kit'"|15501
                > > N00005285|"Roberts, Pat"|14852
                > > N00005331|"Bereuter, Doug"|14605
                > > N00005372|"Tauzin, Billy"|14679
                > > N00005385|"Breaux, John"|13056
                > > N00005407|"Baker, Richard"|15401
                > > N00005414|"McCrery, Jim"|15451
                > > N00005582|"Inhofe, James M"|15424
                > > N00005617|"Nickles, Don"|14908
                > > N00005645|"Hall, Ralph M"|14828
                > > N00005656|"Barton, Joe"|15085
                > > N00005677|"Frost, Martin"|14626
                > > N00005892|"DeLay, Tom"|15094
                > > N00005906|"Paul, Ron"|14290
                > > N00005998|"Ortiz, Solomon P"|15049
                > > N00006060|"Stenholm, Charles W"|14664
                > > N00006202|"Campbell, Ben Nighthorse"|15407
                > > N00006237|"Cheney, Dick"|14611
                > > N00006246|"Thomas, Craig"|15633
                > > N00006406|"Kyl, Jon"|15429
                > > N00006424|"McCain, John"|15039
                > > N00006486|"Kolbe, Jim"|15105
                > > N00006515|"Domenici, Pete V"|14103
                > > N00006518|"Bingaman, Jeff"|14912
                > > N00006692|"Boxer, Barbara"|15011
                > > N00006932|"Dreier, David"|14813
                > > N00006983|"Hunter, Duncan"|14835
                > > N00007087|"Lewis, Jerry"|14644
                > > N00007124|"Cox, Christopher"|15601
                > > N00007151|"Rohrabacher, Dana"|15621
                > > N00007231|"Gallegly, Elton"|15413
                > > N00007360|"Pelosi, Nancy"|15448
                > > N00007382|"Lantos, Tom"|14837
                > > N00007390|"Miller, George"|14256
                > > N00007397|"Stark, Pete"|14053
                > > N00007584|"Herger, Wally"|15420
                > > N00007653|"Akaka, Daniel K"|14400
                > > N00007665|"Abercrombie, Neil"|15245
                > > N00007724|"Wyden, Ron"|14871
                > > N00007781|"DeFazio, Peter"|15410
                > > N00007918|"Dicks, Norm"|14413
                > > N00007997|"Stevens, Ted"|12109
                > > N00007999|"Young, Don"|14066
                > > N00008094|"Berman, Howard L"|15005
                > > N00009816|"Smith, Chris"|14863
                > > N00009829|"McDermott, Jim"|15613
                > > N00009869|"Hatch, Orrin G"|14503
                > > N00009918|"Leahy, Patrick"|14307
                > > N00009920|"Shelby, Richard C"|14659
                > > N00009922|"Reid, Harry"|15054
                > > N00009926|"Nelson, Bill"|14651
                > > N00010084|"Johnson, Tim"|15425
                > > N00011971|"Lungren, Dan"|14647
                > > N00012508|"Carper, Tom"|15015
                > > N99999981|"Rumsfeld, Donald H."|10622
                > >
                > >
                > > --- In govtrack@yahoogroups.com, Derek Willis <dwillis@> wrote:
                > > >
                > > > In constructing committee membership data for the NYT Congress API, I
                > > used
                > > > data found via Charles Stewart's site:
                > > >
                > > > http://web.mit.edu/17.251/www/data_page.html#2
                > > >
                > > > As Josh says, older data contains errors in it - I recall incorrect party
                > > > affiliations and phantom committee assignments when I checked them
                > > against
                > > > official sources. So much so that our API only vouches for 110-112th
                > > > committee data...
                > > >
                > > > Derek
                > > >
                > > >
                > > >
                > > > On Tue, May 31, 2011 at 12:18 PM, Josh Tauberer <tauberer@>wrote:
                > >
                > > >
                > > > >
                > > > >
                > > > > Hi, Peter.
                > > > >
                > > > > > 1.) What is the source for committees.xml? Is there a page like
                > > > > people.xml's "Source Data" page for committees.xml on Govtrack?
                > > > >
                > > > > The committee data is automatically scraped from
                > > > >
                > > > >
                > > > >
                > > http://www.senate.gov/pagelayout/committees/b_three_sections_with_teasers/membership.htm
                > > > >
                > > > > and
                > > > >
                > > > > http://clerk.house.gov/committee_info/index.html
                > > > >
                > > > > (As I mentioned on the labs list, the file was last generated a few
                > > > > months ago- March 3. I can re-generate it to get the latest info.)
                > > > >
                > > > > > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the
                > > > > same data available for other years?
                > > > >
                > > > > No I wasn't collecting that info then, and archival data is not
                > > > > available from the House/Senate (at least in the form I scrape).
                > > > >
                > > > > > 3.) The "Source Data" page says that people.xml "has been put
                > > > > together from a variety of sources and is maintained by hand." What are
                > > > > the sources?
                > > > >
                > > > > A full list of data sources is at govtrack.us/credits.xpd. That said,
                > > > > the amount of data I used from each source and the quality of the
                > > > > sources varied a lot. For information from about 2003 and on, the info
                > > > > about Members of Congress has been entered by hand by me.
                > > > >
                > > > > As you go further back in time, the quality of party affiliations,
                > > > > district assignments, and links to other IDs (e.g. ICPSR) grows worse.
                > > > > But it's probably the best anyone has anyway.
                > > > >
                > > > > Good luck. If you manage to put together a database of additional
                > > > > information, I hope you'll share it.
                > > > >
                > > > > - Josh Tauberer
                > > > > - CivicImpulse
                > > > >
                > > > > http://razor.occams.info
                > > > > http://www.civicimpulse.com
                > > > >
                > > > > "Yields falsehood when preceded by its quotation! Yields
                > > > > falsehood when preceded by its quotation!" Achilles to
                > > > > Tortoise (in "Godel, Escher, Bach" by Douglas Hofstadter)
                > > > >
                > > > > On 05/31/2011 11:29 AM, codekiln wrote:
                > > > > > I found out about the govtrack database from my post [1] over at the
                > > > > google group for Sunlight Labs' API. As I mentioned there, I have a
                > > list of
                > > > > CRP ID / bioguide id / name triples. I would like to
                > > > > > assemble information on what committees each legislator was a member
                > > > > > of, and any special titles they held in that committee, such as
                > > > > > chairman or ranking member. Mr. Tauberer suggested I use govtrack's
                > > > > people.xml [2] and committees.xml [3].
                > > > > >
                > > > > > I am helping a professor assemble this data, so it is important for
                > > me to
                > > > > be able to explain the paper trail of the data. I have seen Govtrack's
                > > > > "Source Data" page [4], but I still have some questions.
                > > > > >
                > > > > > 1.) What is the source for committees.xml? Is there a page like
                > > > > people.xml's "Source Data" page for committees.xml on Govtrack?
                > > > > >
                > > > > > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the
                > > same
                > > > > data available for other years?
                > > > > >
                > > > > > 3.) The "Source Data" page says that people.xml "has been put
                > > together
                > > > > from a variety of sources and is maintained by hand." What are the
                > > sources?
                > > > > >
                > > > > > Thanks,
                > > > > > Peter
                > > > > >
                > > > > > [1]
                > > > >
                > > http://groups.google.com/group/sunlightlabs/browse_thread/thread/7f35405ebf184e0e
                > > > > >
                > > > > > [2] http://www.govtrack.us/data/us/people.xml
                > > > > >
                > > > > > [3] http://www.govtrack.us/data/us/112/committees.xml
                > > > > >
                > > > > > [4] http://www.govtrack.us/developers/data.xpd
                > > > > >
                > > > > >
                > > > > >
                > > > > > ------------------------------------
                > > > > >
                > > > > > Yahoo! Groups Links
                > > > > >
                > > > > >
                > > > > >
                > > > >
                > > > >
                > > >
                > >
                > >
                > >
                >
              • Derek Willis
                Hey Peter, I think we ll put them in anyway, but I think the bioguide ID should be the canonical reference. Derek
                Message 7 of 8 , Jun 2, 2011
                  Hey Peter,

                  I think we'll put them in anyway, but I think the bioguide ID should be the canonical reference.

                  Derek

                  On Thu, Jun 2, 2011 at 11:49 AM, codekiln <ptr.nore@...> wrote:
                   



                  Hi Darek,
                  Thanks for your detailed response.

                  Your record of ICPSRIDs would be a valuable to the project I am working on, but I have no idea if ICPSRIDs belong in the NYT Congress API. Netizens definitely have a need some kind of database id Rosetta Stone. Bioguide ID - ICPSRID isn't the only important connection. When I have assembled all the different corresponding ids I will ask the professor if I may publish the cross-references I have found.

                  Thanks,
                  Peter



                  --- In govtrack@yahoogroups.com, Derek Willis <dwillis@...> wrote:
                  >
                  > Peter,
                  >
                  > We have ICPSRIDs for more than 2,600 members - if you think that would be a
                  > valuable addition to the API, I'm happy to add that to the members response.
                  > The committee names are tough because they can change from congress to
                  > congress. So what was the House Banking Committee is now the House Financial
                  > Services Committee. Depending on which party is in the majority, the House
                  > Education and Labor Committee becomes the House Education and the Workforce
                  > Committee, and so on. For my money, the official source is Congress itself,
                  > which means looking up committee assignments in the Record, among other
                  > methods. I haven't found phantom committees in Stewart's data, but rather
                  > phantom assignments in which a member is recorded as a member of a committee
                  > when official records do not reflect such an assignment.
                  >
                  > Derek
                  >
                  >
                  >
                  > On Tue, May 31, 2011 at 1:30 PM, codekiln <ptr.nore@...> wrote:
                  >
                  > >
                  > >
                  > >
                  > >
                  > > Darek and Josh,
                  > > Thanks for the citations and the info. Using people.xml I was able to
                  > > extract ICPSRID for 165 out of the 793 identities I am researching; I've put
                  > > the results below for google [2].
                  > >
                  > > It seems as though the four or five data sources I have located are not
                  > > always consistent on the committee names. Darek, what official sources did
                  > > you use to double check Stewart's data? I had actually wondered whether
                  > > there were phantom committees in Stewart's data because the committee name
                  > > capitalization is inconsistent and the assignments lack subcommittees.
                  > >
                  > > Thanks for tipping me off to your work at NYT Congress API; I noticed the
                  > > "data for earlier Congresses will be available soon" note in the committee
                  > > call documentation [1].
                  > >
                  > > What a lively digital community there is around politician data; it seems
                  > > like every few months a new API appears online.
                  > >
                  > > Thanks,
                  > > Peter
                  > >
                  > > [1] http://developer.nytimes.com/docs/congress_api#h3-committees
                  > > [2]
                  > > OSID|CRPNAME|govtrack_icpsrid
                  > > N00000010|"Burton, Dan"|15014
                  > > N00000048|"Hefley, Joel"|15419
                  > > N00000153|"Neal, Richard E"|15616
                  > > N00000245|"Kerry, John"|14920
                  > > N00000270|"Markey, Edward J"|14435
                  > > N00000275|"Frank, Barney"|14824
                  > > N00000308|"Kennedy, Edward M"|10808
                  > > N00000444|"Gregg, Judd"|14826
                  > > N00000480|"Snowe, Olympia J"|14661
                  > > N00000534|"Jeffords, James M"|14240
                  > > N00000561|"Johnson, Nancy L"|15028
                  > > N00000581|"Dodd, Christopher J"|14213
                  > > N00000616|"Lieberman, Joe"|15704
                  > > N00000652|"Shays, Christopher"|15449
                  > > N00000659|"Lautenberg, Frank R"|14914
                  > > N00000716|"Payne, Donald M"|15619
                  > > N00000781|"Pallone, Frank Jr"|15454
                  > > N00000834|"Saxton, Jim"|15112
                  > > N00000964|"Rangel, Charles B"|13035
                  > > N00001003|"Engel, Eliot L"|15603
                  > > N00001024|"Lowey, Nita M"|15612
                  > > N00001082|"Towns, Edolphus"|15072
                  > > N00001093|"Schumer, Charles E"|14858
                  > > N00001143|"Ackerman, Gary"|15000
                  > > N00001214|"McNulty, Michael R"|15614
                  > > N00001261|"Walsh, James T"|15630
                  > > N00001267|"Boehlert, Sherwood"|15007
                  > > N00001311|"Slaughter, Louise M"|15444
                  > > N00001329|"Houghton, Amo"|15423
                  > > N00001408|"Murtha, John P"|14072
                  > > N00001509|"Kanjorski, Paul E"|15104
                  > > N00001535|"Weldon, Curt"|15447
                  > > N00001604|"Specter, Arlen"|14910
                  > > N00001669|"Biden, Joseph R Jr"|14101
                  > > N00001685|"Rockefeller, Jay"|14922
                  > > N00001691|"Levin, Carl"|14709
                  > > N00001701|"Mineta, Norman Y."|14257
                  > > N00001758|"Grassley, Chuck"|14226
                  > > N00001762|"Inouye, Daniel K"|4812
                  > > N00001764|"Lugar, Richard G"|14506
                  > > N00001783|"Dingell, John D"|2605
                  > > N00001806|"Oberstar, James L"|14265
                  > > N00001811|"Smith, Lamar"|15445
                  > > N00001813|"Serrano, Jose E"|29134
                  > > N00001817|"Young, C W Bill"|13047
                  > > N00001821|"Hoyer, Steny H"|14873
                  > > N00001861|"Waxman, Henry A"|14280
                  > > N00001945|"Mikulski, Barbara A"|14440
                  > > N00001955|"Cardin, Ben"|15408
                  > > N00001979|"Sarbanes, Paul S"|13039
                  > > N00002061|"Warner, John W"|14712
                  > > N00002073|"Wolf, Frank R"|14869
                  > > N00002091|"Craig, Larry"|14809
                  > > N00002171|"Boucher, Rick"|15010
                  > > N00002198|"Rahall, Nick"|14448
                  > > N00002200|"Byrd, Robert C"|1366
                  > > N00002214|"Mollohan, Alan B"|15083
                  > > N00002247|"Coble, Howard"|15092
                  > > N00002260|"Price, David"|15438
                  > > N00002377|"Ballenger, Cass"|15402
                  > > N00002423|"Hollings, Fritz"|11204
                  > > N00002492|"Spratt, John M Jr"|15064
                  > > N00002577|"Lewis, John"|15431
                  > > N00002742|"Graham, Bob"|15503
                  > > N00002782|"Stearns, Cliff"|15627
                  > > N00002858|"Ros-Lehtinen, Ileana"|15634
                  > > N00002877|"Shaw, E Clay Jr"|14860
                  > > N00002982|"Bilirakis, Michael"|15006
                  > > N00003126|"Gordon, Bart"|15100
                  > > N00003132|"Cooper, Jim"|15019
                  > > N00003209|"Duncan, John J Jr"|15455
                  > > N00003254|"Tanner, John"|15628
                  > > N00003328|"Cochran, Thad"|14009
                  > > N00003329|"Lott, Trent"|14031
                  > > N00003350|"Taylor, Gene"|15637
                  > > N00003389|"McConnell, Mitch"|14921
                  > > N00003437|"Bunning, Jim"|15406
                  > > N00003473|"Rogers, Hal"|14854
                  > > N00003522|"Kaptur, Marcy"|15029
                  > > N00003651|"Regula, Ralph"|14045
                  > > N00003660|"Gillmor, Paul E"|15604
                  > > N00003709|"DeWine, Mike"|15020
                  > > N00003736|"Oxley, Michael G"|14875
                  > > N00003813|"Visclosky, Pete"|15124
                  > > N00003950|"Levin, Sander"|15033
                  > > N00004029|"Conyers, John Jr"|10713
                  > > N00004070|"Kildee, Dale E"|14430
                  > > N00004133|"Upton, Fred"|15446
                  > > N00004207|"Harkin, Tom"|14230
                  > > N00004280|"Leach, Jim"|14432
                  > > N00004291|"Sensenbrenner, F James Jr"|14657
                  > > N00004309|"Kohl, Herb"|15703
                  > > N00004330|"Kleczka, Jerry"|15082
                  > > N00004394|"Obey, David R"|12036
                  > > N00004426|"Petri, Tom"|14675
                  > > N00004489|"Sabo, Martin Olav"|14656
                  > > N00004583|"Daschle, Tom"|14617
                  > > N00004613|"Conrad, Kent"|15502
                  > > N00004615|"Dorgan, Byron L"|14812
                  > > N00004638|"Burns, Conrad"|15701
                  > > N00004643|"Baucus, Max"|14203
                  > > N00004698|"Crane, Phil"|12041
                  > > N00004702|"Hyde, Henry J"|14239
                  > > N00004781|"Hastert, Dennis"|15417
                  > > N00004856|"Lipinski, Bill"|15036
                  > > N00004912|"Evans, Lane"|15023
                  > > N00004956|"Costello, Jerry F"|15453
                  > > N00004981|"Durbin, Dick"|15021
                  > > N00005037|"Gephardt, Richard A"|14421
                  > > N00005105|"Skelton, Ike"|14451
                  > > N00005178|"Bond, Christopher S 'Kit'"|15501
                  > > N00005285|"Roberts, Pat"|14852
                  > > N00005331|"Bereuter, Doug"|14605
                  > > N00005372|"Tauzin, Billy"|14679
                  > > N00005385|"Breaux, John"|13056
                  > > N00005407|"Baker, Richard"|15401
                  > > N00005414|"McCrery, Jim"|15451
                  > > N00005582|"Inhofe, James M"|15424
                  > > N00005617|"Nickles, Don"|14908
                  > > N00005645|"Hall, Ralph M"|14828
                  > > N00005656|"Barton, Joe"|15085
                  > > N00005677|"Frost, Martin"|14626
                  > > N00005892|"DeLay, Tom"|15094
                  > > N00005906|"Paul, Ron"|14290
                  > > N00005998|"Ortiz, Solomon P"|15049
                  > > N00006060|"Stenholm, Charles W"|14664
                  > > N00006202|"Campbell, Ben Nighthorse"|15407
                  > > N00006237|"Cheney, Dick"|14611
                  > > N00006246|"Thomas, Craig"|15633
                  > > N00006406|"Kyl, Jon"|15429
                  > > N00006424|"McCain, John"|15039
                  > > N00006486|"Kolbe, Jim"|15105
                  > > N00006515|"Domenici, Pete V"|14103
                  > > N00006518|"Bingaman, Jeff"|14912
                  > > N00006692|"Boxer, Barbara"|15011
                  > > N00006932|"Dreier, David"|14813
                  > > N00006983|"Hunter, Duncan"|14835
                  > > N00007087|"Lewis, Jerry"|14644
                  > > N00007124|"Cox, Christopher"|15601
                  > > N00007151|"Rohrabacher, Dana"|15621
                  > > N00007231|"Gallegly, Elton"|15413
                  > > N00007360|"Pelosi, Nancy"|15448
                  > > N00007382|"Lantos, Tom"|14837
                  > > N00007390|"Miller, George"|14256
                  > > N00007397|"Stark, Pete"|14053
                  > > N00007584|"Herger, Wally"|15420
                  > > N00007653|"Akaka, Daniel K"|14400
                  > > N00007665|"Abercrombie, Neil"|15245
                  > > N00007724|"Wyden, Ron"|14871
                  > > N00007781|"DeFazio, Peter"|15410
                  > > N00007918|"Dicks, Norm"|14413
                  > > N00007997|"Stevens, Ted"|12109
                  > > N00007999|"Young, Don"|14066
                  > > N00008094|"Berman, Howard L"|15005
                  > > N00009816|"Smith, Chris"|14863
                  > > N00009829|"McDermott, Jim"|15613
                  > > N00009869|"Hatch, Orrin G"|14503
                  > > N00009918|"Leahy, Patrick"|14307
                  > > N00009920|"Shelby, Richard C"|14659
                  > > N00009922|"Reid, Harry"|15054
                  > > N00009926|"Nelson, Bill"|14651
                  > > N00010084|"Johnson, Tim"|15425
                  > > N00011971|"Lungren, Dan"|14647
                  > > N00012508|"Carper, Tom"|15015
                  > > N99999981|"Rumsfeld, Donald H."|10622
                  > >
                  > >
                  > > --- In govtrack@yahoogroups.com, Derek Willis <dwillis@> wrote:
                  > > >
                  > > > In constructing committee membership data for the NYT Congress API, I
                  > > used
                  > > > data found via Charles Stewart's site:
                  > > >
                  > > > http://web.mit.edu/17.251/www/data_page.html#2
                  > > >
                  > > > As Josh says, older data contains errors in it - I recall incorrect party
                  > > > affiliations and phantom committee assignments when I checked them
                  > > against
                  > > > official sources. So much so that our API only vouches for 110-112th
                  > > > committee data...
                  > > >
                  > > > Derek
                  > > >
                  > > >
                  > > >
                  > > > On Tue, May 31, 2011 at 12:18 PM, Josh Tauberer <tauberer@>wrote:
                  > >
                  > > >
                  > > > >
                  > > > >
                  > > > > Hi, Peter.
                  > > > >
                  > > > > > 1.) What is the source for committees.xml? Is there a page like
                  > > > > people.xml's "Source Data" page for committees.xml on Govtrack?
                  > > > >
                  > > > > The committee data is automatically scraped from
                  > > > >
                  > > > >
                  > > > >
                  > > http://www.senate.gov/pagelayout/committees/b_three_sections_with_teasers/membership.htm
                  > > > >
                  > > > > and
                  > > > >
                  > > > > http://clerk.house.gov/committee_info/index.html
                  > > > >
                  > > > > (As I mentioned on the labs list, the file was last generated a few
                  > > > > months ago- March 3. I can re-generate it to get the latest info.)
                  > > > >
                  > > > > > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the
                  > > > > same data available for other years?
                  > > > >
                  > > > > No I wasn't collecting that info then, and archival data is not
                  > > > > available from the House/Senate (at least in the form I scrape).
                  > > > >
                  > > > > > 3.) The "Source Data" page says that people.xml "has been put
                  > > > > together from a variety of sources and is maintained by hand." What are
                  > > > > the sources?
                  > > > >
                  > > > > A full list of data sources is at govtrack.us/credits.xpd. That said,
                  > > > > the amount of data I used from each source and the quality of the
                  > > > > sources varied a lot. For information from about 2003 and on, the info
                  > > > > about Members of Congress has been entered by hand by me.
                  > > > >
                  > > > > As you go further back in time, the quality of party affiliations,
                  > > > > district assignments, and links to other IDs (e.g. ICPSR) grows worse.
                  > > > > But it's probably the best anyone has anyway.
                  > > > >
                  > > > > Good luck. If you manage to put together a database of additional
                  > > > > information, I hope you'll share it.
                  > > > >
                  > > > > - Josh Tauberer
                  > > > > - CivicImpulse
                  > > > >
                  > > > > http://razor.occams.info
                  > > > > http://www.civicimpulse.com
                  > > > >
                  > > > > "Yields falsehood when preceded by its quotation! Yields
                  > > > > falsehood when preceded by its quotation!" Achilles to
                  > > > > Tortoise (in "Godel, Escher, Bach" by Douglas Hofstadter)
                  > > > >
                  > > > > On 05/31/2011 11:29 AM, codekiln wrote:
                  > > > > > I found out about the govtrack database from my post [1] over at the
                  > > > > google group for Sunlight Labs' API. As I mentioned there, I have a
                  > > list of
                  > > > > CRP ID / bioguide id / name triples. I would like to
                  > > > > > assemble information on what committees each legislator was a member
                  > > > > > of, and any special titles they held in that committee, such as
                  > > > > > chairman or ranking member. Mr. Tauberer suggested I use govtrack's
                  > > > > people.xml [2] and committees.xml [3].
                  > > > > >
                  > > > > > I am helping a professor assemble this data, so it is important for
                  > > me to
                  > > > > be able to explain the paper trail of the data. I have seen Govtrack's
                  > > > > "Source Data" page [4], but I still have some questions.
                  > > > > >
                  > > > > > 1.) What is the source for committees.xml? Is there a page like
                  > > > > people.xml's "Source Data" page for committees.xml on Govtrack?
                  > > > > >
                  > > > > > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the
                  > > same
                  > > > > data available for other years?
                  > > > > >
                  > > > > > 3.) The "Source Data" page says that people.xml "has been put
                  > > together
                  > > > > from a variety of sources and is maintained by hand." What are the
                  > > sources?
                  > > > > >
                  > > > > > Thanks,
                  > > > > > Peter
                  > > > > >
                  > > > > > [1]
                  > > > >
                  > > http://groups.google.com/group/sunlightlabs/browse_thread/thread/7f35405ebf184e0e
                  > > > > >
                  > > > > > [2] http://www.govtrack.us/data/us/people.xml
                  > > > > >
                  > > > > > [3] http://www.govtrack.us/data/us/112/committees.xml
                  > > > > >
                  > > > > > [4] http://www.govtrack.us/developers/data.xpd
                  > > > > >
                  > > > > >
                  > > > > >
                  > > > > > ------------------------------------
                  > > > > >
                  > > > > > Yahoo! Groups Links
                  > > > > >
                  > > > > >
                  > > > > >
                  > > > >
                  > > > >
                  > > >
                  > >
                  > >
                  > >
                  >


                • Derek Willis
                  Just as an FYI, we ll probably make this official next week but the ICPSR IDs that we have are now in the Member response of the NYT Congress API.
                  Message 8 of 8 , Jun 3, 2011
                    Just as an FYI, we'll probably make this official next week but the ICPSR IDs that we have are now in the Member response of the NYT Congress API. 


                    On Thu, Jun 2, 2011 at 11:49 AM, codekiln <ptr.nore@...> wrote:
                     



                    Hi Darek,
                    Thanks for your detailed response.

                    Your record of ICPSRIDs would be a valuable to the project I am working on, but I have no idea if ICPSRIDs belong in the NYT Congress API. Netizens definitely have a need some kind of database id Rosetta Stone. Bioguide ID - ICPSRID isn't the only important connection. When I have assembled all the different corresponding ids I will ask the professor if I may publish the cross-references I have found.

                    Thanks,
                    Peter



                    --- In govtrack@yahoogroups.com, Derek Willis <dwillis@...> wrote:
                    >
                    > Peter,
                    >
                    > We have ICPSRIDs for more than 2,600 members - if you think that would be a
                    > valuable addition to the API, I'm happy to add that to the members response.
                    > The committee names are tough because they can change from congress to
                    > congress. So what was the House Banking Committee is now the House Financial
                    > Services Committee. Depending on which party is in the majority, the House
                    > Education and Labor Committee becomes the House Education and the Workforce
                    > Committee, and so on. For my money, the official source is Congress itself,
                    > which means looking up committee assignments in the Record, among other
                    > methods. I haven't found phantom committees in Stewart's data, but rather
                    > phantom assignments in which a member is recorded as a member of a committee
                    > when official records do not reflect such an assignment.
                    >
                    > Derek
                    >
                    >
                    >
                    > On Tue, May 31, 2011 at 1:30 PM, codekiln <ptr.nore@...> wrote:
                    >
                    > >
                    > >
                    > >
                    > >
                    > > Darek and Josh,
                    > > Thanks for the citations and the info. Using people.xml I was able to
                    > > extract ICPSRID for 165 out of the 793 identities I am researching; I've put
                    > > the results below for google [2].
                    > >
                    > > It seems as though the four or five data sources I have located are not
                    > > always consistent on the committee names. Darek, what official sources did
                    > > you use to double check Stewart's data? I had actually wondered whether
                    > > there were phantom committees in Stewart's data because the committee name
                    > > capitalization is inconsistent and the assignments lack subcommittees.
                    > >
                    > > Thanks for tipping me off to your work at NYT Congress API; I noticed the
                    > > "data for earlier Congresses will be available soon" note in the committee
                    > > call documentation [1].
                    > >
                    > > What a lively digital community there is around politician data; it seems
                    > > like every few months a new API appears online.
                    > >
                    > > Thanks,
                    > > Peter
                    > >
                    > > [1] http://developer.nytimes.com/docs/congress_api#h3-committees
                    > > [2]
                    > > OSID|CRPNAME|govtrack_icpsrid
                    > > N00000010|"Burton, Dan"|15014
                    > > N00000048|"Hefley, Joel"|15419
                    > > N00000153|"Neal, Richard E"|15616
                    > > N00000245|"Kerry, John"|14920
                    > > N00000270|"Markey, Edward J"|14435
                    > > N00000275|"Frank, Barney"|14824
                    > > N00000308|"Kennedy, Edward M"|10808
                    > > N00000444|"Gregg, Judd"|14826
                    > > N00000480|"Snowe, Olympia J"|14661
                    > > N00000534|"Jeffords, James M"|14240
                    > > N00000561|"Johnson, Nancy L"|15028
                    > > N00000581|"Dodd, Christopher J"|14213
                    > > N00000616|"Lieberman, Joe"|15704
                    > > N00000652|"Shays, Christopher"|15449
                    > > N00000659|"Lautenberg, Frank R"|14914
                    > > N00000716|"Payne, Donald M"|15619
                    > > N00000781|"Pallone, Frank Jr"|15454
                    > > N00000834|"Saxton, Jim"|15112
                    > > N00000964|"Rangel, Charles B"|13035
                    > > N00001003|"Engel, Eliot L"|15603
                    > > N00001024|"Lowey, Nita M"|15612
                    > > N00001082|"Towns, Edolphus"|15072
                    > > N00001093|"Schumer, Charles E"|14858
                    > > N00001143|"Ackerman, Gary"|15000
                    > > N00001214|"McNulty, Michael R"|15614
                    > > N00001261|"Walsh, James T"|15630
                    > > N00001267|"Boehlert, Sherwood"|15007
                    > > N00001311|"Slaughter, Louise M"|15444
                    > > N00001329|"Houghton, Amo"|15423
                    > > N00001408|"Murtha, John P"|14072
                    > > N00001509|"Kanjorski, Paul E"|15104
                    > > N00001535|"Weldon, Curt"|15447
                    > > N00001604|"Specter, Arlen"|14910
                    > > N00001669|"Biden, Joseph R Jr"|14101
                    > > N00001685|"Rockefeller, Jay"|14922
                    > > N00001691|"Levin, Carl"|14709
                    > > N00001701|"Mineta, Norman Y."|14257
                    > > N00001758|"Grassley, Chuck"|14226
                    > > N00001762|"Inouye, Daniel K"|4812
                    > > N00001764|"Lugar, Richard G"|14506
                    > > N00001783|"Dingell, John D"|2605
                    > > N00001806|"Oberstar, James L"|14265
                    > > N00001811|"Smith, Lamar"|15445
                    > > N00001813|"Serrano, Jose E"|29134
                    > > N00001817|"Young, C W Bill"|13047
                    > > N00001821|"Hoyer, Steny H"|14873
                    > > N00001861|"Waxman, Henry A"|14280
                    > > N00001945|"Mikulski, Barbara A"|14440
                    > > N00001955|"Cardin, Ben"|15408
                    > > N00001979|"Sarbanes, Paul S"|13039
                    > > N00002061|"Warner, John W"|14712
                    > > N00002073|"Wolf, Frank R"|14869
                    > > N00002091|"Craig, Larry"|14809
                    > > N00002171|"Boucher, Rick"|15010
                    > > N00002198|"Rahall, Nick"|14448
                    > > N00002200|"Byrd, Robert C"|1366
                    > > N00002214|"Mollohan, Alan B"|15083
                    > > N00002247|"Coble, Howard"|15092
                    > > N00002260|"Price, David"|15438
                    > > N00002377|"Ballenger, Cass"|15402
                    > > N00002423|"Hollings, Fritz"|11204
                    > > N00002492|"Spratt, John M Jr"|15064
                    > > N00002577|"Lewis, John"|15431
                    > > N00002742|"Graham, Bob"|15503
                    > > N00002782|"Stearns, Cliff"|15627
                    > > N00002858|"Ros-Lehtinen, Ileana"|15634
                    > > N00002877|"Shaw, E Clay Jr"|14860
                    > > N00002982|"Bilirakis, Michael"|15006
                    > > N00003126|"Gordon, Bart"|15100
                    > > N00003132|"Cooper, Jim"|15019
                    > > N00003209|"Duncan, John J Jr"|15455
                    > > N00003254|"Tanner, John"|15628
                    > > N00003328|"Cochran, Thad"|14009
                    > > N00003329|"Lott, Trent"|14031
                    > > N00003350|"Taylor, Gene"|15637
                    > > N00003389|"McConnell, Mitch"|14921
                    > > N00003437|"Bunning, Jim"|15406
                    > > N00003473|"Rogers, Hal"|14854
                    > > N00003522|"Kaptur, Marcy"|15029
                    > > N00003651|"Regula, Ralph"|14045
                    > > N00003660|"Gillmor, Paul E"|15604
                    > > N00003709|"DeWine, Mike"|15020
                    > > N00003736|"Oxley, Michael G"|14875
                    > > N00003813|"Visclosky, Pete"|15124
                    > > N00003950|"Levin, Sander"|15033
                    > > N00004029|"Conyers, John Jr"|10713
                    > > N00004070|"Kildee, Dale E"|14430
                    > > N00004133|"Upton, Fred"|15446
                    > > N00004207|"Harkin, Tom"|14230
                    > > N00004280|"Leach, Jim"|14432
                    > > N00004291|"Sensenbrenner, F James Jr"|14657
                    > > N00004309|"Kohl, Herb"|15703
                    > > N00004330|"Kleczka, Jerry"|15082
                    > > N00004394|"Obey, David R"|12036
                    > > N00004426|"Petri, Tom"|14675
                    > > N00004489|"Sabo, Martin Olav"|14656
                    > > N00004583|"Daschle, Tom"|14617
                    > > N00004613|"Conrad, Kent"|15502
                    > > N00004615|"Dorgan, Byron L"|14812
                    > > N00004638|"Burns, Conrad"|15701
                    > > N00004643|"Baucus, Max"|14203
                    > > N00004698|"Crane, Phil"|12041
                    > > N00004702|"Hyde, Henry J"|14239
                    > > N00004781|"Hastert, Dennis"|15417
                    > > N00004856|"Lipinski, Bill"|15036
                    > > N00004912|"Evans, Lane"|15023
                    > > N00004956|"Costello, Jerry F"|15453
                    > > N00004981|"Durbin, Dick"|15021
                    > > N00005037|"Gephardt, Richard A"|14421
                    > > N00005105|"Skelton, Ike"|14451
                    > > N00005178|"Bond, Christopher S 'Kit'"|15501
                    > > N00005285|"Roberts, Pat"|14852
                    > > N00005331|"Bereuter, Doug"|14605
                    > > N00005372|"Tauzin, Billy"|14679
                    > > N00005385|"Breaux, John"|13056
                    > > N00005407|"Baker, Richard"|15401
                    > > N00005414|"McCrery, Jim"|15451
                    > > N00005582|"Inhofe, James M"|15424
                    > > N00005617|"Nickles, Don"|14908
                    > > N00005645|"Hall, Ralph M"|14828
                    > > N00005656|"Barton, Joe"|15085
                    > > N00005677|"Frost, Martin"|14626
                    > > N00005892|"DeLay, Tom"|15094
                    > > N00005906|"Paul, Ron"|14290
                    > > N00005998|"Ortiz, Solomon P"|15049
                    > > N00006060|"Stenholm, Charles W"|14664
                    > > N00006202|"Campbell, Ben Nighthorse"|15407
                    > > N00006237|"Cheney, Dick"|14611
                    > > N00006246|"Thomas, Craig"|15633
                    > > N00006406|"Kyl, Jon"|15429
                    > > N00006424|"McCain, John"|15039
                    > > N00006486|"Kolbe, Jim"|15105
                    > > N00006515|"Domenici, Pete V"|14103
                    > > N00006518|"Bingaman, Jeff"|14912
                    > > N00006692|"Boxer, Barbara"|15011
                    > > N00006932|"Dreier, David"|14813
                    > > N00006983|"Hunter, Duncan"|14835
                    > > N00007087|"Lewis, Jerry"|14644
                    > > N00007124|"Cox, Christopher"|15601
                    > > N00007151|"Rohrabacher, Dana"|15621
                    > > N00007231|"Gallegly, Elton"|15413
                    > > N00007360|"Pelosi, Nancy"|15448
                    > > N00007382|"Lantos, Tom"|14837
                    > > N00007390|"Miller, George"|14256
                    > > N00007397|"Stark, Pete"|14053
                    > > N00007584|"Herger, Wally"|15420
                    > > N00007653|"Akaka, Daniel K"|14400
                    > > N00007665|"Abercrombie, Neil"|15245
                    > > N00007724|"Wyden, Ron"|14871
                    > > N00007781|"DeFazio, Peter"|15410
                    > > N00007918|"Dicks, Norm"|14413
                    > > N00007997|"Stevens, Ted"|12109
                    > > N00007999|"Young, Don"|14066
                    > > N00008094|"Berman, Howard L"|15005
                    > > N00009816|"Smith, Chris"|14863
                    > > N00009829|"McDermott, Jim"|15613
                    > > N00009869|"Hatch, Orrin G"|14503
                    > > N00009918|"Leahy, Patrick"|14307
                    > > N00009920|"Shelby, Richard C"|14659
                    > > N00009922|"Reid, Harry"|15054
                    > > N00009926|"Nelson, Bill"|14651
                    > > N00010084|"Johnson, Tim"|15425
                    > > N00011971|"Lungren, Dan"|14647
                    > > N00012508|"Carper, Tom"|15015
                    > > N99999981|"Rumsfeld, Donald H."|10622
                    > >
                    > >
                    > > --- In govtrack@yahoogroups.com, Derek Willis <dwillis@> wrote:
                    > > >
                    > > > In constructing committee membership data for the NYT Congress API, I
                    > > used
                    > > > data found via Charles Stewart's site:
                    > > >
                    > > > http://web.mit.edu/17.251/www/data_page.html#2
                    > > >
                    > > > As Josh says, older data contains errors in it - I recall incorrect party
                    > > > affiliations and phantom committee assignments when I checked them
                    > > against
                    > > > official sources. So much so that our API only vouches for 110-112th
                    > > > committee data...
                    > > >
                    > > > Derek
                    > > >
                    > > >
                    > > >
                    > > > On Tue, May 31, 2011 at 12:18 PM, Josh Tauberer <tauberer@>wrote:
                    > >
                    > > >
                    > > > >
                    > > > >
                    > > > > Hi, Peter.
                    > > > >
                    > > > > > 1.) What is the source for committees.xml? Is there a page like
                    > > > > people.xml's "Source Data" page for committees.xml on Govtrack?
                    > > > >
                    > > > > The committee data is automatically scraped from
                    > > > >
                    > > > >
                    > > > >
                    > > http://www.senate.gov/pagelayout/committees/b_three_sections_with_teasers/membership.htm
                    > > > >
                    > > > > and
                    > > > >
                    > > > > http://clerk.house.gov/committee_info/index.html
                    > > > >
                    > > > > (As I mentioned on the labs list, the file was last generated a few
                    > > > > months ago- March 3. I can re-generate it to get the latest info.)
                    > > > >
                    > > > > > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the
                    > > > > same data available for other years?
                    > > > >
                    > > > > No I wasn't collecting that info then, and archival data is not
                    > > > > available from the House/Senate (at least in the form I scrape).
                    > > > >
                    > > > > > 3.) The "Source Data" page says that people.xml "has been put
                    > > > > together from a variety of sources and is maintained by hand." What are
                    > > > > the sources?
                    > > > >
                    > > > > A full list of data sources is at govtrack.us/credits.xpd. That said,
                    > > > > the amount of data I used from each source and the quality of the
                    > > > > sources varied a lot. For information from about 2003 and on, the info
                    > > > > about Members of Congress has been entered by hand by me.
                    > > > >
                    > > > > As you go further back in time, the quality of party affiliations,
                    > > > > district assignments, and links to other IDs (e.g. ICPSR) grows worse.
                    > > > > But it's probably the best anyone has anyway.
                    > > > >
                    > > > > Good luck. If you manage to put together a database of additional
                    > > > > information, I hope you'll share it.
                    > > > >
                    > > > > - Josh Tauberer
                    > > > > - CivicImpulse
                    > > > >
                    > > > > http://razor.occams.info
                    > > > > http://www.civicimpulse.com
                    > > > >
                    > > > > "Yields falsehood when preceded by its quotation! Yields
                    > > > > falsehood when preceded by its quotation!" Achilles to
                    > > > > Tortoise (in "Godel, Escher, Bach" by Douglas Hofstadter)
                    > > > >
                    > > > > On 05/31/2011 11:29 AM, codekiln wrote:
                    > > > > > I found out about the govtrack database from my post [1] over at the
                    > > > > google group for Sunlight Labs' API. As I mentioned there, I have a
                    > > list of
                    > > > > CRP ID / bioguide id / name triples. I would like to
                    > > > > > assemble information on what committees each legislator was a member
                    > > > > > of, and any special titles they held in that committee, such as
                    > > > > > chairman or ranking member. Mr. Tauberer suggested I use govtrack's
                    > > > > people.xml [2] and committees.xml [3].
                    > > > > >
                    > > > > > I am helping a professor assemble this data, so it is important for
                    > > me to
                    > > > > be able to explain the paper trail of the data. I have seen Govtrack's
                    > > > > "Source Data" page [4], but I still have some questions.
                    > > > > >
                    > > > > > 1.) What is the source for committees.xml? Is there a page like
                    > > > > people.xml's "Source Data" page for committees.xml on Govtrack?
                    > > > > >
                    > > > > > 2.) I need data for 103rd to 110th congresses (2003-2009). Is the
                    > > same
                    > > > > data available for other years?
                    > > > > >
                    > > > > > 3.) The "Source Data" page says that people.xml "has been put
                    > > together
                    > > > > from a variety of sources and is maintained by hand." What are the
                    > > sources?
                    > > > > >
                    > > > > > Thanks,
                    > > > > > Peter
                    > > > > >
                    > > > > > [1]
                    > > > >
                    > > http://groups.google.com/group/sunlightlabs/browse_thread/thread/7f35405ebf184e0e
                    > > > > >
                    > > > > > [2] http://www.govtrack.us/data/us/people.xml
                    > > > > >
                    > > > > > [3] http://www.govtrack.us/data/us/112/committees.xml
                    > > > > >
                    > > > > > [4] http://www.govtrack.us/developers/data.xpd
                    > > > > >
                    > > > > >
                    > > > > >
                    > > > > > ------------------------------------
                    > > > > >
                    > > > > > Yahoo! Groups Links
                    > > > > >
                    > > > > >
                    > > > > >
                    > > > >
                    > > > >
                    > > >
                    > >
                    > >
                    > >
                    >


                  Your message has been successfully submitted and would be delivered to recipients shortly.