Loading ...
Sorry, an error occurred while loading the content.

Re: [govtrack] Re: Where to get BioguideIDs for People in House.

Expand Messages
  • Josh Tauberer
    ... I now update it by hand. It was initially generated from a few data sources. -- - Josh Tauberer - GovTrack.us http://razor.occams.info Yields falsehood
    Message 1 of 15 , Nov 6, 2008
    • 0 Attachment
      Narasimha Lakkakula & Sirisha Pogadadanda wrote:
      > Josh... could you tell us how do you fetch and keep people.xml update
      > to date...

      I now update it by hand.

      It was initially generated from a few data sources.

      --
      - Josh Tauberer
      - GovTrack.us

      http://razor.occams.info

      "Yields falsehood when preceded by its quotation! Yields
      falsehood when preceded by its quotation!" Achilles to
      Tortoise (in "Godel, Escher, Bach" by Douglas Hofstadter)
    • scottyrockit
      ... update ... So I m back in the saddle again and things will be different when I m in San Diego.
      Message 2 of 15 , Nov 6, 2008
      • 0 Attachment
        --- In govtrack@yahoogroups.com, "Neil Drumm" <drumm@...> wrote:
        >
        > If all you need is the 110th, look under the Download All Data heading
        > at http://services.sunlightlabs.com/api/.
        >
        > -Neil
        >
        > On Thu, Nov 6, 2008 at 5:25 PM, Narasimha Lakkakula & Sirisha
        > Pogadadanda nlakkakula@... wrote:
        > > Thanks Scott... But I want BioguideID though..... List of
        > > representatives I could get in different formats at
        > > http://clerk.house.gov/member_info/index.html
        > >
        > > The problem is it doesnt have BioguideID field....
        > >
        > > Josh... could you tell us how do you fetch and keep people.xml
        update
        > > to date...
        > >
        > > thanks
        > >
        > >
        >
        >
        >
        > --
        > Neil Drumm
        > http://delocalizedham.com
        >
        So I'm back in the saddle again and things will be different when I'm in
        San Diego.
      • Narasimha Lakkakula & Sirisha Pogadadanda
        I see... I think this serves my purpose for now.... So it seems there is NO Official XML data available either from house or GPO which gives list of Names
        Message 3 of 15 , Nov 7, 2008
        • 0 Attachment
          I see... I think this serves my purpose for now.... So it seems
          there is NO Official XML data available either from house or GPO which
          gives list of Names with BioguideIDs in a regular intervals......


          --- In govtrack@yahoogroups.com, "Neil Drumm" <drumm@...> wrote:
          >
          > If all you need is the 110th, look under the Download All Data heading
          > at http://services.sunlightlabs.com/api/.
          >
          > -Neil
        • Derek Willis
          This isn t completely accurate - the House roll call XML files have the bioguide IDs in them. Do a view source on this page, for example:
          Message 4 of 15 , Nov 7, 2008
          • 0 Attachment
            This isn't completely accurate - the House roll call XML files have
            the bioguide IDs in them. Do a view source on this page, for example:

            http://clerk.house.gov/evs/2008/roll683.xml

            But there is no single list of current members that gets updated
            regularly that includes the bioguide IDs.

            Derek Willis

            On Fri, Nov 7, 2008 at 10:48 AM, Narasimha Lakkakula & Sirisha
            Pogadadanda <nlakkakula@...> wrote:
            > I see... I think this serves my purpose for now.... So it seems
            > there is NO Official XML data available either from house or GPO which
            > gives list of Names with BioguideIDs in a regular intervals......
            >
            > --- In govtrack@yahoogroups.com, "Neil Drumm" <drumm@...> wrote:
            >>
            >> If all you need is the 110th, look under the Download All Data heading
            >> at http://services.sunlightlabs.com/api/.
            >>
            >> -Neil
            >
            >



            --
            --
            Derek Willis
            dwillis@...
            http://www.thescoop.org/docs/
          • Joe Carmel
            A generalized approach for this purpose is to use the http://bioguide.congress.gov website to generate a well-formed HTML file containing your specified
            Message 5 of 15 , Nov 13, 2008
            • 0 Attachment
              A generalized approach for this purpose is to use the http://bioguide.congress.gov website to generate a well-formed HTML file containing your specified criteria and then using that HTML file which will contain the Bioguide IDs as hrefs. 
               
              Joe


              --- On Fri, 11/7/08, Narasimha Lakkakula & Sirisha Pogadadanda <nlakkakula@...> wrote:
              From: Narasimha Lakkakula & Sirisha Pogadadanda <nlakkakula@...>
              Subject: [govtrack] Re: Where to get BioguideIDs for People in House.
              To: govtrack@yahoogroups.com
              Date: Friday, November 7, 2008, 9:48 AM

              I see... I think this serves my purpose for now.... So it seems
              there is NO Official XML data available either from house or GPO which
              gives list of Names with BioguideIDs in a regular intervals... ...

              --- In govtrack@yahoogroup s.com, "Neil Drumm" <drumm@...> wrote:
              >
              > If all you need is the 110th, look under the Download All Data heading
              > at http://services. sunlightlabs. com/api/.
              >
              > -Neil


            • Justin Palmer
              You can also get this information from Sunlight s API which I assume will be kept up to date when the transition happens.
              Message 6 of 15 , Nov 14, 2008
              • 0 Attachment
                You can also get this information from Sunlight's API which I assume will be kept up to date when the transition happens.


                Justin Palmer
                http://alternateidea.com | http://lighthouseapp.com

                On Thu, Nov 13, 2008 at 3:44 PM, Joe Carmel <joe.carmel@...> wrote:

                A generalized approach for this purpose is to use the http://bioguide.congress.gov website to generate a well-formed HTML file containing your specified criteria and then using that HTML file which will contain the Bioguide IDs as hrefs. 
                 
                Joe


                --- On Fri, 11/7/08, Narasimha Lakkakula & Sirisha Pogadadanda <nlakkakula@...> wrote:
                From: Narasimha Lakkakula & Sirisha Pogadadanda <nlakkakula@...>
                Subject: [govtrack] Re: Where to get BioguideIDs for People in House.
                To: govtrack@yahoogroups.com
                Date: Friday, November 7, 2008, 9:48 AM

                I see... I think this serves my purpose for now.... So it seems
                there is NO Official XML data available either from house or GPO which
                gives list of Names with BioguideIDs in a regular intervals... ...

                --- In govtrack@yahoogroup s.com, "Neil Drumm" <drumm@...> wrote:
                >
                > If all you need is the 110th, look under the Download All Data heading
                > at http://services. sunlightlabs. com/api/.
                >
                > -Neil



              • scottyrockit
                You are a brother for life ! I reserve a place for you beyond your wildest dreams with my family and the big guy. Your name in the book of life ! I can feel
                Message 7 of 15 , Nov 14, 2008
                • 0 Attachment
                  You are a brother for life ! I reserve a place for you beyond your
                  wildest dreams with my family and the big guy. Your name in the book
                  of life ! I can feel you
                  ---
                  In govtrack@yahoogroups.com, "Justin Palmer" <encytemedia@...> wrote:
                  >
                  > You can also get this information from Sunlight's API which I
                  assume will be
                  > kept up to date when the transition happens.
                  > http://services.sunlightlabs.com/api/docs/legislators/
                  >
                  > Justin Palmer
                  > http://alternateidea.com | http://lighthouseapp.com
                  >
                  > On Thu, Nov 13, 2008 at 3:44 PM, Joe Carmel <joe.carmel@...> wrote:
                  >
                  > > A generalized approach for this purpose is to use the
                  > > http://bioguide.congress.gov website to generate a well-formed
                  HTML file
                  > > containing your specified criteria and then using that HTML file
                  which will
                  > > contain the Bioguide IDs as hrefs.
                  > >
                  > > Joe
                  > >
                  > >
                  > > --- On *Fri, 11/7/08, Narasimha Lakkakula & Sirisha Pogadadanda <
                  > > nlakkakula@...>* wrote:
                  > >
                  > > From: Narasimha Lakkakula & Sirisha Pogadadanda <nlakkakula@...>
                  > > Subject: [govtrack] Re: Where to get BioguideIDs for People in
                  House.
                  > > To: govtrack@yahoogroups.com
                  > > Date: Friday, November 7, 2008, 9:48 AM
                  > >
                  > > I see... I think this serves my purpose for now.... So it seems
                  > > there is NO Official XML data available either from house or GPO
                  which
                  > > gives list of Names with BioguideIDs in a regular intervals... ...
                  > >
                  > > --- In govtrack@yahoogroup s.com <govtrack%
                  40yahoogroups.com>, "Neil
                  > > Drumm" <drumm@> wrote:
                  > > >
                  > > > If all you need is the 110th, look under the Download All Data
                  heading
                  > > > at http://services. sunlightlabs.
                  com/api/.<http://services.sunlightlabs.com/api/>
                  > > >
                  > > > -Neil
                  > >
                  > >
                  > >
                  > >
                  >
                • scottyrockit
                  You are a brother for life ! I reserve a place for you beyond your wildest dreams with my family and the big guy. Your name in the book of life ! I can feel
                  Message 8 of 15 , Nov 14, 2008
                  • 0 Attachment
                    You are a brother for life ! I reserve a place for you beyond your
                    wildest dreams with my family and the big guy. Your name in the book
                    of life ! I can feel you
                    ---
                    In govtrack@yahoogroups.com, "Justin Palmer" <encytemedia@...> wrote:
                    >
                    > You can also get this information from Sunlight's API which I
                    assume will be
                    > kept up to date when the transition happens.
                    > http://services.sunlightlabs.com/api/docs/legislators/
                    >
                    > Justin Palmer
                    > http://alternateidea.com | http://lighthouseapp.com
                    >
                    > On Thu, Nov 13, 2008 at 3:44 PM, Joe Carmel <joe.carmel@...> wrote:
                    >
                    > > A generalized approach for this purpose is to use the
                    > > http://bioguide.congress.gov website to generate a well-formed
                    HTML file
                    > > containing your specified criteria and then using that HTML file
                    which will
                    > > contain the Bioguide IDs as hrefs.
                    > >
                    > > Joe
                    > >
                    > >
                    > > --- On *Fri, 11/7/08, Narasimha Lakkakula & Sirisha Pogadadanda <
                    > > nlakkakula@...>* wrote:
                    > >
                    > > From: Narasimha Lakkakula & Sirisha Pogadadanda <nlakkakula@...>
                    > > Subject: [govtrack] Re: Where to get BioguideIDs for People in
                    House.
                    > > To: govtrack@yahoogroups.com
                    > > Date: Friday, November 7, 2008, 9:48 AM
                    > >
                    > > I see... I think this serves my purpose for now.... So it seems
                    > > there is NO Official XML data available either from house or GPO
                    which
                    > > gives list of Names with BioguideIDs in a regular intervals... ...
                    > >
                    > > --- In govtrack@yahoogroup s.com <govtrack%
                    40yahoogroups.com>, "Neil
                    > > Drumm" <drumm@> wrote:
                    > > >
                    > > > If all you need is the 110th, look under the Download All Data
                    heading
                    > > > at http://services. sunlightlabs.
                    com/api/.<http://services.sunlightlabs.com/api/>
                    > > >
                    > > > -Neil
                    > >
                    > >
                    > >
                    > >
                    >
                  • Josh Tauberer
                    ... Ooops, I should have remembered to suggest that. I actually have a parser for that site. Code here:
                    Message 9 of 15 , Nov 15, 2008
                    • 0 Attachment
                      Joe Carmel wrote:
                      > A generalized approach for this purpose is to use the
                      > http://bioguide.congress.gov website to generate a well-formed HTML file
                      > containing your specified criteria and then using that HTML file which
                      > will contain the Bioguide IDs as hrefs.

                      Ooops, I should have remembered to suggest that.

                      I actually have a parser for that site. Code here:
                      http://razor.occams.info/code/repo/?/govtrack/gather/us/bioguide.pl

                      The whole bioguide directory is here (just updated, now UTF-8):

                      http://www.govtrack.us/data/us/bioguide1.csv
                      CSV file with bioguide ID, name, birth year, death year

                      http://www.govtrack.us/data/us/bioguide2.csv
                      CSV file with bioguide ID, role (Senator, Representative, ContCong,
                      Delegate (I think that's it, not sure)), party, state, and congress number

                      And it's currently updating this:

                      http://www.govtrack.us/data/us/bioguide3.csv
                      CSV file with bioguide ID and the biographical text

                      which will probably be done in four hours (it rate limits one page per
                      second).

                      An interesting limitation of that data is that reps don't have district
                      numbers last I checked.

                      --
                      - Josh Tauberer
                      - GovTrack.us

                      http://razor.occams.info

                      "Yields falsehood when preceded by its quotation! Yields
                      falsehood when preceded by its quotation!" Achilles to
                      Tortoise (in "Godel, Escher, Bach" by Douglas Hofstadter)
                    • Derek Willis
                      ... There is one more role: Resident Commissioner (used for Puerto Rico s member of the House) ... Yeah, that s likely because district numbers can change over
                      Message 10 of 15 , Nov 16, 2008
                      • 0 Attachment
                        On Sat, Nov 15, 2008 at 7:33 PM, Josh Tauberer <tauberer@...> wrote:
                        > The whole bioguide directory is here (just updated, now UTF-8):
                        >
                        > http://www.govtrack.us/data/us/bioguide1.csv
                        > CSV file with bioguide ID, name, birth year, death year
                        >
                        > http://www.govtrack.us/data/us/bioguide2.csv
                        > CSV file with bioguide ID, role (Senator, Representative, ContCong,
                        > Delegate (I think that's it, not sure)), party, state, and congress number

                        There is one more role: Resident Commissioner (used for Puerto Rico's
                        member of the House)


                        > And it's currently updating this:
                        >
                        > http://www.govtrack.us/data/us/bioguide3.csv
                        > CSV file with bioguide ID and the biographical text
                        >
                        > which will probably be done in four hours (it rate limits one page per
                        > second).
                        >
                        > An interesting limitation of that data is that reps don't have district
                        > numbers last I checked.

                        Yeah, that's likely because district numbers can change over time with
                        redistricting, or they can run for other offices, so the bioguide
                        biotext plays it safe by just using the role. There are other sites
                        that provide information about specific districts and more. I like
                        Charles Stewart's page at MIT, but even that data has typos:

                        http://web.mit.edu/17.251/www/data_page.html

                        Derek
                        --
                        Derek Willis
                        dwillis@...
                        http://www.thescoop.org/docs/
                      Your message has been successfully submitted and would be delivered to recipients shortly.